The Starting Point: ESA WorldCover#
This project started with the ESA WorldCover dataset — a global land cover product at 10 m resolution based on Sentinel-1 and Sentinel-2 data. The full dataset consists of 2561 GeoTIFF files totaling 116 GB. I wanted to convert it into a single PMTiles archive for use in a web-based flight simulator.
PMTiles is a single-file tile archive format designed for the cloud. Instead of hosting millions of individual tile files or running a tile server, you store one .pmtiles file on any static hosting (e.g. S3). Clients fetch tiles via HTTP range requests. There is no server-side logic required, which makes deployment trivially simple.
I used the recommended tool rio-pmtiles, a Python-based tool built on top of Rasterio and GDAL. I let it run for over a week — and it never finished. Worse, the process consumed so much memory that I had to configure 512 GB of swap space just to keep it alive. Thus I got frustrated and I decided leverage Cursor to create a new tool based on Go.
GeoTiff2PMTiles#
Two weeks working on a& nd off the project and 100$ in Opus 4.6 tokens later I have geotiff2pmtiles that does the whole conversion in 16 hours. I already had a working solution after a couple of hours but I spent some time fleshing out the project. The project supports multiple projections, GeoTIFF source types and a slew of resampling methods.
Most of the time spent computing the ESA-Worldview dataset is spent on zoom level 13 - the highest avilable resolution. With a tile size of 512 px an impressive 45 million tiles are computed.
Here’s how long each layer takes to process:
| Zoom | Tiles | Throughput | Duration |
|---|---|---|---|
| 13 | 45,572,096 | 1,069/s | 710m 14s |
| 12 | 11,395,072 | 1,852/s | 102m 33s |
| 11 | 2,850,816 | 1,667/s | 28m 30s |
| 10 | 712,704 | 1,504/s | 7m 53s |
| 9 | 178,688 | 1,369/s | 2m 10s |
| 8 | 44,800 | 1,235/s | 36s |
| 7 | 11,264 | 995/s | 11s |
| Total | 21,211,681 | 15h 49m 42s |
I could have a tool that does the job after a single week, but I decided to add additional features and improve the performance.
Architecture#
geotiff2pmtiles reads Cloud Optimized GeoTIFF (COG) tiles on demand, from disk, generates map tiles in parallel, and writes them directly into a PMTiles archive — all in a single pass.
Memory-Mapped I/O#
Instead of loading entire GeoTIFF files into memory, the tool uses memory-mapped I/O (mmap) to access TIFF tiles on demand. The operating system manages paging — only the portions of the file currently being accessed reside in physical memory. This keeps the memory footprint bounded regardless of input file size.
Parallel Tile Generation#
Tiles are grouped into Hilbert-contiguous batches of 32 and distributed across a configurable worker pool. This batch size balances spatial locality (good cache reuse) with load distribution (minimal tail imbalance). Each worker performs per-pixel inverse projection and bilinear or nearest-neighbor resampling from the cached COG tiles.
Pyramid Downsampling#
Lower zoom levels are generated by combining 2x2 child tiles into parent tiles via box-filter downsampling, rather than re-reading from the source COGs. This avoids redundant I/O and resampling work. For elevation data (Terrarium encoding), the downsampler decodes RGB back to elevation values, averages valid samples, and re-encodes — ensuring correct interpolation even across tile boundaries.
Different downsampling modes are supported:
- Bicubic (4×4 Catmull-Rom) - default
- Mode (most common value) - ideal for categorical data
- Bilinear
- Nearest-neighbor
- Lanczos-3
Profiling-Driven Optimizations#
To achieve the desired performance I let Cursor run and profile the tool in a loop. It then decided to focus on what to work on and improve. Amongst other improvements forgotten I ended up with this list of optimizations:
| Optimization | CPU before | CPU after | Impact |
|---|---|---|---|
| Integer cache keys | 101s (18%) | < 1s | Eliminated string hashing |
| Type-specific pixel reads | 57s (10%) | 0s | Eliminated interface boxing |
| Remove madvise | 48s (9%) | 0s | Eliminated redundant syscalls |
| Per-tile source/level prep | 37s (6%) | < 1s | Eliminated per-pixel recomputation |
| Batch work queue | — | — | Improved parallelism from 3.2× to 3.3× |
| Total | 594s CPU, 188s wall | 248s CPU, 81s wall | 2.3× faster |
I also got this nice summary how the rest of the time is spent:
The remaining CPU time is dominated by fundamental work: page faults for reading mmap’d COG data (19%), writing output pixels (16%), and YCbCr→RGB color conversion (14%). These represent the actual cost of decoding JPEG tiles and compositing output images — not overhead.
Pretty impressive, right? I should probably re-check the performance since I added a lot of features which I haven’t benchmarked yet.
Let’s see if I can find a global satellite data set.
Source Code#
The project is open source and available on GitHub: pspoerri/geotiff2pmtiles.
Pre-built binaries for Linux and macOS (amd64/arm64) are available on the releases page.