Eight Layers Nationwide, and the 19-Hour-52-Minute Land-Use Mesh Build

Hello from the MAPRISE team. Last time, we wrote about taking registry cadastral maps and PLATEAU nationwide. Work on other layers continued in parallel, and eight more layers have now gone nationwide in production. This isn't a feature announcement — it's a record of that rollout itself. In particular, we want to be honest about why the land-use detailed mesh, the heaviest of the eight, took 19 hours and 52 minutes, and what actually happened along the way.

The eight layers that went nationwide this round

For every one of these, we confirmed existing data was untouched before and after ingestion, and verified after deployment that the actual tile-serving endpoints return real data for the newly added areas.

Putting Kyushu's lessons to work at national scale

The approach we used for Kyushu's 7 prefectures doesn't scale cleanly to 47 prefectures and hundreds of municipalities if applied as-is — verification alone would never finish. This round, we built on what we'd learned in Kyushu and pushed several techniques further.

Parallel execution: For the 21-year real-estate transaction backfill, our initial estimate was "roughly 1.75x speedup at 2-way parallelism." In practice we ran 8-9 processes in parallel, after confirming by measurement that the reinfolib API has no clear IP-based rate limit and that 8-9 concurrent psycopg2 connections don't cause lock contention on the DB side. All 6 batches completed with zero errors, and the measured speedup came out to roughly 3.4-4.9x.

Staged verification, every time: Every one of the eight ingestions followed the same pattern: dry-run (full rollback, confirming zero writes) → count verification → real ingestion → a before/after diff across all prefecture counts confirming untouched prefectures stayed exactly untouched. It's tedious, but skipping this step is exactly how the performance bugs and spec edge cases described below would have gone undetected until after launch.

Working around an environment constraint: Across several sessions this round, we repeatedly hit a constraint where the browser pane wouldn't composite visually (document.visibilityState stuck at hidden, so MapLibre's initialization never fires). As a workaround, we issued PMTiles-protocol-compliant HTTP Range GETs directly against the production domain, or ran fetch() inside the actual production page's JavaScript context and byte-compared the result against a server-side curl. We couldn't eyeball pixels, but we could verify the exact delivery path a real end user's browser would take — which, as a check, is arguably more rigorous than a screenshot.

Why the land-use mesh alone took 19 hours 52 minutes

Of the eight layers, the land-use detailed mesh (all 9 fiscal years: 1976, 1987, 1991, 1997, 2006, 2009, 2014, 2016, 2021) stood out by a wide margin. The pipeline started at 20:28 JST on August 16, 2026, and all 9 years finished at 16:20 JST on August 17 — a measured 19 hours 52 minutes.

Finding the bottleneck by measurement, not guessing

Before designing any parallelization, we ran a pilot build on 20 Kyushu meshes from fiscal year 2006 and timed each phase.

Download time was negligible (sub-second per file), while tippecanoe alone took over 15 minutes — clearly identifying the bottleneck as CPU/memory, specifically tippecanoe's geometry re-indexing. The GeoJSON-to-compressed size ratio came out to 16.40x, and stayed nearly constant between urban and rural samples, so we treated that ratio as reliable enough to extrapolate to national scale.

The extrapolation was wrong — in a good way

Our initial worry was that naively extrapolating Kyushu's results (19 meshes, several hundred MB to just over 1GB) by mesh-count ratio (nationwide 175-180 meshes, roughly 9.2-9.5x) would put the nationwide total in the tens-of-GB to 100GB range. Actual measurement came in at just 9.28GB. The reason: Kyushu is dense terrain and land use with relatively poor compression, while at national scale, mountain, island, and offshore-boundary meshes — low density, high compression — pull the average way down. A simple mesh-count ratio overestimates badly.

That said, working backward from tippecanoe's measured throughput (327 sec per GB of GeoJSON), the estimated total time for all 9 years nationwide came to roughly 15.7 hours — which accounts for most of the actual runtime (the measured 19h52m also includes time spent handling the two incidents described below).

Parallelization strategy, phase by phase

Since the bottleneck differs by phase, so does the parallelization strategy for each.

• Download (I/O-bound): parallelized with ThreadPoolExecutor(max_workers=6). We deliberately avoided excessive concurrent connections out of consideration for the government site.
• GeoJSON conversion (CPU-bound): capped at max_workers=2, matching the instance's 2 physical vCPUs — applying the lesson that parallelism shouldn't exceed physical core count.
• tippecanoe (memory-bound): a single process, fully sequential even across fiscal years, based on prior measurements showing a single process can use 500MB+ of memory.

We also considered a temporary EC2 upgrade, but decided against it after confirming CPU credits were in Unlimited mode, and that the self-healing logic behind an earlier incident — where a false-positive pg_isready failure under production DB load triggered a kill -9 — had already been removed the day before. Instead, we mitigated contention with nice and ionice against the production DB and Martin, running sequentially on the existing instance spec.

The two incidents

Two runtime blockers occurred over the 19h52m. In both cases we diagnosed the actual root cause by measurement before fixing anything — no guessing.

Incident 1: S3 upload failure for fiscal year 1976

The cause: an AWS profile flag (--profile) meant only for a local development machine had been carried over into the EC2-targeted script by mistake. That profile doesn't exist on EC2, so the upload failed — but the tippecanoe build itself had already succeeded, so nothing was lost. We recovered using the correct authentication (the EC2 instance role) and, as a permanent fix, removed the --profile flag from the script entirely.

Incident 2: tippecanoe failed with "No space left on device" for fiscal year 2006

The cause: our disk-capacity planning was based on the Kyushu-only baseline (350MB-1.2GB of output per fiscal year). At national scale, actual output came to 7-8GB per fiscal year, and the design of holding every year's intermediate GeoJSON files simultaneously simply didn't hold up across all 9 years. We recovered 42GB by releasing the local copies of fiscal years that had already been verified and successfully uploaded to S3. As a permanent fix, we changed the design so that, after a successful S3 upload, we verify the size via head-object before deleting the local output file (raw ZIP source files are retained as an audit trail).

In the end, roughly 64.1GB shipped to S3 nationwide, and an independent audit script — pulling raw headers directly from S3 rather than trusting the build script's own completion report — confirmed a full pass across all 9 fiscal years.

Designing for ongoing operation

Finishing a nationwide rollout once doesn't mean these datasets are done. New fiscal years get published, and municipality-level readiness keeps shifting. This round, we revisited how we detect newly published data for both PLATEAU and the land-use mesh.

PLATEAU: On top of the existing monthly monitor (querying the G-Spatial Information Center's CKAN metadata), we added a mechanism to keep tracking the "unknown structure-type code" problem discovered during Stage 1 ingestion. We now monitor monthly for revisions to MLIT's own official code list, and only when a revision is detected do we re-examine the currently excluded municipalities via a dry-run (a re-run of the quality gate that makes zero production writes). It's a two-layer design — a cheap monitor watches for a signal, and the expensive re-check only runs when that signal fires. Unconditional high-frequency automation simply isn't worth the cost.

Land-use mesh: The existing monitor watched a fixed URL. Investigating further, we found that the government's own site keeps three different URLs alive simultaneously for the same dataset — an old version, the current version, and a link surfaced from the master listing page — and that a new spec version can be published under an entirely new URL rather than as an update to the existing one. A fixed-URL monitor would miss exactly that case: a new version shipping at a new URL while the old one sits frozen and unchanged. So we switched to dynamically resolving the current representative URL from the government's own master listing page every time, and monitoring that.

Why not fully automate?: In both designs, the decision of "should we actually ingest this" is left to a human — a staged notify → L3 approval → ingest flow. The reason is simple: Stage 1 actually hit 7 municipalities with quality-gate failures nobody had anticipated, so "a new municipality or fiscal year will always ingest safely" isn't a premise we can rely on. A single land-use mesh build alone also produces over ten hours of CPU and disk load — the kind of change a human should be aware of before it happens. What gets automated is detecting and surfacing the change; the final call on whether to actually ingest stays human. This round made that split explicit again.

Quality and regression checks

Going nationwide meant designing separate safeguards for two different concerns: not breaking the existing Kyushu data, and keeping the quality of the newly ingested data itself.

Fail-closed quality gates: For PLATEAU building ingestion, 238 of 245 targeted municipalities succeeded, but 7 were held back because their building data included structure-type codes (607, 612, 613) that don't appear in the official code list. Rather than guessing and extending the code mapping to cover the unknown values, the gate simply held the ingestion back, leaving the actual investigation — checking the source spec's revision history — to a human. That pause is exactly what kept an unverified mapping out of production.

Non-contact verification before/after ingestion: For the real-estate transaction backfill, we diffed per-prefecture counts before and after each of the 6 batches, confirming that only the batch's target prefectures changed. The ground-boring ingestion SQL includes a safety guard that aborts immediately if even a single Kyushu record shows up in the incoming data.

Independent auditing: For the land-use mesh, rather than trusting the build script's own completion report, we ran a separate audit script that pulls raw PMTiles headers directly from S3. For the inland flood layer, we deliberately sent tile requests against excluded municipalities — areas where no data should exist — and confirmed the response correctly came back empty, a negative control.

None of this is glamorous work, but refusing to simply trust "ingestion complete" and instead cross-checking against the real data every time is, we think, how you catch the bugs that would otherwise only surface after launch.

Closing

The nationwide rollout of these eight layers, and the 19-hour-52-minute tippecanoe build behind one of them, isn't flashy work — but it's unavoidable when you're dealing with map data, which never truly finishes. We think of map data as a genuine trove of big data: it lets you stack fundamentally different kinds of information — statistics, land prices, disaster history, buildings, ground conditions — on top of one another through the one thing they share, location. Quantifying an accumulated, continuous history like that, and making it analyzable from many angles with digital tools and AI, is how more people get to benefit from it. We'll keep building toward that, one layer at a time.

Questions or feedback? Reach us any time at info@maprise.jp.

The data referenced in this post is sourced from open data published by Japan's Ministry of Land, Infrastructure, Transport and Tourism, the Ministry of Justice, the Geospatial Information Authority of Japan, and the Statistics Bureau of Japan, among others. Record counts, data volumes, and processing times are measured values drawn from each task's primary completion report and reflect this post's publish date; they may change as operations continue.