Methods
The question was whether a geospatial foundation model's embeddings carry information about European surface climate that a hand-crafted covariate stack does not. The comparison, the metrics, and the threshold were fixed in writing before the probes were fitted.
The Six-Arm Comparison
Every arm predicts the same target: COSMO-REA6 daily temperature at 6 km over Europe, given ERA5 as the coarse input. The arms differ only in the static feature block appended to that input, so the difference between two rows is the difference between two representations of place and nothing else. Each is a quantile regression fitted per fold, scored on the fold it never saw.
| Arm | Static features | RMSE, tasmax (K) |
|---|---|---|
| bilinear | none, the interpolation floor | 1.860 |
| cov | hand-crafted covariates | 1.469 |
| aef | AlphaEarth, 64 dimensions | 1.530 |
| aef_cov | both | 1.467 |
| clay | Clay v1.5, per-fold PCA | 1.621 |
| mosaiks | random convolutional features | 1.480 |
MOSAIKS is the arm that locates the problem. Random convolutional features are built to be spatially averaged, so pooling is an operation they tolerate, and MOSAIKS finishes closer to the covariate stack than either embedding does. The measured mechanism is in the repository's why-AEF-lost note: native AlphaEarth vectors have a norm coefficient of variation of 1.26%, and mean-pooling them to 6 km takes that to 9.01%, which puts the features off the manifold the encoder was trained to produce.
Decision rule, as registered
The claim was that embeddings beat the covariate stack by at least 10% on both RMSE and CRPS, pooled over 5 held-out spatial folds. Measured: 4.2% and 4.3% in the wrong direction. The claim is not met, so the headline is the comparison itself.
Spatial Blocks, Not Random Splits
A random pixel split on an autocorrelated field measures interpolation, not generalisation, because a held-out cell's neighbours are in the training set. The folds here are contiguous spatial blocks, and Iberia is held out entirely as a super-holdout that never enters the pooled statistic. The same block structure carries into Act 2's bootstrap, where the resampling unit is a 33-cell block rather than a cell.
What Was Fixed Before the Result
The pre-registration document names the arms, the target variables, the metrics, the pooling, and the 10% threshold. It was committed before any probe artifact was written, and the gate that reports the result also checks that ordering: it compares the document's commit timestamp against the earliest modification time of the 72 probe artifacts, and it compares the sha256 of the fold assignment recorded in the document against the fold file on disk. Both checks pass, and they are checks rather than assurances because a claim about what was decided when is worth exactly what it can be audited by.
The Persistence Filter, and Why It Is Not Shipped
Act 3 was designed around a filter that would separate lasting change from the annual churn of agriculture, by scoring how persistent a year-to-year embedding shift is rather than how large. Four criteria were declared in code before the filter was measured against a construction site and a cropland control.
| Criterion | Required | Measured | Result |
|---|---|---|---|
| Village des Athletes median score percentile | >= 90 | 96.3 | PASS |
| control median / construction median | <= 0.25 | 0.291 | FAIL |
| modal change year at the village | in [2021, 2022, 2023, 2024] | 2023 | PASS |
| control percentile in the NAIVE delta | >= 90 | 100.0 | PASS |
The filter suppresses the cropland control by a factor of 3.4 where the gate asked for 4.0. It separates the two sites on 3 measures out of four and misses on the one that decides whether the panel can be described as isolating construction. So the panel shows the naive year difference, labelled as such, and this section stays on the site. Tuning the estimator until it clears a gate it has already been measured against would make the gate meaningless.
The two site medians are on one quantity, the median consecutive-year angle: 50.92 degrees at the control and 16.66 degrees at the construction site.
What the Browser Downloads
The continent bundle is 15.3 MB gzipped, of which the coastline is 2.9 MB at 0.05 degree vertex spacing. The city rasters are a further 12.9 MB of WebP across 36 files. Every figure on both pages is substituted at build time from a file generated by the pipeline, so a stale number is a build failure rather than a typo nobody catches.