# Preliminary OMPS cross-sensor benchmark, January-May 2026

Run date: 6 July 2026.

Status: **retrospective cross-sensor validation for TCOscope; not yet a fully
independent or blind CAMS comparison**.

## Result

On the same 107 forecast initializations used by the TROPOMI benchmark, TCOscope
has lower global RMSE than CAMS against raw NASA OMPS-NPP NMTO3 Level 3 daily total
ozone at all five leads.

| Lead | TCOscope RMSE, DU | CAMS RMSE, DU | Persistence RMSE, DU | TCOscope reduction vs CAMS |
| --- | ---: | ---: | ---: | ---: |
| T+1 | 20.49 | 24.74 | 27.19 | 17.2% |
| T+2 | 20.46 | 24.84 | 29.48 | 17.6% |
| T+3 | 20.61 | 24.97 | 30.63 | 17.5% |
| T+4 | 20.88 | 25.21 | 31.87 | 17.2% |
| T+5 | 21.68 | 25.60 | 33.15 | 15.3% |
| **All** | **20.83** | **25.08** | **30.53** | **16.9%** |

The paired seven-day block-bootstrap 95% interval for the overall reduction is
16.0%...17.8%; the TCOscope-minus-CAMS RMSE difference interval is -4.53...-3.97
DU. TCOscope has lower daily global RMSE on all 107/107 init dates at every lead.

Additional global checks:

- Raw bias: +4.21 DU for TCOscope and -1.78 DU for CAMS.
- Centered RMSE: 20.40 DU for TCOscope and 25.01 DU for CAMS, an 18.4% reduction.
- Area-weighted RMSE: 19.00 DU for TCOscope and 22.57 DU for CAMS, a 15.8%
  reduction.
- Skill against persistence: 0.318 for TCOscope and 0.179 for CAMS.
- Co-observed TROPOMI-vs-OMPS disagreement is itself 21.41 DU RMSE with a +5.98
  DU TROPOMI-minus-OMPS bias.

The cross-sensor result is weaker than the 30.2% reduction found when both systems
are scored against TROPOMI. This is expected: TCOscope was trained on TROPOMI-like
targets. The advantage nevertheless remains positive and tightly bounded against
the external OMPS product.

## Latitude bands and sensor bias

Raw OMPS scores:

| Band | TCOscope RMSE | CAMS RMSE | Raw reduction | Centered reduction |
| --- | ---: | ---: | ---: | ---: |
| Antarctic | 16.10 | 19.65 | 18.1% | 18.0% |
| SH midlatitudes | 18.95 | 22.13 | 14.4% | 17.8% |
| Tropics | 9.32 | 8.21 | **-13.5%** | **+9.6%** |
| NH midlatitudes | 31.72 | 39.31 | 19.3% | 20.2% |
| Arctic | 29.61 | 36.62 | 19.1% | 18.4% |

TCOscope wins the raw comparison in four of five bands. CAMS wins raw tropical
RMSE. That reversal is explained by a cross-sensor offset rather than worse
centered variability:

| Band | TROPOMI-OMPS bias | TROPOMI-OMPS RMSE | TCOscope bias vs OMPS | CAMS bias vs OMPS |
| --- | ---: | ---: | ---: | ---: |
| Antarctic | +3.41 | 16.47 | -0.14 | -0.58 |
| SH midlatitudes | +7.01 | 20.47 | +5.41 | -1.20 |
| Tropics | +6.07 | 8.94 | +5.64 | -0.31 |
| NH midlatitudes | +7.00 | 33.34 | +5.48 | -3.63 |
| Arctic | +5.57 | 28.82 | +1.36 | -5.25 |

In the tropics, TCOscope inherits almost exactly the TROPOMI-to-OMPS offset. Its
centered RMSE is 7.42 DU versus 8.20 DU for CAMS, but its +5.64 DU raw bias makes
the raw RMSE worse. No test-period bias correction was applied. A future
harmonized score may only use a correction fitted on a separate calibration
period and must be reported alongside, not instead of, the raw score.

The very high northern midlatitude and Arctic errors should also be interpreted
against the TROPOMI-vs-OMPS disagreement floor: 33.34 and 28.82 DU respectively.
This indicates substantial retrieval/sampling disagreement in addition to actual
forecast error.

## Protocol

- Forecast cases: 107 T0 dates from 2026-01-21 through 2026-05-07.
- Truth dates: 111 daily fields covering all valid dates through 2026-05-12.
- Truth product: NASA OMPS-NPP NMTO3 L3 daily total ozone, native 1-degree grid.
- Mean accepted OMPS coverage on the common grid: 88.6%.
- A destination cell is accepted when at least 50% of its spherical area has valid
  OMPS observations.
- OMPS and CAMS are conservatively remapped to the actual TCOscope 1.2-degree cell
  bounds; no cross-sensor bias correction is applied.
- CAMS: `gtco3`, parameter 210206, 00 UTC T+24/48/72/96/120.
- Statistical uncertainty: paired circular block bootstrap over init dates,
  seven-day blocks, 10,000 iterations, seed 42. Days with no accepted observations
  in a polar band have zero weight and are not counted as wins or losses.

Machine-readable outputs:

- `forecast/outputs/omps_benchmark_2026/summary.json`
- `forecast/outputs/omps_benchmark_2026/cases.csv`
- `forecast/outputs/omps_benchmark_2026/latitude_cases.csv`

The implementation is
[`forecast/src/evaluation/compare_cams.py`](../forecast/src/evaluation/compare_cams.py).

## Independence limitations

The production TCOscope checkpoint was trained on TROPOMI shards and did not use
the OMPS NMTO3 target, so OMPS is a genuine cross-sensor test for TCOscope.

For CAMS, the situation is less clean. Current CAMS documentation lists ozone
products from OMPS instruments among assimilated real-time observations, although
the assimilated `O3 layers` are not necessarily the same NMTO3 L3 total-column
product used here. CAMS also assimilates TROPOMI O3. Therefore neither satellite
benchmark is a completely independent test of CAMS:

https://confluence.ecmwf.int/spaces/CKB/pages/212454117/

Ground-based Brewer/Dobson WOUDC observations remain the required primary
independent validation. Other existing limitations also remain: the historical
split boundary overlap, prior inspection of 2026 metrics, mismatch between 00 UTC
CAMS fields and daily satellite sampling, and absence of the Antarctic ozone-hole
season.

Allowed wording at this stage:

> Against raw OMPS-NPP NMTO3 daily observations over January-May 2026, TCOscope
> reduced global RMSE by 16.9% relative to CAMS. The reduction remained 18.4%
> after removing each system's single aggregate bias, while CAMS retained lower
> raw error in the tropics because of a TROPOMI-to-OMPS offset.

