# Preliminary 2026 NRTI benchmark

This is a diagnostic near-real-time benchmark for the 2026 test period. It
evaluates the same 107 forecast initialisation dates as the preliminary CAMS
benchmark, first with an `NRTI-O3 hybrid` replay and then with a strict
`all-NRTI` replay.

## Protocol

- Period: forecast initialisations from 2026-01-21 to 2026-05-07.
- Forecast horizon: T+1 through T+5 days.
- Model: existing TCOscope checkpoint (`forecast/artifacts/best_model.pt`).
- Competitor: CAMS global atmospheric composition forecast, 00 UTC, T+24...T+120.
- Truth: TROPOMI NRTI O3 observed-cell mask on the model grid.
- Strict all-NRTI input history: O3, CO, NO2, SO2, HCHO, and AER_AI are all from
  TROPOMI NRTI monthly shards.
- Hybrid sensitivity: O3/O3_count/O3_filled/O3_fill_mask were replaced with
  TROPOMI NRTI O3, while other chemistry variables remained from the existing
  OFFL shards.

The strict all-NRTI replay is the cleaner operational benchmark. The hybrid run is
kept as a sensitivity check to separate the effect of NRTI ozone from the effect
of using NRTI auxiliary chemistry fields.

## Main result

Strict all-NRTI unweighted global RMSE, Dobson Units:

| Lead | TCOscope | CAMS | Persistence | RMSE reduction vs CAMS |
|---:|---:|---:|---:|---:|
| T+1 | 10.31 | 15.61 | 19.84 | 33.9% |
| T+2 | 11.69 | 16.03 | 25.78 | 27.0% |
| T+3 | 12.73 | 16.53 | 28.32 | 23.0% |
| T+4 | 13.94 | 17.36 | 29.68 | 19.7% |
| T+5 | 15.81 | 18.62 | 30.95 | 15.1% |
| All | 13.03 | 16.86 | 27.20 | 22.7% |

Paired block bootstrap over forecast initialisation dates:

- RMSE difference, TCOscope minus CAMS: median -3.84 DU; 95% CI [-4.28, -3.34] DU.
- RMSE reduction vs CAMS: median 22.8%; 95% CI [20.1%, 25.1%].

Area-weighted global all-lead RMSE:

- TCOscope: 10.91 DU.
- CAMS: 14.98 DU.
- RMSE reduction vs CAMS: 27.2%.

## Latitude-band result

Unweighted all-lead RMSE:

| Latitude band | TCOscope | CAMS | RMSE reduction vs CAMS |
|---|---:|---:|---:|
| Antarctic | 12.59 | 14.65 | 14.1% |
| SH-midlat | 8.97 | 14.89 | 39.8% |
| Tropics | 5.06 | 6.39 | 20.8% |
| NH-midlat | 17.70 | 24.98 | 29.2% |
| Arctic | 21.75 | 24.81 | 12.3% |

## Comparison with the OFFL-TROPOMI 2026 benchmark

| Benchmark | TCOscope RMSE | CAMS RMSE | Persistence RMSE | Reduction vs CAMS | Skill vs persistence |
|---|---:|---:|---:|---:|---:|
| OFFL TROPOMI 2026 | 12.46 | 17.86 | 27.19 | 30.2% | 54.2% |
| NRTI-O3 hybrid 2026 | 13.08 | 16.86 | 27.20 | 22.4% | 51.9% |
| Strict all-NRTI 2026 | 13.03 | 16.86 | 27.20 | 22.7% | 52.1% |
| OMPS 2026 | 20.83 | 25.08 | 30.53 | 16.9% | 31.8% |

The strict all-NRTI benchmark is weaker than the OFFL-TROPOMI benchmark, mostly
because the CAMS error against NRTI TROPOMI is also smaller than against OFFL
TROPOMI. The model still keeps a clear advantage over CAMS and persistence.

The strict all-NRTI result is almost identical to the NRTI-O3 hybrid result. This
suggests that the NRTI/OFFL gap in this test is driven mainly by the ozone field
and verification target, not by auxiliary NRTI chemistry variables.

## Interpretation

This is a positive operational signal:

- The model does not depend on an unrealistically clean OFFL ozone target to retain
  skill on 2026 dates.
- The advantage over CAMS remains statistically clear under paired date bootstrap.
- The largest gains remain in mid-latitudes; polar gains are present but smaller.

The main caveat is that this is still an in-distribution 2026 test using the
existing model architecture and checkpoint. For a publication-grade operational
claim, the next step is a prospective blind run: freeze the pipeline now, save
daily forecasts as they are issued, and score them only after independent
observations arrive.

## Output artifacts

- Summary: `forecast/outputs/cams_benchmark_2026_nrti_o3_hybrid/summary.json`
- Daily cases: `forecast/outputs/cams_benchmark_2026_nrti_o3_hybrid/cases.csv`
- Latitude cases: `forecast/outputs/cams_benchmark_2026_nrti_o3_hybrid/latitude_cases.csv`
- Strict all-NRTI summary: `forecast/outputs/cams_benchmark_2026_nrti_full/summary.json`
- Strict all-NRTI daily cases: `forecast/outputs/cams_benchmark_2026_nrti_full/cases.csv`
- Strict all-NRTI latitude cases: `forecast/outputs/cams_benchmark_2026_nrti_full/latitude_cases.csv`
- NRTI O3 source shards: `research/data/external/tropomi_nrti_2026_o3`
- Strict all-NRTI source shards: `research/data/external/tropomi_nrti_2026_full`
- Hybrid benchmark shards: `research/data/external/tropomi_nrti_o3_hybrid_2026`
