# Preliminary WOUDC Brewer/Dobson benchmark, January-May 2026

Run date: 6 July 2026.

Status: **retrospective validation against independent ground observations; not
yet a prospective blind test**.

## Result

Against WOUDC Level-1 daily Brewer/Dobson total-column ozone observations,
TCOscope has lower RMSE than CAMS at every forecast lead.

| Lead | TCOscope RMSE, DU | CAMS RMSE, DU | Persistence RMSE, DU | TCOscope reduction vs CAMS |
| --- | ---: | ---: | ---: | ---: |
| T+1 | 17.53 | 23.23 | 29.16 | 24.6% |
| T+2 | 17.81 | 23.29 | 33.80 | 23.5% |
| T+3 | 18.55 | 23.44 | 35.54 | 20.9% |
| T+4 | 19.85 | 24.55 | 35.91 | 19.2% |
| T+5 | 21.92 | 26.38 | 36.74 | 16.9% |
| **All** | **19.19** | **24.20** | **34.33** | **20.7%** |

The paired seven-day block-bootstrap 95% interval for the overall reduction is
18.6%...22.8%; the TCOscope-minus-CAMS RMSE difference interval is
-5.72...-4.27 DU. Depending on lead, TCOscope has lower daily station-network RMSE
on 81 to 94 of the 107 forecast initialization dates.

Other global checks:

- Bias is +0.84 DU for TCOscope and -6.13 DU for CAMS.
- Centered RMSE is 19.17 DU for TCOscope and 23.42 DU for CAMS.
- Skill against persistence is 0.441 for TCOscope and 0.295 for CAMS.
- When every station is given equal weight regardless of data availability, RMSE
  is 20.32 DU for TCOscope and 24.44 DU for CAMS, a 16.9% reduction.
- On the strict direct-sun subset, RMSE is 17.06 versus 21.59 DU, a 21.0%
  reduction. Its 95% bootstrap interval is 17.3%...24.4%.
- TCOscope has lower aggregate RMSE at 30 of the 38 stations. No station was
  excluded based on the result.

This is currently the strongest validation result because WOUDC measurements are
not the TROPOMI target used to train TCOscope and do not share the satellite
retrieval grid. It is stronger evidence than the TROPOMI consistency benchmark
and cleaner for TCOscope than the OMPS cross-sensor comparison. It remains
retrospective: the 2026 TROPOMI test metrics had already been inspected, and the
checkpoint was not frozen in a public prospective protocol before these ground
scores were calculated.

## Instrument and latitude checks

| Instrument group | Matched forecast-station pairs | Stations | TCOscope RMSE | CAMS RMSE | Reduction |
| --- | ---: | ---: | ---: | ---: | ---: |
| Brewer only | 5,000 | 15 | 19.67 | 26.81 | 26.6% |
| Dobson only | 7,030 | 24 | 19.28 | 22.74 | 15.2% |
| Brewer and Dobson at one station-day | 1,742 | 5 | 17.33 | 21.96 | 21.1% |

The result is positive for both instrument families. The Brewer/Dobson station
counts overlap because some stations reported both during the period.

| Latitude band | Pairs | Stations | TCOscope RMSE | CAMS RMSE | Reduction |
| --- | ---: | ---: | ---: | ---: | ---: |
| Antarctic | 1,327 | 4 | 14.72 | 18.19 | 19.1% |
| SH midlatitudes | 1,966 | 4 | 13.68 | 16.59 | 17.6% |
| Tropics | 3,874 | 12 | 18.93 | 19.40 | 2.4% |
| NH midlatitudes | 5,495 | 15 | 20.66 | 27.31 | 24.3% |
| Arctic | 1,110 | 3 | 24.78 | 37.15 | 33.3% |

TCOscope wins in every band, but the tropical advantage is small and the polar
and Southern Hemisphere samples contain few stations. These are network scores,
not area-weighted global estimates.

The eight station-level losses are Tamanrasset, Makerere University, Varanasi,
Macquarie Island, Manila, Syowa, Princess Elisabeth and Amberd. The largest loss
is Tamanrasset: 10.26 DU for TCOscope versus 7.07 DU for CAMS. Station-level
diagnostics should be retained in a paper rather than reporting only the pooled
win.

## Protocol

- Forecast cases: 107 T0 dates from 2026-01-21 through 2026-05-07.
- Truth dates: 2026-01-22 through 2026-05-12.
- WOUDC snapshot: 200 monthly records, 3,535 instrument-day rows, 2,849 unique
  station-days and 38 stations.
- Primary truth is the published `#DAILY` value in each WOUDC TotalOzone Level-1
  monthly file. WOUDC defines it as the daily best representative value, usually
  prioritizing direct sun, zenith sky and focused moon observations.
- Multiple instruments at the same station and day are reduced to their median,
  so a station-day receives one truth value.
- TCOscope is sampled on its native 1.2-degree grid and CAMS on its native
  0.4-degree grid using periodic bilinear interpolation to station coordinates.
  Neither system is first remapped to the other system's grid.
- CAMS uses `gtco3` at 00 UTC and T+24/48/72/96/120. TCOscope uses its daily
  T+1...T+5 fields. No test-period bias correction is applied.
- The pooled score contains 13,772 matched forecast-station pairs. A station-day
  can appear at different forecast leads from different initializations; the
  uncertainty calculation therefore resamples seven-day blocks of initialization
  dates rather than treating all pairs as independent.
- Statistical uncertainty uses 10,000 paired circular block-bootstrap iterations,
  seven-day blocks and seed 42.

WOUDC documents the [TotalOzone dataset](https://woudc.org/en/data/dataset-information/totalozone)
and the [`#DAILY` extended-CSV field](https://guide.woudc.org/en/). The data are
retrieved from the public [WOUDC OGC API](https://api.woudc.org/collections/data_records/items).

Machine-readable outputs:

- `forecast/outputs/woudc_benchmark_2026/summary.json`
- `forecast/outputs/woudc_benchmark_2026/cases.csv`
- `forecast/outputs/woudc_benchmark_2026/station_days.csv`

The implementation is
[`forecast/src/evaluation/compare_woudc.py`](../forecast/src/evaluation/compare_woudc.py).

## Limitations and next step

This comparison is not yet sufficient for a final publication claim:

- WOUDC daily representative measurements have different observation types and
  times. CAMS 00 UTC and the daily TCOscope field are not interpolated to each
  station's `UTC_Mean`.
- A point instrument and a grid-cell mean do not represent exactly the same air
  column; the mismatch is larger for the 1.2-degree TCOscope grid than for CAMS.
- Station coverage is sparse and geographically uneven, especially in the
  Southern Hemisphere and polar bands.
- The interval covers January-May only and does not test a complete annual cycle
  or the Antarctic ozone-hole season.
- WOUDC records can be revised after this snapshot. A publication run needs a
  versioned data manifest with file hashes.
- The model was trained on TROPOMI and the historical split touches the
  2025/2026 boundary. A new untouched period is still required.

The decisive next experiment is prospective: freeze the checkpoint, code,
station list, filters and metrics now; register hashes; then score at least the
next six to twelve months without tuning. Also report raw, centered,
station-balanced, direct-sun-only, per-instrument and per-station results. If the
20% advantage survives that protocol, the project has a credible central paper
claim rather than only an encouraging retrospective benchmark.

Allowed wording now:

> In a retrospective January-May 2026 evaluation against 38 WOUDC Brewer/Dobson
> stations, TCOscope reduced pooled total-column-ozone RMSE by 20.7% relative to
> CAMS (95% block-bootstrap interval 18.6%...22.8%). The advantage remained 21.0%
> on strict direct-sun observations and 16.9% when stations were weighted equally.
