Skip to main content
checkweather.io
Methodology
GuidesUSUKCAAUNZ

How we test forecast accuracy

Every accuracy number on checkweather.io is measured, not guessed. Here is exactly how we grade the weather models, where the data comes from, and the caveats we are upfront about.

Quick answer

We grade each open model's archived forecast against ERA5 reanalysis (the recorded weather) over the last 30 days across 300 cities per country, scoring temperature, wind and rain into a 0–100 CheckWeather Score. ERA5 is ECMWF-based, which modestly favours the European model — we disclose that rather than hide it.

The method, step by step

1

Take the models' real past forecasts

For every test city we pull each model's archived forecast — what it actually predicted ahead of time, not a hindsight rerun — for daily high temperature, maximum wind and precipitation.

2

Compare to what actually happened

We grade each forecast against ERA5, the European Centre's reanalysis of the real, recorded weather. This is the benchmark of truth.

3

Score temperature, wind and rain

We compute the mean absolute error for temperature and wind, plus a rain/no-rain hit-rate, over the last 30 days across 300 cities per country.

4

Blend into the CheckWeather Score

The three measures combine into a single 0–100 score per model. Lower error means a higher score. We publish the underlying numbers, not just the ranking.

What we're upfront about

ERA5 favours ECMWF slightly. Our truth dataset is built by the same centre that runs the European model, so the European model gets a small structural edge. We flag it on every scoreboard.

30 days is a snapshot, not a verdict. A month captures current form, not all-time skill. No model wins every day — which is the whole reason we show them side by side, live.

Closed apps can't be graded. AccuWeather, The Weather Channel and BBC Weather run proprietary blends we cannot verify. We grade the open models they are built on instead, and say so.

Who we are

checkweather.io is an independent weather-comparison project published by Nortra Online ApS (CVR DK43186388), Kongens Lyngby, Denmark. We have no affiliation with any national weather service or consumer weather brand, and we make money through the site itself, not by promoting any provider. Forecast and reanalysis data is sourced via Open-Meteo (CC BY 4.0); rain radar via RainViewer.

Common questions

How does checkweather.io measure forecast accuracy?
For each city we take each model's archived forecast of the daily high temperature, maximum wind and rain, and compare it to ERA5 — a reanalysis of the weather that actually happened — over the last 30 days. We calculate the mean absolute error for temperature and wind, plus a rain hit-rate, and blend them into a 0–100 CheckWeather Score. Lower error scores higher. Nothing is opinion; everything is measured.
What is ERA5 and why use it?
ERA5 is the European Centre's reanalysis dataset — the best available reconstruction of what the weather actually was, built from observations. We use it as the 'truth' to grade each model's earlier forecast against. Because ERA5 is itself ECMWF-based, it slightly favours the European model, and we disclose that rather than bury it.
Why compare models instead of picking one?
No model wins every day or every place. The disagreement between models is the honest signal of how uncertain a forecast is — and it is exactly what a single-number app hides. Showing all the models side by side lets you see the confidence for yourself.
Is the data free and open?
Yes. We source forecasts and reanalysis via Open-Meteo (CC BY 4.0), which is based on the national and global models: NWS/NOAA, ECMWF, UK Met Office, Environment Canada, DWD ICON and MET Norway. Radar is from RainViewer. We are not affiliated with any weather service or app.

See it in action

Most accurate in USMost accurate in UKMost accurate in CAMost accurate in AUMost accurate in NZWeather models compared