Forecast Validation

Step 1 — Record. Every saved forecast keeps its exact issue time and hourly predictions from six public models.

Step 2 — Check. After a complete ERA5 day is available (about six calendar days later), we fetch it explicitly as the hourly verification reference.

Step 3 — Score. We compare equal 1-hour, 3-hour or 6-hour rain windows at the true lead time and rank each public source model separately.

Ranking (#1, #2, ...)

Models are ranked by overall skill — how well they balance catching rain, avoiding false alarms, and correctly calling dry spells, versus random chance. #1 is the most skillful model for the selected region and time period.
Technical: Ranked by Heidke Skill Score (HSS)

“Catches rain?”

When it actually rained, how often did this model predict it? Higher is better. 47% means it correctly predicted 47 out of 100 rain hours.
Technical: Probability of Detection (POD)

“False alarms?”

When this model predicted rain, how often was it wrong? Lower is better. 72% means 72 out of 100 rain predictions didn’t happen.
Technical: False Alarm Ratio (FAR)

Rain events

“Caught” = model predicted rain and it rained. “Missed” = it rained but model didn’t predict it. “False alarms” = model predicted rain but it stayed dry.

• ERA5 is a model-assimilated reanalysis at roughly 25 km, not a rain gauge or street-level ground truth.
• ERA5 shares ECMWF lineage with IFS and AIFS, so it is not fully independent of those models.
• Complete ERA5 verification is delayed by about six calendar days.
• City rankings are exploratory: short city histories are usually too noisy to identify a true local winner.
• MausamGram (IMD) is only available ~64% of the time.
• Pre-monsoon thunderstorms are inherently hard to predict — low scores don't mean models are broken.
• First season of data collection — scores become more meaningful over time.