A discussion of AI weather models

  • November 2024
  • Meteorological Insight

The rainbow each individual sees is wonderfully unique. 

Your eyes are seeing a slightly different refraction than mine even when we’re standing next to one another, and also we each observe colour differently.

In the same way, our view of AI weather forecasts will be different: how we view them will likely depend on our scientific background, and personal AI experiences.  Mine is coloured by my doctoral thesis on distinguishing initial condition error (chaos) from model error in weather forecasts, and by being an operational weather forecasts for almost two decades.  My refraction of AI weather forecasts has some aspects which are shared by many, but is more practically oriented, so I thought to share some thoughts below. 

Isla presented her ideas at the Royal Meteorological Society summer conference 2024, as well as at an ECMWF workshop in 2024 so this isn’t their first outing.  They are her personal views and I am glad other views exist.   Constructive discussion is welcome!

AI landscapes are changing rapidly, and weather AI is no different.  If you read this after Dec 2024, it’s likely to be out of date.

First steps

There are a number of different weather AI forecast models, using a variety of AI methods (including Pangu weather, Deep Mind Graph Cast, FourCast, ECMWF’s AIFS).  Like all AI, good data is needed if you want a good model.  Most AI models are created from a dataset of our best estimate of historical weather (usually ERA5), and initialised using the same initial conditions used by ECMWF’s physics based numerical weather prediction (NWP) model. 

A few “hybrid” models exists combining AI and physics, but they are in the minority and the most successful are for climate applications, so far.  This seems a natural route to follow, and further exploration is hoped for (both for weather and climate).

Most AI models show slightly improved root mean square error compared to their physics based (NWP) equivalents, but (due to using RMSE optimisation over shorter-lead times in the build) have a tendency to smooth the fields at longer lead-times.  Some also suffer from a lack of temporal and/or spatial coherence (which matters for many practical applications, for example modelling energy requirements across Europe).  And since data used in building the models is limited to the troposphere (lower atmosphere), any dynamics originating higher in the atmosphere is not expected to be captured.  Finally, they are as sensitive to the initial conditions they start from as the NWP models – discussed more below.

Access and limitations to use for operational forecasting

All of the above deterministic models give access to their models and weights, but licenses vary.  ECMWF publishes the charts for many of the models on their website. 

Data for ECMWF’s AIFS deterministic model is available with a commercial use license, and so has been the model I’ve had most experience with.

Current AI reliance on physics based Numerical Weather Prediction

AI models currently rely on NWP for building and running, which raises two points

  • An  oft-quoted advantage of the AI models is that they require less computer power to run.  True for the daily forecast element.  But to build them they rely on the massive ERA5 database, which required a lot of computer power to generate.
  • The ERA5 data base used to build the AI models, and the files used daily to initialise the AI models, are from NWP models and use physics constraints. For a pure AI model, it would need to take the vast set of observations (station based, satellite etc) and create an AI history and an AI initialisation file. This should be a strength of AI, and some players are beginning to investigate these challenges. Whilst AI models rely on NWP to generate the initialisation file, experience shows that when we see “jumps”, or shifts, in the NWP solutions they tend to also occur in the AI models. Deterministic AI models have, so far, explored model error rather than uncertainty in the initial conditions. The first AI model built and/or initialised from raw observations will be a significant step – time to reach to the stars?

From deterministic to probabilistic

The second generation weather AI models shift from being deterministic (one run), to probabilistic models. ECMWF and Deep Mind both opted for a diffusion based ensemble approach, showing improvement in smoothing and temporal and spatial coherence.  (ECMWF has a second AI ensemble method, but it is not run operationally at the time of writing.)  Data for these is not yet available, making evaluation a challenge.  However, the choice of initial conditions for both is simplistic, with much that could be done.  The next steps here are exciting, since the cost of the ensemble is considerably less than for their NWP counterparts.  We have advocated for flow-dependent perturbations for some years but the complexity of implementing this idea is a major deterrent (tho one group in Europe is exploring the idea).  Whilst the SPP in the new cycle of the ECMWF NWP model out next week is a step in the right direction, ensemble AI models enable a much easier way to experiment with this approach.

Climate change

How are AI models coping with climate change?  Their training set is from a period when climate is changing, and often ends in 2020.  Since then we’ve seen record warm sea surface temperatures and numerous records broken.  How good will the AI be as we move into weather pattern evolutions that we haven’t previously seen?  Whilst they can derive physical bounds, they are not constrained by them and this is where hybrid models could help.

Operational forecasting: can AI contribute now?

AI models are being developed to forecast extremes for localised areas, with considerable success and this is one of the best examples of how they can help inform operational forecasters. 

For larger regions, AI is an exciting evolving development that shows great promise.  ECMWF AIFS forecasts can add value now in some circumstances (e.g. they seem better are forecasting hurricane land interaction and the consequences later in the forecast).  As they evolve, their contribution should increase – the more so if they build on the extensive existing knowledge.