Numerical weather prediction (NWP) has brought great benefit to society over many years. NWP forecasts have been gradually improving in quality over time . This is due to improved numerical models, increased numbers of observations and improved methods for assimilating those observations.

Radio occultation (RO) observations from global navigation satellite systems (GNSSs) provide one source of observations to the global observing system. These measurements use the time delay in receiving signals from GNSS satellites at a low-Earth orbiting (LEO) satellite, caused by refraction of these signals in Earth's atmosphere. As the LEO orbits Earth the GNSS satellite will appear to rise above or set below Earth's horizon, depending on the relative motion of the satellites. Thus a sequence of observations of the amount of refraction provides a profile through Earth's atmosphere – each such profile is referred to as an occultation. For more information, see for instance and .

Over recent years the number of GNSS-RO observations has been steadily increasing. This has been largely driven by the launch of the COSMIC-2 constellation , Sentinel-6A and the purchase of observations from private companies such as Spire and PlanetIQ .

The Coordination Group for Meteorological Satellites (CGMS) provides recommendations on the number of observations that should be made each day with the various observation platforms. Whilst the CGMS is not able to mandate the number of observations to be made, it does provide guidance to national meteorological centres and world meteorological organisation (WMO) members on the number of observations that should be made. The CGMS high-level priority plan provides the following recommendation :

Advance the atmospheric radio occultation constellation, with the long-term goal of providing 20 000 occultations per day with uniform spatial and local time coverage on a sustained basis.

The Radio Occultation Modelling Experiment (ROMEX; ) was developed as an approach to provide evidence of the impact of increased numbers of GNSS-RO observations on NWP. Previous experiments have considered the impact of simulated observations. However, it was noted that a large number of GNSS-RO observations were available during 2022, largely from commercial providers. With the acquisition of these observations it would be possible to test the impact of increasing the number of observations available without the limitations caused by the use of simulated observations.

ROMEX brought together a large number of organisations from around the world. This included a variety of data providers from Europe, the USA and China, including commercial companies from within those regions. These organisations made their observations available to the project. Most of the data used in ROMEX was processed to bending angles by either the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) or the University Corporation for Atmospheric Research (UCAR), which helped to minimise differences or problems in the data. These data were provided to NWP centres within the ROMEX consortium in order to assess the impact of the data on NWP. It was requested that all NWP centres run at least the control experiment (consisting of data from the Metop, COSMIC-2, KOMPSAT-5, PAZ, TerraSAR-X, TanDEM-X and Sentinel-6A satellites) and an experiment with the entire dataset. Additional experiments were suggested, but these were considered optional.

The expected impact of increasing the number of GNSS-RO observations on NWP was studied by using an ensemble of data assimilations (EDA). In that study synthetic observations were created, and the impact of the observations were estimated by examining the change in the spread of the ensemble. They simulated 128 000 occultations per day and the effect of using certain fractions of these observations was considered. They found that 16 000 occultations per day gave approximately half the reduction in the ensemble spread of using the full set of observations. Therefore they recommended that 16 000–20 000 globally distributed occultations per day be considered as a minimum requirement for the global observing system.

conducted an observation system simulation study (OSSE) to assess the impact of increasing the number of GNSS-RO observations. An OSSE uses synthetic observations, like an EDA study. However, the simulated observations are derived using a “nature run” from a model which is independent of the model used to produce the forecasts. Thus the effects of differences between the models will affect the use of the observations. For this reason, we might expect that the results from an OSSE would be more reliable than those from an EDA. They simulated up to 100 000 occultations per day and found that there was no saturation in the benefit to the forecast from assimilating additional data. However, they noted that the rate of improvement diminished after 50 000 occultations per day was reached. They speculated that saturation of benefit might be reached at around 150 000 occultations per day in their system. Their experiment found forecast degradation in certain regions related to the addition of GNSS-RO observations – this was believed to be due to sub-optimal specification of the observation uncertainties for the GNSS-RO data.

A summary of the changes in the RMSE scores for various variables and forecast lead times for the initial experiments assimilating all ROMEX observations, compared to the control, are shown in Fig. . Whilst there is an improvement in the forecast for some quantities, such as tropical winds and temperatures at certain heights, there is a large degradation in the extra-tropical forecasts of geopotential height. This is most apparent when considering verification against European Centre for Medium-Range Weather Forecasting (ECMWF) analyses at short lead times. The cause of this degradation is due to a large shift in the bias of the geopotential height forecasts. Figure  shows the forecast bias of the control, the experiment assimilating all ROMEX observations, as well as two further experiments which are discussed in the next section. The bias of the short-range forecast becomes strongly negative when the additional observations are assimilated. At longer lead times the forecast drifts towards a positive bias, but this does not appear to lead to improved results in terms of the RMSE (as seen in Fig. ). Whilst it is hard to be certain about the true value of the average geopotential height of a given pressure level, the shift in the bias is large enough to be confident that it is detrimental change.

Since the largest detrimental impacts were seen in geopotential heights which are related to the integral of atmospheric temperature from the surface, it was thought that the sources of the bias would be related to observations in the lower troposphere. Therefore, an experiment was run which adjusted the GNSS-RO observations in this region. The following experiments are not an attempt to indicate that there is any bias in the observations, just test the sensitivity of the results to perturbations in certain regions. have investigated the properties of the ROMEX observations and found that the differences between the observation datasets are small. Figure  shows the mean difference between the observations and the model background of the observations after quality control, normalised by their average, that is 1μ=2N∑i=1NOi-BiOi+Bi where Oi and Bi are the observation and model background values for observation i, respectively, and N is the number of occultations. Each profile of observations is interpolated to a 200 m grid, thus the number of observations is approximately constant in height.

In the lower troposphere the mean values in Fig.  are substantially below zero, indicating that the observations are consistently smaller than the model background. This is a region where data processing choices can affect the bias. For instance, the length of the data record into Earth's shadow can affect lower-tropospheric biases . Between 15 and 20 km there is a small-scale oscillation in the statistics. This is due to the interpolation of the atmospheric quantities from model levels to the impact heights of the observations. These oscillations were reduced by the use of pseudo-levels , but were not fully eliminated. If we modify the observed bending angles between 0 and 7 km impact height to be larger, then the bias in O-B in this region will be reduced. An experiment was run where the observed bending angles were adjusted by a factor, linearly increasing from 1 (no adjustment) at 7 km impact height, to 1.025 at 0 km impact height (noting that this is below Earth's surface). This increase is designed to approximately fit the O-B bias seen in Fig. . The results of this experiment are shown in Fig. . Whilst the overall results have improved, there are still substantial degradations in many forecast quantities relative to the control. In particular, the geopotential height forecasts have improved, but are still substantially degraded compared to the control, especially when verified against ECMWF analyses.

Whilst the geopotential height at 500 hPa represents an integral of the atmospheric state from the surface up to 500 hPa, there is some evidence that we should be considering observations which are located above this level. noted that the forward operator for bending angle observations contains sensitivity to the atmospheric state well below the height of the tangent point of the ray. An adjustment to the atmospheric state below the observation will alter the modelled height of the observation, thus adjusting the model forecast of the observation. This has been referred to as a hydrostatic tail for the operator . Given this behaviour, one might ask whether the observations at higher altitudes can affect the behaviour of the forecasts of 500 hPa geopotential height. Looking again at Fig. , we notice that there is a small negative bias to the O-B statistics between approximately 7 and 30 km impact height. This region is known colloquially as the “golden region” , since it is where the observations are most accurate and are given the highest weight in data assimilation systems. Therefore, it is possible that very small biases in this region can play a role in forecast quality.

To test this hypothesis, we ran a further experiment where the observed bending angles were increased by 0.05 % throughout the entire profile, since 0.05 % is approximately the difference in the O-B statistics between 7 and 30 km impact height. This experiment performs much better than the initial experiment, especially for tropospheric geopotential heights (not shown). However, the overall performance is still negative for many forecast variables, and therefore we ran a further experiment where all observations are increased by 0.1 %. The results of this experiment are shown in Fig. . These results show that by bias correcting the GNSS-RO observations by 0.1 % we achieve a situation where the addition of the ROMEX observations is now a positive change overall, rather than a negative one. Certain results are still negative, such as the 50 hPa geopotential height forecasts measured against ECMWF analyses, and some of the medium-range forecasts when measured against radiosondes. Hence, this bias correction of the observations cannot be regarded as a complete solution.

As previously noted much of the degradation in the forecast quality was due to a change in the bias of geopotential height forecasts in the troposphere. As well as the original results, Fig.  shows the forecast bias for 500 hPa geopotential height of the experiments which apply a bias correction to the observations. The experiment which adjusts the observations by 0.05 % approximately halves the negative bias in the short-range forecasts of this quantity. The experiment adjusting by 0.1 % eliminates the negative bias, replacing it with a slight positive bias, similar to the control NWP system. With increasing lead time, the forecast tends towards a positive 500 hPa geopotential height bias. The change of bias in the short-range forecast seems to be the main reason that the adjusted experiments perform better than the initial experiment – they are able to remove the large negative bias in the geopotential height forecasts. Figure  shows verification against ECMWF analyses in the northern extra-tropics. Similar results are seen in the southern extra-tropics and for verification against sondes.

To further understand how this bias arises, we use a 1-dimensional variational (1D-Var) assimilation to examine the analysis increments. A 1D-Var assimilation is run as part of the quality control of observations at the Met Office. If the minimisation does not converge within 20 iterations, then the entire profile of observations is rejected . The 1D-Var process is not identical to treatment of observations within the 4D-Var minimisation used to initialise the forecasts, since no account is made for the drift of the tangent point, and the same 1D background-error covariance matrix is used for all observations. Nonetheless, analysis increments from this system can provide useful insights.

To demonstrate the behaviour of the ROMEX experiments, we take observations from the second cycle of the experimental period – centred around 06:00 UTC on 1 September 2022. The model background state is taken from the operational suite on that day, and all the ROMEX observations between 03:00 and 09:00 UTC are used in the processing. To show how observations at different heights affect the increment produced by the 1D-Var analysis, the observations were perturbed by multiplying them by the factor 1.001 in various height ranges. The observations were only perturbed if their impact height was within 2.5 km of a given value, where the value was either 5, 10, 15 or 20 km. These perturbed analysis increments were then compared against an analysis without any perturbation.

Figure  shows the analysis increment from the perturbed and unperturbed experiments for pressure, temperature and specific humidity, averaged over all the observations. Since it is a global average, the increment is rather small and potentially changed substantially by the perturbations. Notably, the average increment to pressure is negative below 23 km impact height whereas the temperature increment changes sign at various points in the profile. The average increment to specific humidity is negative, and very similar for both the perturbed and unperturbed simulations.

It is interesting to consider the difference between the analysis increments from the perturbed and unperturbed systems. This is shown in Fig.  for temperature, pressure and specific humidity. For temperature, the main change in the increment is a reduction in the temperature at the region where the observations have been perturbed, and an increase at other levels. For pressure, the maximum change is at the bottom of the perturbed region, with an increase in the pressure both within and below the perturbed region. The perturbations to specific humidity serve to increase it, with the largest changes for those perturbations which occur in the troposphere.

The above results indicate that the reduction in the short-range geopotential height forecasts when assimilating the additional GNSS-RO observations is likely due to a systematic reduction in the atmospheric pressure. This is caused by observations in the upper troposphere and above, due to the hydrostatic tail present in the forward operator .

Following the experiments adjusting the observations by a constant factor, it was pointed out (Katrin Lonitz, personal communication, 2025) that a very similar effect could be achieved by adjusting the refractivity coefficients in the observation operator. To calculate the refractivity from a model forecast, the Met Office uses the two-term formula of

2N=k1pT+k2eT2 where p is the total atmospheric pressure (including both the dry gases and water vapour, in Pa), T is the temperature in Kelvin, e is the partial pressure of water vapour, also in Pa, k1 and k2 are constants derived from laboratory measurements and are k1=0.776 and k2=3.73×103. The constants used in this equation are derived from rather old experiments, and more recent formulations of the refractivity may lead to more accurate simulations of the atmosphere . quotes an estimated error of around 0.1 % in the formula for refractivity. Since it is derived from first principles, give an estimated error of 0.01 % in their formula. Therefore, assuming changes of 0.1 % in the formula of is reasonable.

To run an experiment equivalent to the ones above, k1 was reduced by 0.1 %. The results are very similar to those shown in Fig. , and are therefore not included here. Changing the k1 value gives slightly better forecasts of extratropical temperature and wind, but slightly worse forecasts of extratropical geopotential height. This highlights that a bias correction to the observations can have a very similar effect to an adjustment to the observation operator.

Considering Fig.  once more, one will note that in addition to the very small bias between 7 and 30 km impact height, there are also large biases in the lower troposphere, as noted previously. Therefore, we consider combining changes to both k1 and k2 in the refractivity operator. In order to achieve an approximately unbiased average value of the O-B statistics we reduce k1 to 0.775612 (a 0.05 % reduction) and k2 to 3600 (a 3.5 % reduction). Whilst we cannot offer any theoretical justification for either of these changes, they are chosen to approximately match the O-B statistics. With these operator changes, the mean O-B statistics are closer to zero than the statistics shown in Fig. . The results of this experiment are shown in Fig. . Aside from high altitude geopotential heights against ECMWF analyses, this experiment performs well, with many more positive results than negative ones. The results for geopotential heights in the troposphere are now generally positive, indicating that the former bias issues have been effectively addressed.

Another important measurement of the quality of a given change to the data assimilation system is the fit of observations to the model background. Since a 6 h data assimilation window is used, the background forecast lead time is between 3 and 9 h. We measure this using the standard deviation of the difference between the observations and either the model background or the analysis. The ratio of this standard deviation, relative to the same quantity for the control is plotted for each experiment. If this ratio is less than one then it indicates that the experiment has a better fit to the observations than the control. Figure  shows this ratio for various observation groups using the background forecast for the experiment which reduced k1 by 0.05 % and k2 by 3.5 %. For all three instruments shown, the experiment is closer to the observations for many of the channels. However, for each of the instruments there are some channels where the ratio is greater than one, indicating that the fit to the observations is worse than the control. Cross-track infrared sounder (CrIS) channels with numbers below 25, and Infrared Atmospheric Sounding Interferometer (IASI) channels with numbers below 70 are typically temperature sounding channels which peak in the stratosphere. Similarly, Advanced Microwave Sounding Unit (AMSU-A) channel 8 peaks in the lower stratosphere. Therefore, all those channels for which the addition of GNSS-RO observations has a negative impact are in the stratosphere.

Figure  shows the same statistics as Fig. , but for the analysis rather than the background. We see that the fit to the observations is degraded for almost all channels when using the analysis. In a perfect system you might expect that adding extra GNSS-RO observations would improve the representation of truth in the analysis leading to improvements of fits to other observations. This is often not the behaviour observed due to differences between observation types including their resolution, coverage, bias etc, which means that one often finds that adding much more of one dataset pulls the analysis away from some others. The overall skill of the analysis might still be improved and give rise to the more widespread (although not universal) improvements in short-range forecast fits.

The verification results presented above are based on the root-mean-square error of the forecast. It is common instead to consider the standard deviation of the forecast error, as this is unaffected by forecast biases. Typically, the root-mean-square error is calculated using the following formula

3RMSE=1Nt∑i=1Nt1No∑i=1Nofi,t-oi,t2 where oi,t is the verifying reference value at time t and location i, fi,t is the forecast value, and Nt and No are the number of times and locations, respectively. This means that the mean-square error is calculated for each reference time, and then this is averaged over all times. In a similar way, we can calculate the forecast bias via 4Bias=1Nt∑i=1Nt1No∑i=1Nofi,t-oi,t. The calculation of the standard deviation of the forecast error is based on these two quantities 5σ=RMSE2-Bias2 where σ is the standard deviation of the forecast error.

Experiments at ECMWF, Deutscher Wetterdienst (DWD) and the Joint Center for Satellite Data Assimilation (JCSDA) (Katrin Lonitz, Harald Anlauf, personal communications, 2025; ) demonstrated that the addition of ROMEX observations, without any changes to the operator to account for the bias, resulted in reductions in the standard deviation of the forecast error. Figure  shows a scorecard of the standard deviation of the forecast error for the experiment with all ROMEX observations, compared to the control. Like the RMSE scorecard (Fig. ), there are forecast degradations in many quantities. This is different from the equivalent experiments at ECMWF and DWD. One alternative which was considered is whether the above formulation of the standard deviation of the forecast error affected the results. DWD calculates the standard deviation of forecast error for each verification date (Harald Anlauf, personal communication, 2025) and averages these, rather than calculating the mean-square-error and bias for the entire period. This was attempted, but made very little difference to the results (not shown).

When the refractivity operator is adjusted by 0.05 % for k1 and 3.5 % for k2, the results for standard deviation are shown in Fig. . These results are very similar to those shown in Fig.  in that many of the scores are greatly improved, relative to Fig. . It is not clear why the bias corrections applied to the operator are having a clear effect on the standard deviation of the forecast error, unlike other centres.