the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
A Statistically Optimal Analysis of Systematic Differences between Aeolus HLOS Winds and NOAA’s Global Forecast System
- 1NOAA/NESDIS/Center for Satellite Applications and Research (STAR), College Park, MD 20740, USA
- 2Cooperative Institute for Satellite Earth System Studies (CISESS), University of Maryland, College Park, MD 20740, USA
- 3University of Maryland, College Park, MD 20740, USA
- 1NOAA/NESDIS/Center for Satellite Applications and Research (STAR), College Park, MD 20740, USA
- 2Cooperative Institute for Satellite Earth System Studies (CISESS), University of Maryland, College Park, MD 20740, USA
- 3University of Maryland, College Park, MD 20740, USA
Abstract. The European Space Agency Aeolus mission launched the first of its kind spaceborne Doppler wind lidar in August 2018. To optimize assimilation of the Aeolus Level-2B (L2B) Horizontal Line-of-Sight (HLOS) winds, systematic differences (referred as biases hereafter) between the observations and numerical weather prediction (NWP) background winds should be removed. Total least squares (TLS) regression is used to estimate speed-dependent biases between Aeolus HLOS winds (L2B10) and the National Oceanic and Atmospheric Administration (NOAA) Finite-Volume Cubed-Sphere Global Forecast System (FV3GFS) 6-h forecast winds. Unlike ordinary least squares regression, TLS regression optimally accounts for random errors in both predictors and predictands. Large well-defined, speed-dependent biases are found particularly in the lower stratosphere and troposphere of the tropics and Southern Hemisphere. These large biases should be corrected to increase the forecast impact of Aeolus data assimilated into global NWP systems.
Hui Liu et al.
Status: closed
-
RC1: 'Comment on amt-2022-20', Anonymous Referee #1, 09 Feb 2022
This manuscript describes the additional bias correction applied to the first reprocessed HLOS winds (B10) that has been developed for the assimilation of Aeolus data in the NOAA FV3GFS model. It is first shown that there are important remaining mean differences between the B10 HLOS winds (O) and corresponding FV3GFS background values (B), particularly in the tropics. These mean differences (referred to as biases in this study), vary with latitude, height, HLOS wind speed and orbit phase. It is assumed that these biases vary linearly with wind speed. A total least squares (TLS) regression is used to calculate the biases for each orbit phase, vertical layer and 10-degree latitude band over a period of 7 days for both Mie and Rayleigh winds. The TLS regression takes into account of the relative error variance between the FV3GFS background and Aeolus HLOS winds, leading to more optimal bias estimates than those from ordinary least squares regression. The error variances are estimated using the Hollingsworth-Lonnberg method, which is also based on O and B statistics.
Two important aspects are missing in this manuscript in order to make it standalone: the merit of using the Hollingworth-Lonnberg method for estimating the error variance ratio and the impact of the proposed bias correction scheme on forecasts. These two aspects are reported in a companion paper recently submitted to QJRMS by the same authors. However, since the manuscript is relatively short, I suggest either merging it with the companion paper or expanding the manuscript by including more details on the Hollingworth-Lonnberg method and by adding some forecast impact results. There are also a number of places in the manuscript where clarifications are needed, as described in the following comments and suggestions.
Lines 62-65 : ‘…global NWP models still have larger errors in regions where conventional observations are sparse…’ Is this statement based on the difference between analyses or short-range forecasts from different NWP centers? Which type of error is most important in these regions? It is true that ‘…NWP models evolve towards their own climatology in the absence of observations…’. However, the various geostationary and polar orbiting satellites now provide nearly global coverage of MW and IR radiances in the troposphere and lower stratosphere every 6 hours such that NWP models do not evolve towards their own climatology but towards the analysis largely constrain by the assimilation of satellite radiances and derived products. Please clarify this sentence.
Lines 67-70 : It is not clear what it is shown in Figure 1. In 4D data assimilation systems, such as 4D-Var and 4D-EnVar, the background is a time series of forecast fields over the assimilation window used to calculate O-B at the ‘appropriate’ observation time. Which forecast lead-time is shown in Figure 1?
Line 79 : The M1 bias correction scheme makes use of the ECMWF O-B distributed over 12-h assimilation windows, not 6-h forecasts. It is true that the use of ECMWF background in the M1 bias correction scheme may have a detrimental effect on the assimilation of Aeolus winds in other NWP systems. However, this effect is expected to be small since the regression (M1 temperatures vs O-B) is made over the globe, not locally.
Lines 83, 96, 104, 136, 295, 300, 324, 419, 432 : The definition of innovation in the meteorological data assimilation field is O-B. Stating ‘innovations of observation minus background’ is redundant, as well as ‘O-B innovations’.
Line 99 : Change Section 0 by Section 2.
Lines 119-124: This paragraph described the quality control applied to the Aeolus winds as recommended by Rennie et al. (2021). It is however not mentioned whether or not an additional quality control (e.g. background check) is applied to remove remaining outliers. Could you elaborate on this?
Line 167-170 : It is assumed in this study that the bias vary linearly with background HLOS wind speed. This is not well justified and needs to be further discussed in this manuscript. For instance, recent work by Marseille et al.(2021) (NWP calibration applied to Aeolus Mie channel winds - Marseille - - Quarterly Journal of the Royal Meteorological Society - Wiley Online Library ) has shown that the bias for the Mie winds is complex and varies nonlinearly with wind speed. In this context, how can a linear model for the HLOS wind bias be justified?
Line 187 : It is true that the OLS regression assumes that the predictors (B, O or (B+O)/2 here) are free of errors. However, the OLS regression cannot be formulated using the same TLS cost function and by setting sigma O or sigma B to zero. Instead of elaborating more on this, I suggest removing this statement since the OLS regression is described in Section 4.
Line 190 : Why are the estimated L2B standard deviation errors for the HLOS winds not used for specifying sigma O?
Lines 192-195 : The HL method also assumes that the innovations are unbiased. How do you proceed to remove the biases in the innovations before applying the HL method to calculate the error variance ratio, which is then used in the TLS regression for estimating the wind-dependent biases?
Section 2.3 : More details on the HL method are needed in this section if this manuscript is not combined with the companion article by Garrett et al. (2021).
Section 3 : This section describes the variation of the bias estimations with orbit phase, latitude, height and wind speed. However, for each orbit phase, the TLS regression is applied only as a function of vertical layer (Section 3.1) and only as a function of latitude band (Section 3.2). On the other hand, the TLS regression is applied as a function of both latitude band and vertical layer in Section 5 to estimate the bias correction values to be used in the data assimilation experiments. Since TLS regressions are not applied the same way in Sections 3 and 5, how can the results shown in Section 3 be representative of the bias estimates obtained in Section 5? In addition, since the variance ratio vary with height, it is not clear how the TLS regression is calculated in Section 3.2.
Section 4 and Figure 12 : The differences between the TLS and the various OLS regressions are large, although the same linear model is used. Do you have an explanation for that? Moreover, why the biases are underestimated (overestimated) when using FV3GFS background (Aeolus) HLOS winds as predictors?
Section 5 : In the case this manuscript is not merged with the article by Garrett et al. (2021), I suggest adding a paragraph describing the impact of the proposed bias correction scheme on forecasts. This is important aspect to present since the main reason to develop a bias correction scheme is to remove the detrimental effect of biases on forecasts.
Line 360 : Publication year : 2000
Figure 9 : There are spikes in the data counts around zero-wind speed for both Mie (a) and Rayleigh (b) winds. What is the cause of this significant increase in the number of observations near this wind speed?
Figures 4 and 14 : The contour interval (0.5 m/s) used in these figures is coarser than the contour interval (0.3 m/s) used in the other figures (8, 11, 12, 13). I suggest using the same 0.3 m/s interval in Figures 4 and 14. This will improve the clarity of Figure 4 and make easier the comparison of the various panels.
Figure 12 : For a better assessment of the quality of the fit, I suggest adding panels showing the average O-B for both Mie and Rayleigh winds. These results are expected to be close to the average bias estimates from the TLS regression. This will prove that the best results are indeed obtained by this regression method.
Figure 14 : What is the minimum number of samples used for the TLS regression? The remaining mean differences for the Mie winds above 10 km and south of 60S (Fig. 14c) look rather noisy and the remaining pattern does not look similar to the corresponding one in Figure 4. What is the reason for this?
- AC1: 'Reply on RC1', Hui Liu, 09 Apr 2022
-
RC2: 'Comment on amt-2022-20', Anonymous Referee #2, 02 Mar 2022
My advice and comments are in the supplemented pdf document
- AC2: 'Reply on RC2', Hui Liu, 09 Apr 2022
Status: closed
-
RC1: 'Comment on amt-2022-20', Anonymous Referee #1, 09 Feb 2022
This manuscript describes the additional bias correction applied to the first reprocessed HLOS winds (B10) that has been developed for the assimilation of Aeolus data in the NOAA FV3GFS model. It is first shown that there are important remaining mean differences between the B10 HLOS winds (O) and corresponding FV3GFS background values (B), particularly in the tropics. These mean differences (referred to as biases in this study), vary with latitude, height, HLOS wind speed and orbit phase. It is assumed that these biases vary linearly with wind speed. A total least squares (TLS) regression is used to calculate the biases for each orbit phase, vertical layer and 10-degree latitude band over a period of 7 days for both Mie and Rayleigh winds. The TLS regression takes into account of the relative error variance between the FV3GFS background and Aeolus HLOS winds, leading to more optimal bias estimates than those from ordinary least squares regression. The error variances are estimated using the Hollingsworth-Lonnberg method, which is also based on O and B statistics.
Two important aspects are missing in this manuscript in order to make it standalone: the merit of using the Hollingworth-Lonnberg method for estimating the error variance ratio and the impact of the proposed bias correction scheme on forecasts. These two aspects are reported in a companion paper recently submitted to QJRMS by the same authors. However, since the manuscript is relatively short, I suggest either merging it with the companion paper or expanding the manuscript by including more details on the Hollingworth-Lonnberg method and by adding some forecast impact results. There are also a number of places in the manuscript where clarifications are needed, as described in the following comments and suggestions.
Lines 62-65 : ‘…global NWP models still have larger errors in regions where conventional observations are sparse…’ Is this statement based on the difference between analyses or short-range forecasts from different NWP centers? Which type of error is most important in these regions? It is true that ‘…NWP models evolve towards their own climatology in the absence of observations…’. However, the various geostationary and polar orbiting satellites now provide nearly global coverage of MW and IR radiances in the troposphere and lower stratosphere every 6 hours such that NWP models do not evolve towards their own climatology but towards the analysis largely constrain by the assimilation of satellite radiances and derived products. Please clarify this sentence.
Lines 67-70 : It is not clear what it is shown in Figure 1. In 4D data assimilation systems, such as 4D-Var and 4D-EnVar, the background is a time series of forecast fields over the assimilation window used to calculate O-B at the ‘appropriate’ observation time. Which forecast lead-time is shown in Figure 1?
Line 79 : The M1 bias correction scheme makes use of the ECMWF O-B distributed over 12-h assimilation windows, not 6-h forecasts. It is true that the use of ECMWF background in the M1 bias correction scheme may have a detrimental effect on the assimilation of Aeolus winds in other NWP systems. However, this effect is expected to be small since the regression (M1 temperatures vs O-B) is made over the globe, not locally.
Lines 83, 96, 104, 136, 295, 300, 324, 419, 432 : The definition of innovation in the meteorological data assimilation field is O-B. Stating ‘innovations of observation minus background’ is redundant, as well as ‘O-B innovations’.
Line 99 : Change Section 0 by Section 2.
Lines 119-124: This paragraph described the quality control applied to the Aeolus winds as recommended by Rennie et al. (2021). It is however not mentioned whether or not an additional quality control (e.g. background check) is applied to remove remaining outliers. Could you elaborate on this?
Line 167-170 : It is assumed in this study that the bias vary linearly with background HLOS wind speed. This is not well justified and needs to be further discussed in this manuscript. For instance, recent work by Marseille et al.(2021) (NWP calibration applied to Aeolus Mie channel winds - Marseille - - Quarterly Journal of the Royal Meteorological Society - Wiley Online Library ) has shown that the bias for the Mie winds is complex and varies nonlinearly with wind speed. In this context, how can a linear model for the HLOS wind bias be justified?
Line 187 : It is true that the OLS regression assumes that the predictors (B, O or (B+O)/2 here) are free of errors. However, the OLS regression cannot be formulated using the same TLS cost function and by setting sigma O or sigma B to zero. Instead of elaborating more on this, I suggest removing this statement since the OLS regression is described in Section 4.
Line 190 : Why are the estimated L2B standard deviation errors for the HLOS winds not used for specifying sigma O?
Lines 192-195 : The HL method also assumes that the innovations are unbiased. How do you proceed to remove the biases in the innovations before applying the HL method to calculate the error variance ratio, which is then used in the TLS regression for estimating the wind-dependent biases?
Section 2.3 : More details on the HL method are needed in this section if this manuscript is not combined with the companion article by Garrett et al. (2021).
Section 3 : This section describes the variation of the bias estimations with orbit phase, latitude, height and wind speed. However, for each orbit phase, the TLS regression is applied only as a function of vertical layer (Section 3.1) and only as a function of latitude band (Section 3.2). On the other hand, the TLS regression is applied as a function of both latitude band and vertical layer in Section 5 to estimate the bias correction values to be used in the data assimilation experiments. Since TLS regressions are not applied the same way in Sections 3 and 5, how can the results shown in Section 3 be representative of the bias estimates obtained in Section 5? In addition, since the variance ratio vary with height, it is not clear how the TLS regression is calculated in Section 3.2.
Section 4 and Figure 12 : The differences between the TLS and the various OLS regressions are large, although the same linear model is used. Do you have an explanation for that? Moreover, why the biases are underestimated (overestimated) when using FV3GFS background (Aeolus) HLOS winds as predictors?
Section 5 : In the case this manuscript is not merged with the article by Garrett et al. (2021), I suggest adding a paragraph describing the impact of the proposed bias correction scheme on forecasts. This is important aspect to present since the main reason to develop a bias correction scheme is to remove the detrimental effect of biases on forecasts.
Line 360 : Publication year : 2000
Figure 9 : There are spikes in the data counts around zero-wind speed for both Mie (a) and Rayleigh (b) winds. What is the cause of this significant increase in the number of observations near this wind speed?
Figures 4 and 14 : The contour interval (0.5 m/s) used in these figures is coarser than the contour interval (0.3 m/s) used in the other figures (8, 11, 12, 13). I suggest using the same 0.3 m/s interval in Figures 4 and 14. This will improve the clarity of Figure 4 and make easier the comparison of the various panels.
Figure 12 : For a better assessment of the quality of the fit, I suggest adding panels showing the average O-B for both Mie and Rayleigh winds. These results are expected to be close to the average bias estimates from the TLS regression. This will prove that the best results are indeed obtained by this regression method.
Figure 14 : What is the minimum number of samples used for the TLS regression? The remaining mean differences for the Mie winds above 10 km and south of 60S (Fig. 14c) look rather noisy and the remaining pattern does not look similar to the corresponding one in Figure 4. What is the reason for this?
- AC1: 'Reply on RC1', Hui Liu, 09 Apr 2022
-
RC2: 'Comment on amt-2022-20', Anonymous Referee #2, 02 Mar 2022
My advice and comments are in the supplemented pdf document
- AC2: 'Reply on RC2', Hui Liu, 09 Apr 2022
Hui Liu et al.
Hui Liu et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 430 | 74 | 17 | 521 | 6 | 8 |
- HTML: 430
- PDF: 74
- XML: 17
- Total: 521
- BibTeX: 6
- EndNote: 8
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1