In this work, we apply a principal component analysis (PCA)-based approach
combined with lookup tables (LUTs) of corrections to accelerate the Vector Linearized Discrete Ordinate Radiative Transfer (VLIDORT) model used in the retrieval of ozone profiles from
backscattered ultraviolet (UV) measurements by the Ozone Monitoring
Instrument (OMI). The spectral binning scheme, which determines the accuracy
and efficiency of the PCA-RT performance, is thoroughly optimized over the
spectral range 265 to 360 nm with the assumption of a Rayleigh-scattering
atmosphere above a Lambertian surface. The high level of accuracy
(

Optimal-estimation-based inversions have become standard for the retrieval
of atmospheric ozone profiles from atmospheric chemistry UV and visible (UV–Vis) backscatter
instruments. This inversion model requires iterative simulations of not only
radiances, but also of Jacobians with respect to atmospheric and surface
variables, until the simulated radiances are sufficiently matched with the
measured radiances. These ozone profile algorithms face a computational
challenge for use in global processing of high spatial–temporal resolution
satellite measurements, due to online radiative transfer (RT) computations
at many spectral points from 270 to 330 nm; it is computationally very
expensive to perform full multiple-scattering (MS) simulations with the
polarized RT model. To reduce the computational cost, a scalar RT model can
be applied together with a polarization correction scheme based on a lookup
table (LUT) (Kroon et al., 2011; Miles et al., 2015). Another approach is to
carry out online vector calculations at a few wavelengths (Liu et al.,
2010) together with other approximations (e.g., low-stream, coarse vertical
layering, Lambertian reflectance for surface and cloud, no aerosol
treatment). However, the computational speed is still insufficient to
process 1 d of measurements from the Aura Ozone Monitoring Instrument
(OMI) within 24 h (30 cross-track pixels

The goal of this paper is to improve both computational efficiency and
accuracy of RT simulations in the OMI ozone profile algorithm (Liu et al.,
2010) by combining a fast-PCA-based RT model with two kinds of correction
techniques. The application of PCA to RT simulations was first proposed by
Natraj et al. (2005) by demonstrating a computational improvement of
intensity simulation in the O

This paper is structured as follows. Section 2 describes the current forward model scheme and evaluates the approximations made in RT calculations, with the determination of the configuration parameters for accurate simulations. The updated forward model scheme is introduced for the PCA-based RT model in Sect. 3.1, and the two kinds of correction schemes to use fewer spectral samples and a less accurate RT configuration are detailed in Sect. 3.2. The evaluation is performed in Sect. 4, and then we summarize and discuss the results in Sect. 5.

We first describe the current v1 SAO OMI ozone profile algorithm that was
implemented in OMI Science Investigator-led Processing Systems (SIPS) to
generate the research OMPROFOZ ozone profile product, publicly available at
the Aura Validation Data Center (AVDC,

Schematic flowcharts of VLIDORT (v1) and PCA-VLIDORT (v2) based
forward models, respectively. Note that VLIDORT was used in the generation
of the OMPROFOZ v1 dataset, while PCA-VLIDORT is in preparation for OMPROFOZ
v2 production. The number of wavelengths used in each process is denoted as

In the first step, we select 93 wavelengths with variable sampling
intervals, 1.0 nm below 295 nm, 0.4 nm from 295–310 nm, and 0.6 nm above 310 nm. The number of these wavelengths is smaller than the OMI native pixels
(229 from 270–330 nm) by more than a factor of 2. The online radiative
transfer model is run to generate the full radiance spectrum (single

Figure 2a shows the reference spectrum where Gaussian smoothing to 0.4 nm is
applied to LBL calculations at the sampling rate (0.01 nm) of the ozone
cross sections (Brion et al., 1993), which is used to evaluate the
approximation errors related to undersampling. Figure 2b illustrates that LBL
calculations are required to be performed at intervals of 0.03 nm or better.
The undersampling correction applied in step 3 allows relaxation of the
sampling rate without loss of the accuracy. This correction is based on the
adjustment of the radiance due to the difference of the optical depth
profiles between fine (

Errors of the radiance simulation due to the RT approximation used
in v1, arising from

The right panel of Fig. 1 illustrates the flowchart of the updated forward model scheme (v2) which employs the PCA-based RT model to perform online scalar simulations using four streams and a 24-layer atmosphere for RT performance enhancement (step 1) and two kinds of correction schemes for accounting for approximation errors (steps 2 and 3). Section 3.1.1 gives an overview on how the PCA tool is combined with the VLIDORT version 2.8 model; full theoretical details may be found in Spurr et al. (2016) and Kopparla et al. (2017). Here, our paper gives details on how the PCA-based RT configuration is optimized for the application to UV ozone profile retrievals for maximizing the speed-up in Sect. 3.1.2. Section 3.2 specifies step 2, wherein the LUT-based correction is applied to approximation errors due to the use of a scalar model, a smaller number of streams, and coarser-resolution vertical grid. In step 3 the undersampling correction is adopted from the v1 implementation, but the Rayleigh scattering term of the Eq. (1) is neglected for the speed up with trivial loss of accuracy.

The PCA-based RT process begins with a grouping of spectral points into
several bins; atmospheric profile optical properties within each bin are
similar. PCA is a mathematical transformation that converts a correlated
mean-subtracted dataset into a series of principal components (PCs). To
enhance RT performance, PCA is used to compress a binned set of correlated
optical profile data into a small set of atmospheric profiles which capture
the vast majority of the data variance within the bin. The layer extinction
optical thickness

In the PCA-based RT package, three independent RT models are combined in
order to generate the full-scattering intensity field
(

So far, we have discussed generation of total intensity field, using values

Of greater importance for us is the need to derive PCA-RT approximations to
profile Jacobians (weighting functions of the total intensity with respect
to ozone profile optical depths). A PCA-RT Jacobians scheme was developed by
Spurr et al. (2013) for total column Jacobians in connection with the
retrieval of total ozone; this scheme involved formal differentiation of the
entire PCA-RT system as outlined above for the intensity field. This is
satisfactory for bulk property Jacobians, but for profile Jacobians it is
easier to write (Efremenko et al., 2014; Spurr et al., 2016)

The major performance saving is achieved by limiting full-MS VLIDORT
calculations to those based on the reduced set of PCA-derived optical states

Optical properties within each bin must be strongly correlated to reduce the
number of EOFs required to attain a given accuracy. According to Kopparla et
al. (2016), the UV region is divided at 340 nm, beyond which O

The PCA-RT configuration optimized over the UV spectral range
265–360 nm. The optical depth of the total gas column (

To evaluate the PCA approximation, the “exact-RT” model is performed, where accurate full-MS VLIDORT calculations are expensively performed at every
wavelengths in addition to accurate SS calculations:

Residuals (%) of the PCA-RT radiance in the wavelength range
265–340 nm compared to the exact-RT calculations, for different binning
steps (different colors) and number of EOFs. Results are plotted as a function of

Same as Fig. 4 but for different windows:

Residuals (%) of the PCA-RT radiances with the binning scheme
given in Table 1 for various sets of

Two sets of LUTs are created: for high-accuracy
(

LUT parameter specification. Note that the relative azimuth dependence is taken into account explicitly through the Fourier coefficients of path radiance (Table 3), and the surface albedo dependence is taken into account by the planetary problem.

LUT variable specification.

Comparisons of radiance simulations at VZA

Example of LUT-based correction spectrum.

The PCA-RT model developed as described in this paper is implemented as the forward model component of an iterative optimal-estimation-based inversion (Rodgers, 2000) for retrieving the ozone profile from OMI measurements. In previous studies, the PCA-RT performance was evaluated against a suite of exact monochromatic baselines of fully accurate VLIDORT simulations. However, such exact RT calculations cannot be applied in the operational data processing system, especially when thousands of spectral points are involved; in other words, the operational capability of the PCA-RT approach has been overestimated in previous studies. Therefore, we evaluate the RT model developed against the existing forward model where many RT approximations are applied to meet the computational budget in the operational system.

List of configurations used in evaluating the different forward model calculations for OMI ozone profile retrievals. The reference, VLIDORT, and PCA-RT models are abbreviated as Ref, VLD, and PCA, respectively.

Mean biases of ozone profile retrievals with different configurations compared to those with the reference configuration. Each configuration is given in Table 4.

Same as Fig. 9 but for individual differences. VLD and PCA represent v1 and v2 forward model configurations, respectively.

Same as Fig. 10 but for

Table 4 contains sets of configurations for seven forward models. OMI spectra
are simulated at the undersampled (US) intervals specified in the first
column of this table and then interpolated at high-resolution (HR)
intervals (second column) with the undersampling correction before
convolution with OMI slit functions. In the v1 forward model, the US
spectral intervals were set at 1.0 nm/0.4 nm intervals below/above 295
and 0.6 nm above 310 nm, while the HR spectral interval was set at 0.05 nm.
In the updated RT model, the spectral points are selected at 0.3 nm (0.1 nm)
intervals below (above) 305 nm, and the HR interval is set as 0.03 nm, which
enables us to achieve very high accuracy, better than 0.01 %, as shown in
Fig. 2c. In the reference configuration (abbreviated to “Ref”), VLIDORT
is run in vector mode with 12 streams and 72 atmospheric layers so that the
RT approximation errors are significantly reduced. The VLIDORT-based forward
model is run with five sets of configurations (abbreviated to VLD in
Table 4) to quantify the impact of RT approximations on ozone retrievals.
Figure 9 compares the mean biases of the retrieved ozone profiles between
VLD/PCA and Ref for three SZA regimes. VLD

We have extended the PCA-based fast-RT method to improve computational challenges for OE-based SAO OMI ozone profile retrievals requiring iterative calculations of the radiance and its Jacobian derivatives. The PCA-RT model is designed to perform MS calculations for a few EOF-derived optical states which are developed from spectrally binned sets of inherent optical properties that possess some redundancy. In this study, the binning scheme is carefully turned for the UV ozone fitting window from 265 to 360 nm in such a way as to choose the number of EOFs to be as small as possible for each bin rather than always using the first four EOFs for all bins selected in previous studies. The spectral windows are divided into three sub-windows: (1) 265–340 nm, (2) 340–350 nm, and (3) 350–360 nm. Then, optical profiles are grouped into bins according to criteria based on the total gas optical depth, as specified in Table 1. We demonstrated that the PCA approximation errors for our application are within 0.03 % for any viewing geometry, optical depth profile, and vertical layering.

The existing (v1) forward model calculations are evaluated to determine the
optimal configuration for the v2 forward model. RT approximation errors
exist due to the use of 24 quite coarse vertical layers (2.5 km thick),
which can cause radiance simulation errors of up to

OMI Level 1b radiance datasets are available at

JB and XL designed the research; RS provided oversight and guidance for using both VLIDORT and PCA-based VLIDORT; KY developed the LUT creation and interpolation scheme; XL contributed to analyzing ozone profile retrievals with different forward model approaches; JB conducted the research and wrote the paper; CRN, CCM, GGA, and KC contributed to the analysis and writing; CCM and GGA contributed to managing the computational resources.

The authors declare that they have no conflict of interest.

We acknowledge the OMI science team for providing their satellite data.
Research at the Smithsonian Astrophysical Observatory is funded by the NASA
Aura science team program (NNX17AI82G). Research at Pusan National
University is funded by the Basic Science Research Program through the National
Research Foundation of Korea (NRF) funded by the Ministry of
Education(2020R1A6A1A03044834). Both calculations and simulations are done on the Smithsonian Institution High-Performance Cluster (SI/HPC) computer system (

This research has been supported by the NASA Aura science team program (grant no. NNX17AI82G) and the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (grant no. 2020R1A6A1A03044834).

This paper was edited by Cheng Liu and reviewed by two anonymous referees.