Linearisation of the effects of spectral shift and stretch in DOAS analysis

Differential Optical Absorption Spectroscopy (DOAS) is a widely used method to quantify atmospheric trace gases from spectroscopic measurements. While DOAS can, in principal, be described by a linear equation system, usually nonlinearities occur, in particular as a consequence of spectral misalignments. Here we propose to linearise the effects of a spectral shift by including a “shift spectrum”, which is the first term of a Taylor expansion, as pseudo-absorber in the DOAS fit. The effects of a spectral stretch are considered as additional wavelength-dependent shifts. Solving the DOAS equation system linearly has several advantages: the solution is unique, the algorithm is robust, and it is very fast. The latter might be particularly important for measurements with high data rates, like for upcoming satellite missions.


Introduction
Differential Optical Absorption Spectroscopy (DOAS) (Platt, 1994) is an established method for the analysis of atmospheric composition.Based on the Beer-Lambert law, and making use of the characteristic absorption features of various trace gases in the UV/vis spectral range, several important trace gases, like ozone (O 3 ) or nitrogen dioxide (NO 2 ), can be quantified.For a comprehensive description of the DOAS method and its application, see e.g., Platt (1994); Platt and Stutz (2008); Richter and Wagner (2011), and references therein.
The DOAS approach can be formulated in terms of optical depth (OD) τ as (compare Richter and Wagner, 2011, Eq. 2.10).I denotes the measured intensity.I 0 is the "reference spectrum", meaning either the intensity of an artificial light source (active DOAS), the solar irradiance (satellite DOAS), or even a spectrum of scattered sun light (MAX-DOAS) (see Sect. 2).σ i is the wavelength-dependent absorption cross-section of the ith relevant trace gas, and s i is the respective slant column density (SCD), i.e., the concentration integrated along the effective light path.P is a closure polynomial which accounts for broad-band wavelength dependencies of Rayleigh or Mie scattering, surface reflectance, etc.Note that besides molecular absorptions, further phenomena can affect the spectral structures of the OD, as, for instance, the so-called "Ring effect" (Solomon et al., 1987), caused by inelastic Raman scattering.In practice, such spectral structures are often accounted for by introducing additional "pseudo-absorbers" like a "Ring spectrum" (Solomon et al., 1987;Chance and Spurr, 1997).Other common pseudo-absorbers are an "undersampling spectrum" to account for the effects of spectral undersampling (Chance, 1998;Slijkhuis et al., 1999), or the inverse irradiance, to account for an additive stray-light offset (Noxon et al., 1979).Mathematically, such pseudo-absorbers have the same functionality as the cross-sections in the fitting procedure, while physical meaning and units of the fit coefficients are different.

S. Beirle et al.: Linearised shift and stretch in DOAS
For measured spectra of I and I 0 , Eq. ( 1) results in a system of N linear equations with M unknowns, where N is the number of detector pixels in the considered wavelength range and M is the sum of (pseudo-)absorbers and polynomial coefficients.If overdetermined (N M), this system can easily be solved by simple Matrix operations, e.g., by calculating the pseudo-inverse, which is mathematically equivalent to a least-squares minimisation (Williams, 1990).
However, if spectral misalignments occur, i.e., if the wavelengths assigned to the detector pixels do not match for I , I 0 , or the involved cross-sections, the SCDs resulting from a linear algorithm are systematically biased (Stutz and Platt, 1996).Such spectral misalignments could be caused by inaccurate reference cross-sections σ i , temperature changes of the spectrometer (I ), instabilities of the light source (I 0 , for active DOAS), etc. Section 2 lists typical shifts for different DOAS applications.
To account for such spectral misalignments, it is common practice to allow for a spectral shift and stretch/squeeze as additional free parameters within the DOAS analysis.This approach can reduce systematic errors significantly, but with these additional parameters, the system is not linear any more.The solution (or simply "fit" hereafter) requires non-linear least square minimisation algorithms, like the iterative Gauss-Newton or Levenberg-Marquardt algorithms (Marquardt, 1963).Compared to the simple and unique solution of the linear equation system, the non-linear fit has several disadvantages: -It is considerably slower, as it requires the calculation of partial derivatives for all parameters in each iteration step.
-It is not unique and the fit results depend on the predefined termination criteria.
-In a non-linear system, local minima can exist.Consequently, the fit results can depend on the initial values of the state vector, and the fit might even diverge for an offside initial state vector.
The aspect of speed might become particularly relevant for future satellite missions like TROPOMI (Veefkind et al., 2012) with high data rates (∼ 12 GB per orbit, or some hundred spectra per second).
Here we propose a linearisation scheme to account for spectral misalignments (shift and stretch) between I and I 0 , similar to the approach used by Rozanov et al. (2011).
After a short summary of DOAS applications and their typical spectral misalignments (Sect.2), we investigate the spectral patterns caused by a shift or stretch by performing a Taylor expansion of I .In first order, the linear term can be simply added as pseudo-absorber to the DOAS Eq. (1), while maintaining its linearity (Sect.3).The accuracy and performance of our proposed linearisation are investigated exemplarily for the retrieval of NO 2 in the blue spectral range, and of BrO in the UV, in Sect. 4. In Sect.5, we discuss various aspects of our approach and sum up our findings in the conclusions.

DOAS applications and their typical shifts
Here we shortly summarise different DOAS applications insofar as related to this study.For details see e.g., Platt and Stutz (2008).
Various effects could physically affect the pixelwavelength allocation of the detector, like contraction/expansion due to temperature changes, or the Doppler effect for moving spectrometers.In addition, other effects (like interfering temperature effects on grating, dark current, and gain), might cause spectral structures similar to those of a spectral misalignment.In this study, we investigate how far the effects of spectral misalignments can be linearised.For this, the actual cause of the (physical or virtual) misalignment is not relevant.
Typical misalignments are small (e.g., about 0.01 nm) compared to the measured spectral structures, which are determined by the detector resolution (e.g., about 0.55 nm FWHM for OMI).Nevertheless, they can impair the DOAS retrieval of trace gases if being ignored (see Sect. 4).

Active DOAS
For an active DOAS system, an artificial light source, like a Xenon arc lamp, or, recently, also a LED, is used.The effects of spectral misalignments depend on the high-frequency structures of the lamp spectrum, which are usually rather small compared to the Fraunhofer lines in solar spectra.
Spectral shifts for active DOAS systems can be caused, for example, by temperature drifts of the spectrometer or instabilities of the light source, and can reach values of ∼ 0.02 nm.Computational costs, however, are generally not critical for active DOAS measurements, due to the limited data sets.Thus, we focus on passive DOAS applications in this study.

Passive DOAS applications
Passive DOAS set-ups use the sun as light source.Thus, the measured spectra of scattered sunlight are usually dominated by Fraunhofer lines.This allows one to create a calibrated set of references, consisting of the reference spectrum I 0 , the absorption cross-sections, and the pseudo-absorbers.This calibration involves the following steps, which can be easily realised by DOAS software platforms as DOASIS (Kraus, 2005) or WinDOAS (Fayt and van Roozendael, 2001): -A reference spectrum I 0 (measured by the spectrometer) is chosen.
-I 0 is calibrated (i.e., the wavelength-pixel allocation is determined) by means of a well-calibrated, high-resolution solar spectrum (e.g.Chance and Kurucz, 2010), making use of the dominant Fraunhofer lines.
The instrument spectral response function (ISRF) has to be known; if necessary, a parameterised (e.g., Gaussian) ISRF can be fitted during calibration by e.g., DOASIS or WinDOAS.
-According to this calibration, laboratory absorption cross-sections are convolved and interpolated on the spectrometer's pixel grid.
-Pseudo-absorbers, in particular the Ring spectrum and the inverse ( 1 I 0 ) in order to account for stray light, are derived from I 0 .

Zenith-sky DOAS
Zenith-sky measurements from ground-based spectrometers provide time series of (mostly stratospheric) trace gases.The solar reference is typically taken from noon-time measurements (Note that this is not a real solar spectrum, but is already affected by atmospheric absorption.The fitted SCDs are, thus, differential SCDs with respect to "I 0 ").If I 0 is used for a time-series of several days to months, typical shifts are of the order of 0.001 up to 0.01 nm, respectively.If daily solar references are used, typical shifts are generally < 0.002 nm.

MAX-DOAS
Multi-Axis (MAX) DOAS measurements provide additional information on surface-near vertical trace gas profiles (e.g.Hönninger et al., 2004;Wagner et al., 2011).Usually a zenith sky measurement is used as solar reference.If I 0 is applied to a time-series of several days to months, typical shifts are of the order of 0.002 to 0.01 nm, respectively.If daily solar references are used, or even a separate I 0 for each elevation angle sequence, shifts are generally < 0.002 nm.

Satellite observations
Several spectrometers with moderate spectral resolution are in orbit, e.g., GOME, SCIAMACHY, GOME-2, or OMI, providing global information on various atmospheric trace gases (e.g.Wagner et al., 2008;Martin, 2008).Direct solar measurements are taken by these spectrometers typically once per day.Due to the high speed of the satellites, spectral misalignments are dominated by the Doppler-effect (Slijkhuis et al., 1999), which causes shifts of about 0.01 nm for λ = 440 nm between the earth-shine measurements and the solar reference I 0 , which is measured in flight direction.The spectral shifts on top of this overall offset are typically below ≈ 0.003 nm.

Method
Based on Eq. (1), the DOAS fit is equivalent to a simple linear equation system, as long as spectral alignment is given.Here, we investigate the spectral structures caused by a spectral misalignment between I and I 0 , and derive pseudoabsorbers which account for these effects in first (linear) order.A potential misalignment of a cross-section σ i can not be linearised in analogous form, as shown in Appendix A.

Spectral shift
To investigate the effects of a spectral shift, i.e., a simple offset of λ in the wavelength-pixel allocation between I and I 0 , we reformulate Eq. (1) as ln(I ) − ln(I 0 ) = s i σ i + P . (2) If the cross-sections σ i and I 0 are spectrally calibrated on the detector's grid (see Sect. 2), but I is shifted by λ, artificial spectral structures are created which can be approximated by a Taylor expansion: O(2) denotes second and higher order terms.The errors introduced by the linearisation are quantified by the second term of the Taylor expansion below.Thus, a spectral shift λ of I causes, in first order, a spectral structure which is proportional to This "shift-spectrum" A Shift , i.e., the derivative of ln(I (λ)), can now be included as pseudo-absorber to the equation system, while maintaining its linearity.The respective fit coefficient directly yields the shift λ.A similar approach was used in Rozanov et al. (2011, Eq. 20) for the retrieval of BrO profiles from SCIAMACHY limb measurements.We illustrate the effect of a spectral shift on I for a simple Gaussian absorption band at λ 0 : where the subscript "SB" stands for "single band".a and ς are the band depth and width, respectively, and λ is the wavelength relative to the band centre: Figure 1a displays I SB (black) for a = 0.2 and ς = 5 pixel, and I SB shifted by 1 pixel (blue).Figure 1b shows, in blue, the spectral structure (in terms of OD) caused by the shift, calculated as ln(I shifted ) − ln(I ).In green, the respective structure resulting from the linearisation in Eq. ( 3) is shown, i.e., the pseudo-absorber A Shift as defined in Eq. ( 4), scaled by the applied shift.
For I SB , the shift-spectrum can directly be calculated: It has its maximum/minimum at λ = ±ς, where A Shift is ± a ς e − 1 2 (compare Fig. 1b).To first order, a spectral shift of λ, thus, causes spectral structures (peak-to-peak OD) of (max(A Shift ) − min(A Shift )) × λ: i.e., they are proportional to the shift itself, to the depth of the absorption lines, and reciprocal to the width of the absorption lines.In other words, deep narrow absorption lines are most critical for a spectral misalignment.For the sample values (a = 0.2, ς = 5 and λ = 1), Eq. ( 9) yields a peak-to-peak OD of 5 %, in agreement with Fig. 1.
It can be seen that A Shift reproduces well the spectral structures caused by the shift.The remaining errors, i.e., the difference of the curves in Fig. 1b, are caused by the neglect of higher orders in Eq. (3).The second order term of the Taylor expansion is This function is maximum at λ = 0 and minimum at λ = ± √ 3ς .Thus, (Eq.5), the peak-to-peak distance is The second-order peak-to-peak effects of a spectral shift are thus For the example shown in Fig. 1, this is ≈ 0.6 %, i.e., about one order of magnitude lower than the first-order term accounted for by A Shift .These orders of magnitudes as derived for a general absorption band (Eqs.9 and 12) also proved to be meaningful for complex spectra (see Sect. 4).

Spectral stretch
In addition to a spectral shift, the wavelength-pixel allocations might also be linearly transformed by a factor q, i.e., stretched (q > 1) or squeezed (q < 1) (below we use the generic term "stretch" in a general meaning for both q > 1 and q < 1).Here we define the stretch q with respect to the central wavelength of the fitting window λ 0 as: By this definition of the stretch-parameter q, the stretch does not affect the central wavelength of the fitting window, i.e., overall, it does not introduce an additional shift.However, locally, such a stretch is equivalent to a (wavelength dependent) shift: at a given wavelength, the spectral displacement D can be expressed as i.e., locally, a shift of (q − 1)(λ − λ 0 ) is added to the overall shift λ.Similarly to λ, this additional shift causes, in first order, spectral structures of A Shift (λ − λ 0 )(q − 1).Thus, a spectral stretch can be accounted for by adding the pseudoabsorber ("stretch-spectrum") to the linear equation system.The respective fit coefficient is (q − 1). Figure 2 shows the effects of a spectral stretch and the stretch spectrum analogously to Fig. 1.In principle, also higher orders of spectral misalignments (quadratic terms, etc.) could be considered as well in an analogous way, i.e., by defining the pseudo-absorber of nth order as A n := A Shift (λ − λ 0 ) n .Here, we focus on the effects of shift ( =A 0 ) and stretch ( =A 1 ).
As a stretch can be regarded as wavelength dependent shift, the errors of the linearisation can be evaluated similar to Sect.3.1 according to the maximum pixel displacement caused by the stretch.

Pseudo-absorbers for I 0
The shift-spectrum was defined in Eq. ( 4) based on a Taylor expansion of a shifted radiance I .Here, we define a similar shift-spectrum based on I 0 :  7), with an optical depth of a = 0.2 and a width ς of 5 pixel, for a shift of λ = 1 pixel.(b) Actual difference of the true and shifted OD (blue), compared to the linear approximation, i.e., the pseudo-absorber A Shift as defined in Eq. ( 4), scaled by the applied shift of 1 pixel (green).7), with an optical depth of a = 0.2 and a width ς of 5 pixel, for a stretch of q = 1.1.(b) Actual difference of the true and stretched OD (blue), compared to the linear approximation, i.e., the pseudo-absorber A Stretch as defined in Eq. ( 15), scaled by the applied stretch q − 1 (green).
The respective stretch-spectrum is defined analogously to Eq. ( 15): Deriving the pseudo-absorbers from I 0 instead of I may seem illogical, as I 0 is spectrally calibrated against a highresolution, well-calibrated solar reference and, thus, not subject to further shifts.But if the difference between B Shift/Stretch and A Shift/Stretch is negligible, which is the case if the spectral structures of both I and I 0 are dominated by the strong Fraunhofer lines and the OD of atmospheric absorbers τ is low, this approach has high practical relevance, as it allows one to create a complete, consistent set of references just based on I 0 , which can be applied to a sequence (e.g., one satellite orbit) of radiance measurements.In contrast, the exact calculation of the pseudo-absorbers based on I would imply the calculation of the derivative of I for each measured spectrum, before the linear fit can be performed.Both approaches are investigated in the next section for concrete examples, and the applicability of B Shift/Stretch instead of A Shift/Stretch is discussed further in Sect.5.2.

Applications
In this section, we analyse the accuracy and performance of the proposed implementation of spectral shift and stretch in the DOAS analysis by pseudo-absorbers, in comparison to the "classical" non-linear DOAS.
In Sect.4.1, a standard DOAS application, i.e., the fit of NO 2 in the blue spectral range, is performed on synthetic spectra, which allows us to compare the fit results to the a-priori "truth".In Sect.4.2, the linearisation is applied to actual satellite measurements for the retrievals of NO 2 and BrO.

Setup
We consider a detector with a spectral resolution of 0.55 nm FWHM and a spectral sampling of 0.2 nm, as for TROPOMI in the blue spectral range (Veefkind et al., 2012).Starting with a high-resolution solar irradiation spectrum (Chance and Kurucz, 2010), I 0 is determined in detector resolution by assuming a Gaussian slit function.The radiance I is calculated by considering rotational Raman scattering and the absorption of NO 2 (Vandaele et al., 1998) optical depths (peak-to-peak) of 5 % and 1 %, respectively, plus a broad-band absorption.Note that the dependency of the NO 2 fit bias on shift and stretch, as investigated below, is not affected by this choice of the a-priori NO 2 SCD.
I is then manipulated by a variety of spectral displacements.Table 1 lists the applied shift and stretch parameters.The fit accuracy is analysed for "ideal" spectra, as well as for spectra where a Gaussian random noise with a standard deviation of 0.015 % of the maximum radiance is added to I .This noise level corresponds to residues of about 0.1 % peak-to-peak.
The pseudo-absorbers A Shift and A Stretch are determined from I according to Eqs. ( 4) and ( 15), and B Shift and B Stretch from I 0 according to Eqs. ( 16) and ( 17), respectively.Note that the calculation of the discrete derivatives I or I 0 has to be done properly, as a simple difference quotient was found to be not accurate enough.This aspect is discussed in Appendix B.
Finally, the shifted and stretched radiances are analysed by linear (with and without the proposed pseudo-absorbers) and non-linear DOAS at 430-450 nm.For this, the commercial software MATLAB, as well as DOASIS (Kraus, 2005), have been used.

Results
Figure 3 illustrates the effect of the shift and the accuracy of the fit exemplarily for a shift of λ = 0.02 nm, i.e., a tenth of a pixel.The figure displays only half of the fitting window to zoom in the small horizontal displacements.
Figure 3a displays I 0 (black) and the original (grey) and shifted (blue) radiances I and I shifted , which can be hardly discriminated.In Fig. 3b, the respective OD is shown for I (grey, revealing the combined structures caused by the applied Ring effect and NO 2 absorption), and I shifted (blue, revealing additional spectral structures due to the shift λ).
In Fig. 3c, the difference of both OD ("true" and "shifted") from Fig. 3b are displayed in blue, analogously to the simple example shown in Fig. 1b, showing spectral structures of about 3.5 % peak-to-peak caused by the spectral mismatch.Note that this structure is very similar to the fit residue of the linear fit without shift-and stretch-spectra (not shown), but not identical, as the fit tries to "explain" part of the shift structures by the considered (pseudo-) absorbers Ring and NO 2 .The shift spectra A Shift and B Shift , as defined in Eqs. ( 4) and ( 16), scaled by the applied shift, are shown in green/orange, respectively.They both reproduce well the spectral structures caused by the spectral misalignment.
Figure 3d displays the residue of the linear fit including A Shift and A Stretch as pseudo-absorbers in green, and B Shift and B Stretch in orange.The remaining residue is about 0.11 % (A) and 0.15 % (B) peak-to-peak, i.e., more than one order of magnitude lower than for the simple linear fit.Note that the residue for the non-linear fit, shown in red, depends on the settings of the Levenberg-Marquardt-Algorithm, in particular on the allowed number of iterations and the predefined termination criteria.
If we estimate the shift effects from Eqs. ( 9) and ( 12), we expect (for a ≈ 0.4, ς ≈ 0.3 nm, and λ = 0.02 nm) spectral structures of the order of 3.2 %, in agreement with Fig. 3c, and unresolved residues of about 0.12 %, in good agreement with Fig. 3d.Therefore, the estimates derived for a single band can also be used to estimate the order of magnitude for a complex multi-band spectrum.
In Fig. 4, the fitted shift and stretch parameters are compared to the actually applied values.Note that for the linear fit (green), the shift and stretch parameters are simply the fit coefficients for the pseudo-absorbers A Shift and A Stretch , i.e., λ and q − 1, respectively, while I is not changed.In contrast, for the non-linear fit (red), a real shift/stretch is performed to I in each iteration step.It can be seen that both fits actually reproduce the applied shifts/stretches with high accuracy.For the non-linear and linear fit, relative deviations between fitted and true shift λ are less than 0.4 % and 3 %, respectively, for displacements < 0.03 nm.For a linear fit including B Shift (not shown), the relative deviations are slightly higher (up to 13 %).Now we investigate the resulting SCDs for the variety of shifts and stretches listed in Table 1.For each shift/stretch combination, we define the "maximum displacement" as the maximum of all detector pixel displacements D within the fit window, in order to have a single abscissa.For instance, for a shift of 0.1 pixel, i.e., 0.02 nm, and a stretch of 1.0001, the spectral displacement, according to Eq. ( 14), is maximum at the right edge of the fitting window, where it is (q − 1)(λ − λ 0 ) + λ = 0.021 nm.
Figure 5a shows the resulting fit residues for all combinations of shift and stretch, up to a maximum displacement of 0.03 nm, on a log-log-scale.The straight lines indicate the expected first (purple) and second (green) order estimate according to Eqs. ( 9) and ( 12) for a = 0.4 and ς = 0.3 nm.The absolute deviations of the fitted NO 2 SCDs from the true (apriori) value are shown in Fig. 5b.In addition, values for a shift of 0.002 nm are listed in Table 2.
Purple dots show the results for the simple linear fit ignoring shift and stretch, revealing residues according to the first order estimate.For a displacement of 0.002 nm (i.e., 1 % of a pixel), residues are 0.33 %.The respective NO 2 SCDs are biased by 3.2 × 10 14 molec cm −2 , which is of the order of fit uncertainties of NO 2 SCDs from satellite retrievals (Boersma ) for the original and shifted Intensities.(c) Difference of the OD from shifted and original Intensities (blue), and the "shift-spectrum" as defined in Eqs. ( 4) and ( 16), scaled by the applied shift of 0.02 nm (green/orange).(d) Fit residues for the non-linear fit (red) and the linear fit with shift and stretch pseudo-absorbers included (green/orange).9) and ( 12), for a = 0.4 and ς = 0.3 nm.et al., 2004).Compared to this, the fit results are significantly improved if shift and stretch are included as pseudoabsorbers to the linear fit (green): both the residue and the NO 2 bias are generally reduced by more than two orders of magnitude.At a displacement of 0.002 nm, the residue and NO 2 bias are now only about 10 −5 and 10 12 molec cm −2 , respectively, which is negligible, and quite close to the results of the non-linear fit.For pseudo-absorbers derived from I 0 (orange), the results are slightly worse (5 × 10 −5 residue and 5 × 10 12 molec cm −2 bias at 0.002 nm), but still far better than those of a linear fit without pseudo-absorbers.For smaller displacements, the bias decreases approximately quadratically (according to Eq. 12) for A, as expected (as the quadratic term is neglected in Eq. 3).For B, the decrease is slower (≈ linear) due to the additional approximation, i.e., the neglect of atmospheric trace gas absorptions in the calculation of the pseudo-absorbers (compare Eq. 18).
Figure 6 shows the respective fit results averaged for 300 spectra exposed to additional Gaussian noise.Table 2 lists the respective means and standard deviations for a shift of 0.002 nm.The noise truncates the fit residues (peak-to-peak) of all fits to 0.1 %.Still, the linear fit without shift/stretch performs poorly, while the other methods (linear with pseudoabsorbers and non-linear) show lower residues by a factor of about 3 for the typical shifts of 0.002 nm for passive DOASapplications.
The bias is quite low on average for all fits except the simple linear case.But the standard deviation of the bias caused by the noise is about 1.6 × 10 14 molec cm −2 for a shift of 0.002 nm.
For the noisy radiances, the non-linear fit shows neither lower residues nor lower NO 2 biasses compared to the linear fit with pseudo-absorbers.Thus, for practical applications, noise levels of > 0.015 % and shifts < 0.002 nm, there is no benefit from the non-linear fit at all, and the pseudo-absorbers can be derived from I 0 instead of I , at least for the analysis of NO 2 (see also Sect.4.2 and Sect.5).

Computation time
The fits have been performed on a Windows XP system with an Intel Core 2 Duo CPU T9300 at a frequency of 2.50 GHz.The synthetic spectra have been analysed using DOASIS (Kraus, 2005) as well as MATLAB; for the linear fit, the resulting SCDs are identical for both implementations, while for the non-linear fit, small (but insignificant) deviations of SCDs occur between MATLAB and DOASIS due to the different implementations and termination criteria.
Table 2. Fit residues and NO 2 bias for a shift of 0.002 nm.For the spectra affected to noise, the mean and standard deviation of 300 random spectra is given.All numbers are given with 2 digits, but in fixpoint notation so that the different orders of magnitude are directly visible.The last two fits (5. and 6.) refer to different methods for the calculation of the discrete derivative (see Appendix B) .

Peak-to-peak residue [% OD]
NO 2 bias [10 14 molec cm The computation times for a single fit (without noise) are compared in Table 3.For the spectra affected by Gaussian noise, computation times are about 2 to 3 times higher.The non-linear fit is faster by a factor of 10 for the DOA-SIS implementation compared to MATLAB, as could be expected, since DOASIS is optimised for this method, and the mathematical operations are processed by pre-compiled, executable code, while MATLAB is a script language.But interestingly, for the linear fit, MATLAB performs better than DOASIS; this is probably due to the fact that MATLAB is optimised for linear matrix operations, while DOASIS does not directly provide the option of a linear fit; instead, the fitted parameters can only be fixed (i.e., the shift can be set to 0).Though we can not comprehend the implementation details of DOASIS, we consider this to be the reason that DOASIS is slower for the "linear" fit compared to MATLAB.We conclude that DOASIS yields realistic numbers (4 × 10 −3 s per fit) for the computation time of the non-linear fit, while for the linear fit, the results from MATLAB gives an upper limit of what can be achieved for linear algorithms (4 × 10 −5 s per fit).Therefore, the linear fit is at least faster by two orders of magnitude.Further acceleration of the linear fit can possibly be gained using optimised code on GPUs (Graphic Processor Units), which are particularly suited for linear operations.
If we apply these numbers to one orbit of TROPOMI with about one million spectra (P.Veefkind, personal communication), the computation time would be about 4000 s (> one hour) for the non-linear fit, but only 40 s for the linear fit, without consideration of input/output operations.

Satellite measurements
In addition to the case study for synthetic spectra, we applied the linearisation scheme also to real satellite measurements for the trace gases NO 2 (Sect.4.2.1) and BrO (Sect.4.2.2), and compared the results to the routine retrievals at Max-Planck-Institute for Chemistry (MPI-C) Mainz.Both MPI-C retrievals are based on a non-linear Levenberg-Marquardt minimisation algorithm.Spectral shifts, but no stretches, are considered in the routine retrievals, as well as below.
We considered three different linear fit set-ups: 1. a simple linear fit ignoring shift, 2. a linear fit including the shiftspectrum A Shift and 3. a linear fit including the shift-spectrum B Shift , and compared the residues and SCDs to the non-linear fit.The resulting SCD bias is set in relation to the respective detection limit, which is determined according to Eq. (8.41) in Platt and Stutz (2008).However, we would like to point out that spatio-temporal means of satellite observations can reveal clear patterns far below the detection limit for single measurements, as statistical noise cancels out.Thus, the tolerable systematic bias is usually lower (about 1/10) than the detection limit for an individual measurement.
For both NO 2 and BrO, the shift along the orbit has been found to be > 0.01 nm (see Tables 4 and 5).This overall shift can mostly, but not completely, be explained by the Doppler effect (see Sect. 2.2.3).As this overall shift is quite large, but consistent along the orbit, it could be a-priori accounted for by shifting all radiances I by this overall shift before performing the fit.However, if undersampling effects can be neglected, this is mathematically equivalent to shifting the set of references (I 0 , cross-sections and pseudo-absorbers) towards I instead.Therefore, I 0 and all cross-sections/pseudoabsorbers are shifted consistently by the mean spectral shift to better match the pixel-wavelength allocation of I .This is much faster, as it has to be done only once per day for satellite measurements.After this "pre-shift" of the set of references, the fit only has to find the remaining shift relative to this mean shift, which is far smaller.Below, we discuss the fit results for both, the original as well as the pre-shifted set of references.9) for a = 0.4 and ς = 0.3 nm.Results from the original (shift > 0.007 nm) and the pre-shifted set of references (shift < 0.007 nm) are combined.

Retrieval of Nitrogen dioxide -NO 2
The routine MPI-C retrieval of NO 2 from SCIAMACHY is described in Beirle and Wagner (2012).Here we analyse the first orbit of 1 June 2006.
Figure 7 shows (a) the resulting residues (peak-to-peak) and (b) NO 2 SCD deviations to the non-linear fit, in dependency of the fitted spectral shift (from the non-linear fit), similar as Figs. 5 and 6.The mean shifts, residues, and SCD biasses are listed in Table 4.Note that Fig. 7 shows absolute values both on x and y axis, as the scales are logarithmic, but the means listed in Table 4 are calculated for the signed values and can thus be far lower if positive and negative numbers cancel each other out.
For the original set of references, a mean shift of ≈ 0.014 nm is found for the complete orbit.The fitted shifts using the pre-shifted set of references are far lower.Combined, both fits reveal a consistent dependency of residues and SCD biasses over a large variety of spectral shifts.Generally, the results are quite similar to the fit of synthetic spectra (compare Figs. 5 and 6).For large shifts (original set of references), the residues of the simple linear fit (purple) show the expected linear dependency on the shift, even matching quantitatively the estimation according to Eq. ( 9).The fit residues for the other fits (linear with pseudo-absorbers A (green)/B (orange) and non-linear (red)) are similar to each other and show no dependency on the shift, i.e., they are dominated by spectral noise and potential systematic spectral structures of about 0.2 % peak-to-peak, which is slightly higher than the noise applied for the synthetic fits.The difference of residues compared to the nonlinear fit (not shown), i.e., the additional residue due to the linearisation, decreases further towards low shifts, following Eq.( 12) and shown as green line in Fig. 5.
The bias in NO 2 SCD for the simple linear fit shows a clear linear dependency on the spectral shift, as for the synthetic case study.It is, on average, 3.4 ×10 15 molec cm −2 for the original set of references, which is significantly higher than the NO 2 detection limit, and would not be tolerable for satellite retrievals of NO 2 .For the pre-shifted set of references, positive and negative shifts cancel each other, leading to a low bias on average (0.06 ×10 14 molec cm −2 ).But the SCD deviations for individual pixels can still reach 1 ×10 15 molec cm −2 .
The linear fit including A Shift performs very well.Deviations from the non-linear fit are only −0.62 and −0.004 ×10 14 molec cm −2 for the original/pre-shifted set of references, respectively.Using B Shift instead of A Shift slightly degrades the agreement (also for the fitted shift), but still, the mean bias would be acceptable even for the original set of references without pre-shift.
Note that the observed SCD bias is proportional to the shift even for small shifts, whereas in Fig. 6 (synthetic spectra with noise), it is rather constant for shifts below ≈ 0.002 nm.This is due to the fact that Fig. 6 compares the fit results of noisy spectra to the a-priori truth, while in Fig. 7, fit results of different fits, but the same noisy spectrum, are compared to each other.Thus, the low biasses at low shifts must not Table 5.Comparison of the results of the non-linear and the different linear fits applied to GOME-2 measurements on 12 April 2009.Spectral shift, fit residues, and BrO SCD bias are displayed as mean ±standard deviation, first for the original, then (separated by "/") for the pre-shifted set of references. .
be interpreted as kind of overall error or detection limit, but nicely illustrate how close the linear results can reproduce the non-linear fit.

Retrieval of Bromine monoxide -BrO
The routine MPI-C retrieval of BrO from GOME-2 is described in Sihler et al. (2012).Here we analyse an orbit of 12 April 2009 for the Northern Hemisphere, where enhanced tropospheric BrO was observed in the Beaufort sea (see page 17 of the Supplement of Sihler et al., 2012).Figure 8 shows (a) the resulting residues (peak-to-peak) and (b) BrO SCD deviations to the non-linear fit, analogue to Fig. 7. Table 5 lists the respective orbit means and standard deviations.
The results are similar to those for NO 2 (Sect.4.2.1), but with some differences in detail.The simple linear fit again results in a clear dependency of residues (for shifts above ≈ 0.005 nm) and BrO biasses on the absolute spectral shift.But the slope of the residue is higher than for NO 2 due to the deeper Fraunhofer lines in the fitting window.The purple line in Fig. 8a corresponds to Eq. ( 9) for a = 0.7 and ς = 0.2 nm.
The residues of the other fits again show no dependency on the shift, and are generally higher (about 0.8 % peak-to-peak OD) than for NO 2 .
The simple linear fit results in a mean bias of −3.2 ×10 13 molec cm −2 for the original set of references, which would not be tolerable.In contrast, the linear fit including A Shift works very well and yields results very close to those of the non-linear fit.The mean bias of −0.52 ×10 12 molec cm −2 for the original set of references is negligible.
Including B Shift yields again lower SCD biasses (1×10 13 molec cm −2 ) than the simple linear fit.But the improvement is less distinct than in the case of NO 2 .This is due to the fact that the difference of A Shift and B Shift is determined by the derivative of the OD, which is generally higher for the BrO fit, mainly due to the Ring effect and ozone absorption (see Sect. 5.2).The resulting SCD bias is slightly below the detection limit of BrO, but still high enough to significantly affect spatio-temporal means.
This case study clearly illustrates that the linearisation including A Shift also works for a weak absorber as BrO.The question whether B Shift could be used instead depends on the accuracy requirements and has to be investigated for each trace gas and fitting window individually.In several cases, the a-priori correction of the Doppler-shift by pre-shifting the set of references will allow the usage of B Shift .

Discussion
The proposed treatment of shift and stretch effects as pseudoabsorbers allows one to solve the DOAS equation system linearly.For typical noise levels, the fit accuracy almost equals that of a non-linear fit.The linearisation significantly speeds up the DOAS fit, which is particularly relevant for satellite measurements.Here we discuss further advantages, aspects and restrictions of our approach.

Quick assessment of the spectral calibration
Apart from fitting trace gas SCDs, the pseudo-absorbers A Shift/Stretch allow the quick assessment of the spectral calibration of a given spectrum: the respective fit parameters directly yield the spectral shift λ and stretch (q −1).Thus, for example, the mean spectral shift of satellite measurements can quickly be determined, or the need for accounting for a spectral stretch can be easily evaluated.

Pseudo-absorbers derived from I 0
As mentioned in Sect.3.3, calculating the pseudo-absorbers from I 0 /I 0 instead of I /I has high practical relevance for satellite retrievals: I 0 /I 0 needs to be calculated only once per reference spectrum, i.e., once per day for satellite measurements, thereby providing one consistent set of references for the complete day.
For passive DOAS applications, both I and I 0 are usually dominated by the strong Fraunhofer lines.Thus, B Shift is generally similar to A Shift ; they differ by the derivative of the OD: (compare Eq. 1).
For τ with a peak-to-peak OD of b, the order of magnitude of τ is about b/ς.If, in addition, the spectral structures of the OD are dominated by the Ring effect, the peak width ς will be similar to that of I .Consequently, the relative deviation between A Shift and B Shift is about b/a, i.e., the ratio of the order of magnitude of τ and the depth of the Fraunhofer lines.Note that the impact on the fit results is even less, as B Shift can be scaled by the fit.Therefore, only different patterns of B Shift compared to A Shift can cause an additional SCD bias.
For NO 2 , the errors induced by B Shift/Stretch have been found to be generally negligible (Figs. 6 and 7).However, this is different for DOAS evaluations for trace gases absorbing in the UV, like BrO, due to the higher OD of Ozone and the Ring effect.
If the errors caused by the pseudo-absorbers B can not be neglected, and A have to be used, the derivative of I has to be calculated for each individual measured spectrum.This reduces the acceleration due to the linearisation by about 20 %, if the derivative is calculated according to Eq. (B3) (see Appendix B).
Alternatively, we propose to apply a pre-shift to the set of references (next section), which results in significantly lower remaining shifts and, thus, allows one to use B instead of A for the investigated examples.

Pre-shifted set-up
For satellite measurements, the spectral shift is dominated by the Doppler-effect of 0.01 nm at 440 nm.For NO 2 as well as BrO, the linearisation based on the pseudo-absorbers A works well (i.e., SCD biasses are negligible) even for such large shifts.However, as the observed shifts are quite consistent along the orbit, we propose to correct for the overall shift by pre-shifting the set of references I 0 , cross-sections and pseudo-absorbers accordingly (see Sect. 4.2).(The a-priori required overall shift could, for example, be taken from the previous day in routine retrievals of satellite data).The remaining shifts (relative to this overall shift) are considerably smaller, resulting in far lower SCD biasses due to the linearisation, thus, facilitating the use of the pseudo-absorbers B instead of A (see previous section), even for the BrO retrieval.

Missing pixels
An additional advantage of the proposed implementation of spectral misalignments by pseudo-absorbers derived from I 0 instead of I is the evaluation of radiance measurements I containing gaps (e.g., due to dead detector pixels); such gaps are problematic for the non-linear fit, as shifting I involves interpolation and one missing pixel, thus, affects (i.e., removes) its neighbours as well.In contrast, for the linear fit, the dead pixels can just be skipped for I , I 0 , σ and the pseudo-absorbers likewise, and the effect of a potential shift is still accounted for appropriately by the pseudo-absorbers.
In the case of gaps in I 0 , however, these have to be filled in first (e.g., by fitting another solar reference without gaps plus a polynomial), before the derivative I 0 can be calculated.

Initialization of the non-linear fit
Even if a non-linear DOAS analysis is unavoidable for a given set-up, e.g., due to large shifts or other effects causing nonlinearities, the linear fit could still be applied for the calculation of the initial state vector.The subsequent non-linear fit benefits from accurate initial values for SCDs and particularly shift and stretch, and will be more robust and faster.For instance, for a shift of 0.002 nm, the non-linear fit becomes 30 % faster if initialised with the linear fit results including shift-and stretch-spectra.

Conclusions
Spectral misalignments cause biasses in a DOAS analysis and can generally not be neglected.As a consequence, nonlinear minimisation algorithms are applied in state-of-the-art DOAS analyses, which are far slower than linear algorithms.
We propose to linearise the effects of spectral shifts by the first-order term of a Taylor expansion and introduce a "shiftspectrum" as pseudo-absorber (compare Rozanov et al., 2011).Spectral stretches can be considered as well by a "stretchspectrum" as they can be regarded as additional wavelength-dependent shifts.
With these pseudo-absorbers, the DOAS-analysis can be performed linearly, reducing the computational costs by at least 2 orders of magnitude.This is particularly relevant for satellite measurements with high data rates.
The error due to the linearisation, in terms of peak-to-peak OD, is 0 where a and ς are the band depths and widths of I , respectively.For typical shifts about 0.14 nm, this error is about 6 × 10 −4 for NO 2 (a ≈ 0.4, ς ≈ 0.3 nm), and about 2.4 × 10 −3 for BrO (a ≈ 0.7, ς ≈ 0.2 nm), thus, significantly lower than the actual noise levels for the investigated retrievals of NO 2 and BrO from SCIAMACHY and GOME-2, respectively.The resulting biasses of SCDs have been found to be negligible.Full advantage of the linearisation could be gained if the pseudo-absorbers are calculated based on I 0 (instead of I ), as these have to be derived only once per reference spectrum instead of once per measurement: For weak absorbers like NO 2 , this is generally applicable.For strong absorbers like O 3 in the BrO or SO 2 fit, the pseudo-absorbers A might have to be calculated for each radiance spectrum, reducing the speed of the linear fit by about 20 %.
Alternatively, the applicability of B can be achieved if the overall shift between I and I 0 is accounted for by an a-priori "pre-shift".For the investigated retrievals of NO 2 and BrO from SCIAMACHY and GOME-2, respectively, the remaining shifts are about 0.003 nm, and the SCD biasses resulting from a linear fit including B are negligible even in the case of BrO.
Even if a non-linear fit is unavoidable (for larger shifts or other non-linear effects), the linear fit can still yield a quick assessment of the spectral calibration of I and an accurate initial state vector for shift and stretch in the non-linear fit.

S. Beirle et al.: Linearised shift and stretch in DOAS
Figure 5.3 shows the results for a linear fit including A σ Shift (i.e., ignoring that the problem is not really linear) for the example of a synthetic spectrum with Ring effect (5 % OD) and NO 2 absorption (5 % OD), where the NO 2 cross-section is shifted and stretched.Obviously, the applied shifts and stretches are not reproduced by the linear fit due to the remaining nonlinearity.The fit accuracy for NO 2 is not much better than that of a simple linear fit ignoring shift effects completely.
Fortunately, spectral shifts of σ do not have a high relevance in most DOAS applications, as cross-sections can generally be measured in laboratory with high accuracy.Additionally, a shift could only significantly affect the DOAS analysis if the respective absorber has a high OD of several percent and narrow absorption lines.Thus, normally, spectral shifts of cross-sections can be neglected.But in the case of an analysis where spectral shifts of σ actually occur for a strong absorber, these shifts have either to be determined (and corrected) independently, or the DOAS analysis has to be performed by a non-linear algorithm.

Discrete derivative
The shift-spectrum involves the first derivative of I , which has to be calculated numerically.For the comparison of the linear and non-linear fit in Sect.4, we calculate the derivative of the discrete functions I and I 0 by mimicking the difference quotient according to where y is I or I 0 , respectively, x is the (integer) pixel number, h is set to 0.0001 pixel, and y(x + h) is derived by cubic spline interpolation of y.This approach proves to yield accurate results for the linear fit, particularly for small shifts, but is a rather timeconsuming method.
Alternatively, we investigate the fit accuracy for the discrete derivatives and according to Bronstein and Semendjajew (1981) (Sect.7.1, Table 7.13 therein), where h is the distance between two y values.The unit chosen for h (i.e., 1 pixel or the corresponding wavelength interval) determines the units of the derivative and the fitted shift, respectively.
Table 2 lists the respective fit residues and SCD bias of the different derivative definitions for a shift of 0.002 nm.Compared to the spline-based derivative, both derivatives result in higher residues and SCD bias.However, for the spectra affected by noise, results from the fit using Eq.(B3) are quite close to the spline-based results.Thus, we propose to calculate the discrete derivative according to Eq. (B3), which is far faster than the spline-based derivative, and shows far better fit results compared to derivatives based on Eq. (B2).
Strictly speaking, the derivative of I 0 would have to be calculated from the high-resolution solar reference, convolved with the instrument spectral response function afterwards.However, here we calculate all derivatives (for A Shift and B Shift ) on the spectrometer's grid and resolution (FWHM 0.55 nm).This approximation does not affect our results; the introduced errors are small compared to other effects (e.g., the appropriate discrete derivative method).

Fig. 1 .
Fig. 1.Illustration of the effect of a spectral shift of I .(a) Original (black) and shifted (blue) Intensity I in artificial units, showing a single absorption line according to Eq. (7), with an optical depth of a = 0.2 and a width ς of 5 pixel, for a shift of λ = 1 pixel.(b) Actual difference of the true and shifted OD (blue), compared to the linear approximation, i.e., the pseudo-absorber A Shift as defined in Eq. (4), scaled by the applied shift of 1 pixel (green).

Fig. 2 .
Fig. 2. Illustration of the effect of a spectral stretch of I .(a) Original (black) and stretched (blue) Intensity I in artificial units, showing a single absorption line according to Eq. (7), with an optical depth of a = 0.2 and a width ς of 5 pixel, for a stretch of q = 1.1.(b) Actual difference of the true and stretched OD (blue), compared to the linear approximation, i.e., the pseudo-absorber A Stretch as defined in Eq. (15), scaled by the applied stretch q − 1 (green).

Fig. 3 .
Fig. 3. Illustration of the linear and non-linear DOAS fits for a simple synthetic spectrum I including the Ring effect and NO 2 absorption.(a) Intensities I 0 , I and I shifted by 0.02 nm.(b) OD (ln I I 0

Fig. 4 .Fig. 5 .
Fig. 4.Comparison of the fitted and the true shift (left panel, for stretch ≡ 1) and stretch (right panel, for shift ≡ 0) for the non-linear (red) and the linear fit including A Shift and A Stretch (green).The applied shifts and stretches are listed in Table1.The grey line shows 1 : 1 correspondence.Note that the scales are non-linear to show the full range of variations over some orders of magnitude.

Fig. 7 .
Fig. 7.Dependency of the peak-to-peak fit residues (a) and the absolute NO 2 SCD bias with respect to the non-linear fit (b) on the absolute spectral shift (as found by the non-linear fit).Colours as in Fig.5.The purple line corresponds to Eq. (9) for a = 0.4 and ς = 0.3 nm.Results from the original (shift > 0.007 nm) and the pre-shifted set of references (shift < 0.007 nm) are combined.

Fig. 8 .
Fig. 8. Dependency of the fit residues (a) and the absolute BrO SCD bias with respect to the non-linear fit (b) on the absolute spectral shift.Colours as in Fig. 5.The purple line corresponds to Eq. (9) for a = 0.7 and ς = 0.2 nm.Results from the original and the preshifted set of references are combined.
Fig. A1.As Fig. 4, but for shifts in the absorption cross-section of NO 2 .The fitted shifts/stretches for A σ Shift and A σ Stretch are off by almost an order of magnitude due to the remaining nonlinearities (see text).

Table 1 .
Variations of shift and stretch applied to the synthetic spectra.

Table 3 .
Computation times for linear and non-linear fits from MATLAB and DOASIS in seconds per fit.

Table 4 .
Comparison of the results of the non-linear and the different linear fits applied to SCIAMACHY measurements on 1 June 2006.Spectral shift, fit residues, and NO 2 SCD bias are displayed as mean ±standard deviation, first for the original, then (separated by "/") for the pre-shifted set of references.