Surface solar irradiance from SCIAMACHY measurements : algorithm and validation

Introduction Conclusions References


Introduction
Information on surface solar radiation is relevant for a better understanding of climate change and global hydrological cycle, and for more efficient utilization of solar energy.Scientists have attempted to derive surface solar radiation from both geostationary and polar-orbiting satellite measurements.Various algorithms have been developed to produce surface solar radiation data sets based on radiative transfer calculations and statistics (e.g.Moeser and Raschke, 1984;Cano et al., 1986;Bishop and Rossow, 1991;Pinker and Laszlo, 1992;Darnell et al., 1988;Li et al., 1993;Pinker et al., 2003;Zhang et al., 2004;Rigollier et al., 2004;Mueller et al., 2009;Wang and Pinker, 2009).The Pinker-Laszlo (1992) algorithm is used in the GEWEX (Global Energy and Water Cycle Experiment) Surface Radiation Budget Project to generate the shortwave radiative fluxes (http://gewex-srb.larc.nasa.gov/).This approach has also been the basis for the first operational version of the EUMETSAT CM-SAF (Satellite Application Facility on Climate Monitoring) scheme (Hollmann et al., 2006).The FRESCO (Fast REtrieval Scheme for Clouds from the Oxygen A band) SSI (Surface Solar Irradiance) algorithm that is presented in this paper uses a similar approach to the clear-sky approach of the new operational CM-SAF surface solar irradiance algorithm (Mueller et al., 2009).Therefore, some detailed introductions are provided for the Pinker-Laszlo algorithm and the new CM-SAF surface solar irradiance algorithm.
The Pinker and Lazlo (1992) algorithm is based on relating the satellite-measured broadband transmission at the surface ( T ) to the broadband reflectance at the top of the atmosphere ( R).The relationship between R and T is calculated with a radiative transfer model which accounts for absorption Published by Copernicus Publications on behalf of the European Geosciences Union.P. Wang et al.: Surface solar irradiance from SCIAMACHY measurements: algorithm and validation by ozone and water vapor, multiple scattering by molecules, multiple scattering by aerosols and clouds, and multiple reflections between the surface and the atmosphere.
The new operational CM-SAF surface solar irradiance algorithm is based on radiative transfer calculations and using satellite-derived parameters as input (Mueller et al., 2009).The radiative transfer calculations are characterized by the combination of parameterizations and eigenvector look-up tables.The clear-sky approach in the CM-SAF algorithm is comparable to the Mesoscale Atmospheric Global Irradiance Code (MAGIC) algorithm (see Sect. 2.3).The cloudy sky approach in the CM-SAF algorithm relates the top of the atmosphere (TOA) albedo derived from the Geostationary Earth Radiation Budget (GERB) instrument to the surface irradiance.In order to calculate the surface solar irradiance using the TOA albedo, a sophisticated hybrid LUT approach is applied and knowledge of the surface albedo and a cloud mask are needed.The cloudy part of this algorithm employs the additional spectral channels of the Meteosat Second Generation (MSG) satellites.However, the algorithm is not suited to be applied to Meteosat First Generation (MFG).For MFG the CM-SAF algorithm uses the Heliosat method to consider the effect of clouds on the surface solar irradiance.
The Heliosat method was originally described by Cano et al. (1986).Later, various modifications have been made to improve the cloud index calculations (see Sect. 2.1.2) and the clear-sky model (Hammer et al., 2003;Rigollier et al., 2004;Mueller et al., 2004;Dagestad and Olseth, 2007).The basic idea of the Heliosat method is that the cloud index retrieved from satellite reflectances provides sufficient information to estimate the effect of clouds on the surface solar irradiance.After the retrieval of cloud index in the first step of the Heliosat method, the surface solar irradiance is derived by the use of a clear-sky model in a second step.The cloud index (n) is determined from the normalized reflectance.The clearsky index (k) is the ratio between the actual (full-sky) surface solar irradiance (G) and the clear-sky surface solar irradiance (G clr ), namely, k = G/G clr .The clear-sky surface solar irradiances calculated by the clear-sky model are converted into the full-sky surface solar irradiances using the n−k relation.In contrast to the CM-SAF method applied to MSG, the Heliosat method only uses the satellite derived reflectances for the cloud information while the surface albedo and cloud mask are not needed.The advantage of the Heliosat method is its capability to retrieve surface solar irradiance in high accuracy across Meteosat satellite generations.Moreover, the consistent treatment of the surface albedo effect (clear-sky reflectance) reduces the uncertainty arising from the usage of external surface albedo information and does not require the application of an angular distribution model, which is in turn an additional error source.
Different Heliosat algorithms were developed to produce surface solar irradiance data sets from the Meteosat (MFG and MSG) measurements.These data sets have been extensively validated against ground-based solar radiation measurements (Perez et al., 2001;Meyer et al., 2003;Rigollier et al., 2004).The relative standard deviation between hourly mean surface solar irradiances derived using the Heliosat algorithms and ground-based measurements is typically 20-25 % (Ineichen and Perez, 1999;Zelenka et al., 1999;Dagestad, 2004;Lorenz, 2007).It is now well established that the surface solar irradiances derived from satellite measurements can be more accurate than those found by interpolation using ground-based measurements which are more than 30 km apart (Perez et al., 1997;Zelenka et al., 1999).
The Heliosat method has been mainly applied to the Meteosat geostationary satellite measurements, which have the advantage of high spatial and temporal resolution.However, the Meteosat satellites do not provide global coverage but are restricted to a specific part of the world, covering Europe and Africa.Global coverage, including the polar regions, is given by sun-synchronous polar orbiting satellites.SCIAMACHY (SCanning Imaging Absorption spectroMeter for Atmospheric CHartographY) is a spectrometer on board the Envisat satellite which flies in a sun-synchronous polar orbit with equator crossing time at about 10:00 LT (Local Time).Therefore, we adapted the Heliosat method to the SCIAMACHY measurements.The pixel size of the SCIA-MACHY nadir measurements is 60 × 30 km 2 and global coverage takes 6 days.The wavelength range of SCIAMACHY is from ultraviolet to near infrared with about 0.2-1.5 nm spectral resolution.The primary mission objective of SCIA-MACHY is global measurements of trace gases in the troposphere and in the stratosphere, such as ozone, nitrogen dioxide, water vapor, methane and carbon monoxide (Bovensmann et al., 1999).
Cloud and aerosol information is required in the trace gas retrievals; therefore, several cloud retrieval algorithms have been developed for SCIAMACHY based on oxygen A band spectra measurements (e.g.Koelemeijer et al., 2001;Kokhanovsky et al., 2006) and based on the PMD (Polarization Measurement Device) measurements (e.g.Loyola, 2004;Grzegorski et al., 2006).The FRESCO cloud algorithm was first developed by Koelemeijer et al. (2001) to derive effective cloud fraction and cloud pressure and was improved by Wang et al. (2008) with the addition of single Rayleigh scattering (called FRESCO+ or FRESCO v5).In the FRESCO SSI algorithm, the effective cloud fraction is converted to surface solar irradiance using the Heliosat method and using the MAGIC algorithm (Mueller et al., 2004(Mueller et al., , 2009) ) for the clear-sky surface solar irradiances.
In this paper we will describe the FRESCO SSI algorithm and evaluate the SSI product derived from SCIAMACHY measurements.The structure of the paper is as follows.The FRESCO SSI algorithm is described in Sect. 2. Section 3 shows the SCIAMACHY SSI results.The evaluations of the SCIAMACHY SSI results are presented in Sect. 4. Conclusions are drawn in Sect. 5.

FRESCO SSI algorithm
Similar to the Heliosat method (Cano et al., 1986;Hammer et al., 2003), the FRESCO SSI algorithm consists of three steps:

FRESCO cloud retrieval algorithm
In the FRESCO cloud algorithm, the independent pixel approximation is used to account for partly cloudy pixels.In order to simulate the reflectance spectrum of a partly cloudy pixel inside and outside the O 2 A band, a simple atmospheric model is used, in which the atmosphere above the ground surface (for the clear-sky part of the pixel) or cloud (for the cloudy part of the pixel) is treated as an absorbing (due to oxygen) and purely Rayleigh scattering medium.Reflection occurs only at the surface and the cloud top.The surface and cloud are assumed to be Lambertian reflectors.The reflectance R sim (λ, θ, θ 0 , φ − φ 0 ) at a wavelength λ, viewing zenith angle θ, solar zenith angle (SZA) θ 0 , and relative azimuth angle ϕ − ϕ 0 is then given by (1) where c is the effective cloud fraction, A c is the cloud albedo, A s is the surface albedo.T (λ, z, θ, θ 0 ) is the direct atmospheric transmittance for light entering the atmosphere from the solar direction, propagating down to a level with surface height z s or cloud height z c , and then propagating to the top of the atmosphere in the direction of the satellite.The O 2 absorption and single Rayleigh scattering are taken into account in the light paths for T .The O 2 line absorption parameters are taken from the HITRAN 2004 database (Rothman et al., 2005).R c (λ, z c , θ, θ 0 , ϕ − ϕ 0 ) and R s (λ, z s , θ, θ 0 , ϕ − ϕ 0 ) are the single Rayleigh scattering reflectances above the cloud and the surface, respectively (Wang et al., 2008).T c , T s , R c , and R s are pre-calculated and stored in look-up tables.Because there is not enough independent information from the O 2 A band to retrieve cloud fraction and cloud albedo simultaneously, in the FRESCO algorithm the cloud albedo is assumed as a fixed value for an optically thick cloud (Koelemeijer et al., 2001;Stammes et al., 2008).
A c is assumed to be 0.8 when the effective cloud fraction is used for the trace gas retrieval (Koelemeijer et al., 2001).
In the FRESCO SSI algorithm A c is assumed to be 0.95 (see Sect. 2.1.2).Effective cloud fraction of a pixel means P. Wang et al.: Surface solar irradiance from SCIAMACHY measurements: algorithm and validation the cloud fraction for a Lambertian cloud with a fixed cloud albedo (of 0.8 or 0.95) which yields the same reflectance at the top of the atmosphere (TOA) as the real clouds in the pixel.The surface albedo (A s ) is taken from a monthly climatology (Koelemeijer et al., 2003).Surface height is from the GTOPO30 digital elevation model (http://eros.usgs.gov/Find Data/Products and Data Available/gtopo30 info).The unknowns in Eq. ( 1) are c and z c .The reflectances at 15 wavelengths in the 758-759 nm, 760-761 nm, and 765-766 nm bands are used in the retrieval.The retrieval method is based on minimizing the difference between the simulated and the measured spectra, using the Levenberg-Marquart nonlinear least-squares method.

Rationale of using effective cloud fraction as cloud index
Because the FRESCO SSI algorithm uses the Heliosat n − k relation, the effective cloud fraction values have to be comparable to the cloud index values.As shown in Eq. ( 1), the retrieved effective cloud fraction depends on the assumed fixed cloud albedo value.Therefore, the effective cloud fraction value can be similar to that of the cloud index by using a proper assumption for the fixed cloud albedo value.
In the Heliosat method, the cloud index (n) is defined as: where R is the reflectance at the top of the atmosphere in the visible spectral channel, R max and R min are the corresponding maximum reflectance for the cloudy situation and the minimum reflectance for the clear-sky situation, respectively.R max is usually selected as the 95-98 percentile of the reflectance at TOA (Hammer et al., 2003;Dagestad, 2004;Dagestad and Olseth, 2007).In order to calculate the cloud index, the reflectance at TOA has to be corrected for the effects due to the surface reflection and the scattering of atmospheric molecules (Dagestad and Olseth, 2007).By defining R cld = T c A c +R c and R clr = T s A s +R s , according to Eq. ( 1) the effective cloud fraction can be calculated from: Apparently Eq. (3) has the same form as Eq.(2).For the clear-sky situation, if the absorption in the atmosphere is negligible, R clr is dominated by the surface albedo and equivalent to R min .If R cld is chosen similar to R max as well, the effective cloud fraction is defined in the same manner as the cloud index.At the continuum of the O 2 A band, the Rayleigh scattering reflectance is about 0.02 at medium SZA and T c is close to 1. Therefore, the assumption of A c to be 0.95 leads to R cld of 0.97, which is comparable to the 95-98 percentile of the reflectance.With this definition, the effective cloud fraction and the cloud index both express the normalized reflectances to the optically thick clouds and their values are similar.Furthermore, the effective cloud fraction information is mainly obtained from the continuum of the O 2 A band in the 758-759 nm wavelength range, which is within the visible spectral channel of Meteosat used originally to derive the cloud index.The wavelength differences of the effective cloud fraction and cloud index can be neglected.
In principle, it is relatively simple and straightforward to replace the cloud index with the effective cloud fraction and to use the established and validated Heliosat relation between cloud and clear-sky index, the n − k relation.

Calculation of clear-sky index
As described in the introduction, the clear-sky index (k) is used to approximate the cloud transmission.When the cloud index is known, the clear-sky index can be calculated from the n−k relation (Hammer et al., 2003).In the FRESCO SSI algorithm, the effective cloud fraction c is equal to the cloud index n and always equal or greater than 0. Therefore, k is calculated using Eqs.( 4)-( 6): The basic relation between the cloud index and the clear-sky index is defined by the law of energy conservation (Dagestad, 2005) and based upon the observation that atmospheric transmission is linearly related to the earth's planetary albedo (Schmetz, 1993).However, above a cloud index value of 0.8 an empirical adjustment has to be applied to correct for the non-linear behavior.The adjustment was determined from the statistical regression using ground-based measurements at European sites and fitted to get the best performance at all the ground sites.The close relationship of the Heliosat method to the law of energy conservation is probably the reason for the stability and "global" applicability of the Heliosat n − k relation.

Calculation of surface solar irradiance
The surface solar irradiance for the full-sky situation (G) is given by, where G clr is the clear-sky surface solar irradiance calculated using the MAGIC algorithm (Mueller et al., 2004(Mueller et al., , 2009)).
The MAGIC algorithm is based on radiative transfer model calculations, following a look-up table (LUT) approach.The MAGIC algorithm includes a basic clear-sky LUT, surface albedo map, water vapor climatology and aerosol climatology.The basic clear-sky LUT consists of radiative transfer model (RTM) results for aerosols with different aerosol optical thickness, single scattering albedo, and asymmetry parameter values.Fixed values for water vapor, ozone and surface albedo have been used for the calculation of the basis LUT: 15 kg m −2 for water vapor column, 345 DU of ozone, and a broadband surface albedo of 0.2.The effect of the solar zenith angle on the transmission, hence the surface solar irradiance, is considered by the use of the Modified Lambert-Beer (MLB) function.The effects of variations in water vapor and surface albedo with respect to the fixed values used in the calculation of the basis LUT are corrected using correction formulas and parameterizations (Mueller et al., 2009).The variation of ozone column amount is not taken into account.The applied parameterizations have been derived by RTM calculations and are in line with explicit RTM results.
The aerosol, water vapor and broadband surface albedo climatology databases in the MAGIC algorithm can be updated without changing the basis LUT and the code.
The sensitivities of the retrieved surface irradiance values to the variations of water vapor column amount, aerosol optical thickness and surface albedo have been analyzed by Mueller et al. (2009) and Wang et al. (2011).The surface solar irradiance shows a weak dependency on surface albedo for clear-sky cases (Mueller et al., 2009), e.g. a variation of the surface albedo of 50 % relative to a 0.2 reference value leads only to a variation of 1 % in clear-sky surface solar irradiance.The sensitivity of SSI to water vapor can be estimated from Fig. 3 in Mueller et al. (2009).For example, if the water vapor column in the climatological database is 15 mm, but the actual water vapor column is 10 or 20 mm, the variation of SSI values is within +20 and −15 W m −2 .The effect of aerosol optical thickness (AOT) on the SSI is briefly discussed by Wang et al. (2011).For clear-sky cases, changes in AOT of ±0.02 lead to changes in global irradiance of about ±5.2 W m −2 at cosine of solar zenith angle of 0.42.For fully cloudy cases (cloud optical thickness of 20.1), AOT changes of ±0.02 cause changes in SSI of ±0.44 W m −2 .Therefore, it is important to use proper aerosol, water vapor and broadband surface albedo data in the MAGIC algorithm.
The input data of the MAGIC algorithm include: date, time, solar zenith angle, latitude, longitude, cloud index (effective cloud fraction), water vapor column density, aerosol optical thickness and single scattering albedo, and broadband surface albedo (see Fig. 1).The choice of water vapor, aerosol and broadband surface albedo data is discussed in Sect.2.4.The output of the MAGIC algorithm is the clearsky and full-sky surface solar irradiances in the 0.2-4.0µm wavelength region.The extraterrestrial total solar irradiance is 1365 W m −2 and is adjusted according to the Earth-Sun distance.The n − k relation is part of the MAGIC algorithm.The MAGIC algorithm is fast, robust and suitable for operational applications.More details about the MAGIC algorithm are given by Mueller et al. (2004Mueller et al. ( , 2009)).

Configuration of the FRESCO SSI algorithm for SCIAMACHY
Within the ESA WACMOS (European Space Agency, Water Cycle Multi-mission Observation Strategy) project (Su et al., 2010), the surface solar irradiances in 2006 and 2008 have been derived from the SCIAMACHY measurements as a demonstration data set.The effective cloud fractions are taken from the TEMIS website directly, because the FRESCO cloud product from SCIAMACHY is one of the operational products from the ESA TEMIS (Tropospheric Emission Monitoring Internet Service, http://www.temis.nl)project.The FRESCO cloud product on the TEMIS web page is reprocessed using the latest version of SCIAMACHY Level 1 data.It is more efficient to use the existing effective cloud fraction than reprocessing the FRESCO cloud product using the SCIAMACHY Level 1 data.In the WACMOS project, we used the FRESCO data version sc-5.2 (Level 1 data v6.03).However, the fixed cloud albedo (in Eq. 1) is assumed to be 0.8 in the operational FRESCO cloud algorithm for the TEMIS project, because it leads to optimal effective cloud fractions for the trace gas retrievals (Koelemeijer et al., 2001;Stammes et al., 2008).In order to be consistent with the definition of the cloud index in the Heliosat method, the original FRESCO effective cloud fractions (c 0.8 ) have to be converted to the effective cloud fractions with a fixed cloud albedo of 0.95 (c 0.95 ), which gives the best performance for the SSI product.The conversion is carried out as follows.
The FRESCO effective cloud fractions c 0.8 and c 0.95 were retrieved by assuming the cloud albedo to be 0.8 and 0.95, respectively, for one SCIAMACHY orbit over Europe, the Sahara desert and Atlantic Ocean on 4 February 2007 (orbit number 25785, measurement start time 10:30:03 UTC).
The ratios of the two effective cloud fraction data sets were fitted as a function of the surface albedo (A s ) using a 3rdorder polynomial.The fit and the polynomial coefficients are shown in Fig. 2. According to the regression, c 0.95 can be calculated from where x is the averaged surface albedo at the O 2 A band.The fit to the SCIAMACHY data was checked with the fit derived from one full day (15 orbits) of GOME-2 (Global Ozone Monitoring Experiment) measurements and good agreement was found.As illustrated in Fig. 2, the fit is quite good except for a few outliers due to the cut off of c 0.   The SCIAMACHY SSI product is only derived for SCIA-MACHY pixels without snow and ice contamination, because the effective cloud fraction data are not available over snow/ice.Duerr and Zelenka (2009) demonstrated that the Heliosat method could be used for MSG measurements over snow/ice with some adjustments.However, more investigations have to be done before it can be applied to the SCIA-MACHY measurements.The maximum solar zenith angle is 89.0 • in the FRESCO SSI algorithm.The monthly mean SCIAMACHY SSI data are gridded at 0.25 • × 0.25 • regular latitude/longitude grid.It is of importance to note that the monthly mean SCIAMACHY SSI is the average of the instantaneous SSI values measured at about 10:00 LT.This generally leads to larger SSI values than the daily mean monthly mean SSI provided by the Meteosat or MSG radiation products.

Results
The seasonal means of the SCIAMACHY effective cloud fraction (c 0.95 ) and SSI for June-July-August 2008 are shown in Fig. 3.A dominant feature of the SSI map is the latitude gradient caused by the solar zenith angle.Because the SCIA-MACHY measurements are taken at 10:00 LT, in Fig. 3 the solar zenith angles are larger at high latitudes and smaller at tropics.SCIAMACHY SSI is anti-correlated with the effective cloud fraction, which is expected according to Eqs. ( 4) and ( 5).For the regions with more clouds, the SSI values are small, for example at the low cloud fields over ocean close to the west of the continents and at the Intertropical Convergence Zone (ITCZ).Large SSI values appear at the desert regions, for example the Sahara desert.The structure of the SSI over the Sahara desert could be due to the artifact in the effective cloud fraction.Because the surface albedo database used in FRESCO cloud retrieval was derived from GOME measurements at about 1 • × 1 • spatial resolution (Koelemeijer et al., 2003), some structures of the surface albedo showed up as clouds in the FRESCO effective cloud fraction, especially over bright surfaces.The FRESCO effective cloud fraction is often overestimated in the desert regions.At coastlines and over islands, the GOME surface albedo usually provides the surface albedo value of ocean, therefore the effective cloud fraction can be overestimated and SSI values can be underestimated.Over islands and at coastlines, the SCIAMACHY pixel often covers both land and ocean.In this case the surface albedo cannot be determined properly for the whole pixel, which causes the retrieved effective cloud fraction to be less accurate than that at only land or ocean sites.This is supported by the findings by Wang and Pinker (2009).They reported that the surface solar irradiances derived from MODIS agreed better with the ground-based measurements over land than at coastal and island sites.The surface height is not taken into account in the FRESCO SSI algorithm, therefore the SCIAMACHY SSI values could be lower than the global irradiances measured at the ground, especially for elevated area such as Tibetan plateau (Yang et al., 2006).The resolution of the atmospheric input does not reflect altitude effects within the respective grid size (e.g. 1 • × 1 • for aerosols), hence the effect of small scale surface height variations is not considered by the algorithm input.The missing data are due to snow/ice pixels or large solar zenith angles.The SCIAMACHY SSI seasonal image seems smoother than the monthly mean SSI image (see Fig. 10) because of a better representation of the seasonal mean due to the larger amount of measurements.

Validation of SCIAMACHY SSI against BSRN measurements
The SCIAMACHY SSI data were validated against one year of BSRN (Baseline Surface Radiation Network) measurements in 2008.BSRN stations provide observations of the best possible quality, for short-and long-wave surface radiation fluxes at 1-min sampling rate.The pyranometers used at the BSRN stations are continually checked against the highest possible scientific standards (Ohmura et al., 1998;McArthur, 2004).The estimated uncertainties in the shortwave global irradiance, as achieved by BSRN in 1995, are 5 W m −2 .This value represents calibration uncertainties, which means that operational uncertainties, referring to field conditions, are generally larger.The reason to choose the BSRN data in 2008 is because of the availability of the BSRN and SCIAMACHY SSI data.In 2008, there were nearly 40 BSRN stations globally.However, not all the BSRN stations have submitted the data measured in 2008 to the data archive; in total 20 stations were chosen for the validation.The BSRN stations used in the validation are shown in Fig. 4.More information about the BSRN stations can be found at the BSRN data center (http://www.bsrn.awi.de/).The BSRN stations at the Arctic and Antarctic were excluded, because there are no SCIAMACHY SSI data over snow/ice surface.The station SBO was not used for the validation, because the FRESCO effective cloud fraction is not accurate over bright surfaces in deserts due to the surface albedo problem (Fournier et al., 2006).Tamanrasset (TAM) is also located in the desert but the surface albedo at the Sahara desert has been corrected by Fournier et al. (2006); therefore this station has been included.Most of the selected stations are located in the Northern Hemisphere and cover a large range of surface types and topographic types.
The instantaneous SCIAMACHY SSI data were compared with the hourly mean BSRN global irradiances.According to Pinker et al. (2003), the best validation results can be obtained when both the satellite and ground-based observations are averaged over 1 h.Therefore, the measured 1-min BSRN global irradiances were averaged over 60 min centered on the SCIAMACHY overpass time to reduce large variance caused by broken cloud fields and to match the SCIAMACHY pixel size of 60 × 30 km 2 .From one year of data in 2008 we obtained 1006 collocated SCIAMACHY SSI and BSRN data points.The collocated data spanned from January to December for most of the stations, but the data for BAR, PAY and XIA were only available from July to December, January to October and January to August, respectively.
The SCIAMACHY SSI and BSRN global irradiances are shown in Fig. 5  solar zenith angle variation also improves the correlation.In general, the results are encouraging: good agreements are found between the SCIAMACHY SSI and BSRN data for all the stations.At tropical stations, e.g.COC, KWA, MAN, the global irradiances have small seasonal variations and the values are relatively large in all months.The amplitude of the seasonal variations for the global irradiances becomes larger at higher latitudes with high values in summer and much lower values in winter.The SCIAMACHY SSI has a relative large negative bias at MAN, because the effective cloud fraction is overestimated.The MAN station is located on an island with an area similar to the SCIA-MACHY pixel size but cannot be resolved in the GOME surface albedo database.Therefore, in FRESCO cloud retrieval the surface albedo of ocean is used and the effective cloud fraction is overestimated.Although COC and KWA are also on islands; compared to the SCIAMACHY pixel size the areas of the islands are negligible.There is no evidence for the overestimation of the effective cloud fraction at COC and KWA although the surface albedo of ocean is used.At TAM, SCIAMACHY SSI values are much lower than BSRN measurements; it is likely that the FRESCO effective cloud fractions are too large due to the error of the surface albedo over deserts (Fournier et al., 2006) and due to the low spatial resolution of the GOME surface albedo (Popp et al., 2011).The SCIAMACHY SSI values are lower than the BSRN measurements for the station BOS, which is located at 1689 m altitude in a hilly area.The topographic effects could cause some differences between the BSRN measurements and SCIAMACHY SSI in the hilly areas, because the FRESCO SSI algorithm does not take into account topographic effects (Lai et al., 2010).The scatter plots of SCIAMACHY SSI versus BSRN global irradiances are shown in Fig. 6 for every BSRN station.As illustrated in Fig. 6, most SCIAMACHY SSI and BSRN global irradiances are aligned along the one-to-one line.At TAM, the scatter is quite small, because there are hardly any clouds in the desert.The scatter density plot of all the validation data are shown in Fig. 7.There are a few outliers, but there is almost no bias.Because the clear-sky irradiances are calculated from the aerosol and water vapor monthly mean climatology data, the SSI variations caused by the water vapor and aerosol fluctuations on an hourly or daily scale cannot be revealed by the SCIAMACHY SSI data.This could explain some differences between the instantaneous SCIAMACHY SSI and the hourly mean BSRN measurements.The scatter of the validation results is probably mainly due to the large SCIAMACHY pixel size, particularly at the pixels with a high frequency of broken clouds and at the coast.
The selected statistical parameters from the evaluation of SCIAMACHY SSI against BSRN global irradiances are presented in Table 1.The stations are sorted according to their latitude from north to south.The comparison of SCIA-MACHY SSI with BSRN global irradiances at the SBO site is also included in Table 1, although it is not included in  the mean of all stations.The averages of the hourly mean global irradiances generally increase from north to south besides the modulations due to clouds.The standard deviations for the hourly mean BSRN global irradiances are relatively large, often greater than 50 %, for nearly every station (not included in the table).This is a clear indication of the large variations of clouds or broken clouds within the one hour time period.The bias and root mean square deviation (RMSD) are quite different for every station, which agrees with the plots shown in Figs. 5 and 6.The RMSD between SCIAMACHY SSI and BSRN global irradiances are slightly correlated with the standard deviations of the BSRN hourly mean global irradiances (r = 0.35).The measurements at BIL and E13 were made at a few kilometers from each other and the bias, RMSD and correlation coefficients were also similar for these two sites.This suggests the consistency of the BSRN and SCIAMACHY SSI data.The large negative bias of SCIAMACHY SSI at the SBO site suggests that the effective cloud fraction is too large and the surface albedo in the database could be too low.The SBO and TAM sites are both in desert regions.At SBO the mean effective cloud fraction of all the validation data is 0.314; however, at TAM the mean effective cloud fraction is only 0.044.The SCIAMACHY SSI values at SBO are systematically lower than the BSRN measurements with a mean difference of −139.6 W m −2 .The correlation coefficient between SCIAMACHY SSI and BSRN measurements is 0.794.This indicates that we have to improve the surface albedo database used in the FRESCO cloud retrieval algorithm.The mean global irradiance for all BSRN measurements is 509.0W m −2 .The mean difference between SCIAMACHY SSI and BSRN global irradiances is −4.1 W m −2 , which is −0.8 % of the mean BSRN global irradiance.The RMSD of all the collocated data is 101.1 W m −2 , corresponding to 19.9 % of the mean BSRN global irradiance.The correlation coefficient between the two data sets is 0.932 for 1006 data points.

Evaluation of monthly mean SCIAMACHY SSI with ISCCP-FD radiation data
In the previous section, SCIAMACHY SSI data have been validated with BSRN station data on an instantaneous basis.The respective validation results have demonstrated the ability of the algorithm for the generation of SSI values with good accuracy.However, due to the relative low overpass frequency of SCIAMACHY, it is interesting to investigate how good SCIAMACHY SSI monthly means perform on a global scale in relation to existing and established climatologies.Therefore, in this section SCIAMACHY SSI is compared with ISCCP-FD radiation data on a global scale.Zhang et al. (2004) have produced an 18-year data set of radiative flux profiles (called ISCCP-FD) at 3-h time steps, global at 280-km intervals, that provides full-and clear-sky, shortwave and longwave, upwelling and downwelling fluxes at five levels (surface (SRF), 680 hPa, 440 hPa, 100 hPa, and TOA).The data set has been created by employing the NASA GISS climate Global Circulation Model (GCM) radiative transfer code and a collection of global data sets describing the properties of the clouds and the surface at every 3 hours (using ISCCP products).The 280-km global interval can be converted into a 2.5 • × 2.5 • regular latitude and longitude grid.The comparisons of monthly, regional mean values from ISCCP-FD TOA and SRF fluxes with Earth Radiation Budget Experiment (ERBE), Clouds and the Earth's Radiant Energy System (CERES) and BSRN values suggest that the overall uncertainties are 5-10 W m −2 at TOA and 10-15 W m −2 at SRF (Zhang et al., 2004).The ISCCP-FD data set provides an independent global data set for the evaluation of the SCIAMACHY SSI data set.
The monthly mean SCIAMACHY SSI values were compared with ISCCP-FD SRF for one year of global data in 2006, because ISCCP-FD was not available for 2008.The 3-hourly monthly mean ISCCP-FD shortwave downwelling fluxes (SDF) at the surface were calculated from the daily 3hourly data at the 2.5 • × 2.5 • grid.The monthly mean SCIA-MACHY SSI and the corresponding cosine of solar zenith angle (SZA) were gridded at the 2.5 • × 2.5 • grid to match the ISCCP-FD grid.The ISCCP-FD SDF 3-hourly monthly means at every grid were interpolated at the SCIAMACHY measurement time according to the cosine of SZA which was included in the SCIAMACHY SSI product.The interpolated ISCCP-FD SDF was only used for the snow/ice free regions.Both SCIAMACHY SSI and ISCCP-FD provided the surface solar irradiances for the clear-and full-sky situations at every grid box, therefore the interpolations were performed on   2. For the monthly mean clear-sky irradiances, SCIAMACHY SSI are slightly higher than ISCCP-FD SDF in the summer months (June-September) with a maximum difference of +2 W m −2 and lower than ISCCP-FD SDF for the rest of the year.Including clouds, SCIAMACHY SSI values are lower than ISCCP-FD SDF for all the months.The maximum of the differences is −11.67 W m −2 (−2 %) and the maximum RMSD is 62 W m −2 (12 %).However, the mean difference is still within the uncertainty of the ISCCP-FD SDF data.The correlation coefficients are larger than 0.96 for every month.The differences could be due to the input parameters, for example, cloud parameters, water vapor, aerosols and surface albedo, and the different algorithms.To understand all the differences between the SCIAMACHY SSI and ISCCP-FD SDF products, we would have to investigate the differences between the algorithms and all the input data (Zhang et al., 2006(Zhang et al., , 2007)).

Discussions
The geostationary satellite-based surface solar irradiances derived using different versions of the Heliosat algorithm have been extensively compared with the ground-based solar radiation measurements in Europe.The same n − k relation is used in the Heliosat algorithm and FRESCO SSI algorithm, hence it can be expected that the FRESCO SSI performs well concerning its precision for instantaneous data.Indeed, the evaluated RMSD values of FRESCO SSI versus BSRN data are in the same order of magnitude as those found from the comparison of SSI derived from Meteosat using Heliosat or Heliosat-like methods with ground-based measurements (e.g.Ineichen and Perez, 1999;Zelenka et al., 1999;Dagestad, 2004;Lorenz, 2007).This demonstrates the performance of the FRESCO SSI algorithm concerning its precision.However, due to the low temporal resolution of SCIA-MACHY, outside of the polar regions we can not expect that the accuracy and uncertainty of the SCIAMACHY SSI product is similar to that of SSI daily and monthly means derived from geostationary satellites (Moeser and Raschke, 1984).
In fact, a higher uncertainty and lower accuracy could be expected in regions with a significant diurnal cycle of clouds, aerosols or water vapor.
In addition to the direct comparison with ISCCP-FD, a brief discussion of the SCIAMACHY SSI validation results in comparison with the NOAA/AVHRR products generated by the CM-SAF is given here.NOAA/AVHRR is a series of polar orbiting satellites with a significant higher spatial resolution than SCIAMACHY (4 × 4 km 2 versus 60 × 30 km 2 ).The CM-SAF radiation algorithm for the retrieval of the SSI from NOAA/AVHRR satellites also uses the MAGIC algorithm for the calculation of clear-sky irradiances, but the cloudy irradiances are calculated with a cloud model that requires detailed cloud parameters as input, which is completely different from the Heliosat method.Hollman et al. ( 2006) compared the CM-SAF AVHRR based surface shortwave global irradiances with several BSRN stations within Europe.The mean bias error was larger than 10 W m −2 for more than 50 % of the investigated monthly means, a result of large bias values in the instantaneous data.These results indicate that the FRESCO SSI algorithm might be able to generate SSI products in a similar accuracy to that derived from NOAA/AVHRR, despite of the larger SCIA-MACHY pixel size.This is a remarkable result which has to be further investigated.
The accuracy of the SCIAMACHY SSI product could still be improved through the FRESCO cloud algorithm and MAGIC algorithm.Because SCIAMACHY can provide simultaneously effective cloud fraction and water vapor at every pixel (Noël et al., 2004;Schrijver et al., 2009), it would be possible to replace the water vapor climatology with the retrieved water vapor per pixel.For the operational SCIA-MACHY SSI product, the effective cloud fraction will be retrieved assuming the cloud albedo to be 0.95 instead of the conversion approach using Eq. ( 8).The effective cloud fraction will be improved by the use of a high resolution surface albedo database, such as MERIS surface albedo database (Popp et al., 2011).
The FRESCO cloud retrieval algorithm is based on radiative transfer theory; therefore the FRESCO SSI algorithm can be applied to different satellite measurements as long as the effective cloud fraction can be derived.We already have 15 years of FRESCO effective cloud fraction time series produced from GOME and SCIAMACHY measurements (Wang et al., 2010).Therefore, we can expect a consistent SSI time series constructed from different satellite measurements using the FRESCO SSI algorithm as well.The FRESCO SSI algorithm could also use the effective cloud fractions retrieved from other cloud algorithms, such as the O 2 -O 2 algorithm (Acarreta et al., 2004) for OMI (Ozone Monitoring Instrument), and PMD algorithms for GOME, GOME-2 and SCIAMACHY with minor modifications.Such a time series might be a good alternative or complement to global SSI data sets retrieved from NOAA/AVHRR.Due to the on-board solar calibration of GOME(-2), SCIAMACHY and OMI, a high potential for the construction of a climate data record with high stability and homogeneity is present.

Conclusions
The FRESCO SSI algorithm was developed to retrieve broadband surface solar irradiances using the Heliosat method.The cloud index required in the Heliosat method was replaced by the effective cloud fraction retrieved from the O 2 A band measurements.In the FRESCO SSI algorithm, the FRESCO cloud algorithm retrieves effective cloud fraction, and then the MAGIC algorithm uses the output of the FRESCO cloud algorithm to calculate the broadband (0.2-4 µm) surface solar irradiance.The cloud albedo is assumed to be 0.95 in the FRESCO cloud algorithm, so that the effective cloud fraction value is close to the cloud index value and the Heliosat n − k relation can be used unchanged.Two years of surface solar irradiances from SCIAMACHY measurements were derived by utilizing the FRESCO SSI algorithm.This is the first time that the broadband surface solar irradiances are derived from SCIAMACHY measurements.The SSI product is also a new application of the effective cloud fraction.
The SCIAMACHY SSI data set was evaluated against the global irradiances from the Baseline Surface Radiation Network (BSRN) and the ISCCP-FD fluxes data sets.The instantaneous SCIAMACHY SSI data were validated using the BSRN hourly mean global irradiances for one year of data in 2008 at 20 BSRN stations.The mean difference between SCIAMACHY SSI and BSRN global irradiances is −4.1 W m −2 (−0.8 %) and the RMSD is 101 W m −2 (20 %) for all the stations.The correlation coefficient is 0.93 for 1006 collocated data points.The monthly mean SCIA-MACHY SSI values were compared with the ISCCP-FD surface shortwave downwelling fluxes at a 2.5 • × 2.5 • grid for 2006.The monthly mean SSI from SCIAMACHY has a good agreement with the ISCCP-FD monthly mean data, with a correlation coefficient greater than 0.96 for every month.The largest difference in the global irradiances (SCIAMACHY-ISCCP) is about −12 W m −2 (−2 %) and the RMSD is 62 W m −2 (12 %).
The validation results indicate that the instantaneous SCIAMACHY SSI data might have similar accuracy as the CM-SAF NOAA/AVHRR SSI products.This might suggest that the lower spatial resolution is compensated by a more sophisticated calibration or accurate cloud information.These hints have to be investigated and analyzed in more detail in a forthcoming paper.However, SCIAMACHY SSI has a small negative bias, mostly at tropical regions.The SCIA-MACHY SSI product over desert areas may not be reliable because of the coarse spatial resolution of the GOME surface albedo database.For areas above 4000 m the SCIA-MACHY SSI can be too low, because the surface height is not included in the algorithm.The SCIAMACHY SSI product could be improved by using the high resolution surface albedo derived from MERIS and by using simultaneous water vapor products from SCIAMACHY.
The FRESCO SSI algorithm is fast, robust and suitable for the operational processing of the SCIAMACHY data.With minor adjustments, the FRESCO SSI algorithm can be applied to GOME(-2), OMI and TROPOMI (Tropospheric Ozone Monitoring Instrument) in the future.Therefore, a consistent SSI time series can be constructed from the launch of GOME in 1995.We can also take advantage of the different overpass times of the satellites to derive the daily mean surface solar irradiances by the combination of the morning orbit (SCIAMACHY, GOME(-2)) and afternoon orbit (OMI, TROPOMI) measurements.The accuracy of the daily mean SSI would be better than the instantaneous product.
(1) calculate the effective cloud fraction; (2) calculate the clear-sky index (k) using the Heliosat n − k relation; (3) calculate the clear-sky and full-sky surface solar irradiances using k and the MAGIC algorithm.Because FRESCO SSI uses the n−k relation, it is essential to demonstrate that the effective cloud fraction value can be equivalent to the cloud index value.The flowchart of the FRESCO SSI algorithm is shown in Fig. 1.
Figure 1.Flowchart of the FRESCO surface solar irradiance (SSI) algorithm.

Figure 2 .
Figure 2. Ratio of the effective cloud fractions retrieved with cloud albedo of 0.95 (c 0.95 ) and cloud albedo of 0.8 (c 0.8 ) as a function of surface albedo (black stars) and the 3 rdorder polynomial fit y = a + bx + cx 2 + dx 3 (red line).The effective cloud fractions are

Fig. 2 .
Fig. 2. Ratio of the effective cloud fractions retrieved with cloud albedo of 0.95 (c 0.95 ) and cloud albedo of 0.8 (c 0.8 ) as a function of surface albedo (black stars) and the 3rd-order polynomial fit y = a + bx + cx 2 + dx 3 (red line).The effective cloud fractions are retrieved from the SCIAMACHY orbit over Europe, the Sahara desert and the Atlantic Ocean on 4 February 2007 (orbit number 25785).

Fig. 3 .
Figure 3. FRESCO effective cloud fraction (c0.95) and surface solar irradiance maps for 2 Fig. 3. FRESCO effective cloud fraction (c 0.95 ) and surface solar irradiance maps for June-July-August 2008 from SCIAMACHY measurements.The missing data are because of the snow/ice pixels or large solar zenith angles.

Figure 5 .Fig. 5 .
Figure 5. Instantaneous surface solar irradiances derived from SCIAMACHY (black line) 2 and the hourly mean BSRN global irradiances (red line) as a function of day of year for 3 every station.The number of cases is from January to December 2008, except for BAR, 4 PAY, XIA. 5

Figure 7 .
Figure 7. Scatter density plot of the instantaneous SCIAMACHY surface solar irradiance (SSI) versus the hourly mean BSRN global irradiances for all the collocated data in 2008.The dashed line is the one-to-one line.The color scale indicates the logarithm of the number density of the data points.

Fig. 7 .
Fig. 7. Scatter density plot of the instantaneous SCIAMACHY surface solar irradiance (SSI) versus the hourly mean BSRN global irradiances for all the collocated data in 2008.The dashed line is the one-to-one line.The color scale indicates the logarithm of the number density of the data points.

Table 1 .
Instantaneous SCIAMACHY SSI evaluation results using the BSRN global irradiances in 2008.Column 4: BSRN W m −2 is the mean BSRN data used for validation at every station, the average of the BSRN data used in Figs.5 and 6.Bias = SCIAMACHY SSI -BSRN, RMSD = root mean square deviation, COR = correlation.
water vapor or aerosols values (from the climatology databases) are used in the MAGIC algorithm.
That analysis is beyond the subject of this paper.The SCIAMACHY SSI data set has a negative bias at the tropical regions compared to the BSRN measurements and ISCCP-FD SDF.It implies that the FRESCO effective cloud fractions might be too high at the tropical regions or www.atmos-meas-tech.net/4/875/2011/Atmos.Meas.Tech., 4, 875-891, 2011 too high