Extension of AVHRR-based climate data records: exploring ways to simulate AVHRR radiances from Suomi NPP VIIRS data

The long series of multispectral measurements from the Advanced Very High Resolution Radiometer (AVHRR), which began in 1979, is now approaching its end, with the last remaining AVHRR sensor currently operating aboard EUMETSAT's Metop-C satellite. Several climate data records (CDRs) built on AVHRR data now face the end of their observational record. However, since many modern imagers contain AVHRR-heritage spectral channels, the potential for an extension of these AVHRR-based climate data records exists. This study investigates the possibility of simulating original National Oceanic and Atmospheric Administration-19 (NOAA-19) AVHRR channels from the Suomi National Polar-orbiting Platform (NPP) Visible Infrared Imaging Radiometer Suite (VIIRS) radiances using collocated AVHRR–VIIRS datasets from 2012–2013. Spectral band adjustments (SBAs) were derived using linear regression and neural networks (NNs). The NN approach produced the best results, and separating daytime from night-time conditions when simulating AVHRR channel 3B at 3.7 µm was key. Furthermore, daytime radiance corrections in this channel must depend on actual surface and cloud reflectances to be realistic, which was achieved only through the NN approach.

The cloud mask, cloud top height, and cloud phase products were produced from the simulated AVHRR radiances using the same retrieval methods for NOAA-19 data used to compile the CLARA-A3 CDR. CLARA-A3 is the third edition of the EUMETSAT Climate Monitoring Satellite Application Facility (CM SAF) CDR, with cloud parameters, surface albedo, surface radiation, and top of atmosphere (TOA) radiation products from AVHRR. Products were validated using Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations–Cloud-Aerosol Lidar with Orthogonal Polarization (CALIPSO–CALIOP) cloud products and agreed well with the original CLARA-A3 products, with the best results provided by the NN simulation approach. The NN-based approach best reproduced the corresponding products for cloud optical thickness (COT), cloud effective radius (CRE), liquid water path (LWP), and ice water path (IWP).

The CLARA-A3 CDR will be complemented and extended with VIIRS-based products to cover the period 1979–2024 (46 years). This edition will be known as CLARA-A3.5. Future extensions and editions can follow a similar approach by applying the same radiance simulation method to collocated data from the Metop-C AVHRR and the Metop Second Generation (SG) METimage sensors, with the first satellite of the latter scheduled for launch in August 2025. The successful simulation of AVHRR radiances from METimage and VIIRS data enables the CLARA CDR extension for several decades.

1 Introduction

Successful climate monitoring depends on the availability of long observational time series from reliable and stable observation platforms and sensors. Observations with a very long temporal coverage (i.e. at the century scale) have been mainly restricted to measurements from land-based surface stations and mostly limited to 2 m temperature measurements (e.g. Morice et al., 2021). For even longer perspectives, various proxy observations must be used (e.g. tree ring and sediment climatologies; Anchukaitis et al., 2017).

However, to fundamentally describe and understand climate and climate change, global observations at high spatio-temporal resolution are needed. Furthermore, a full range of different meteorological parameters need to be covered. The first steps towards realizing an observation system with a truly global coverage were taken when information from polar and geostationary satellites was introduced in the 1960s. These sensors were later upgraded and introduced in operational missions by the end of the 1970s (Kidd et al., 2009; Giri et al., 2025). Additional observations with better coverage of ocean surfaces and upper air were introduced through various technological developments (Lin and Yang, 2020; WMO, 2024; NDACC, 2024). Furthermore, the systematic use of radiation network measurement data (NDACC, 2021) from active and passive remote sensing instruments at surface stations and on space platforms is now standard (Thies and Bendix, 2011; eoPortal, 2024). All these developments made it possible to compile comprehensive and consistent climate datasets by synthesizing data from all types of observation platforms in reanalysis datasets (Hersbach et al., 2020).

Reanalysis datasets are undoubtedly capable of providing the best possible description of the earth's atmospheric and surface state evolution, at least over the last 3–5 decades, with access to a multitude of global observations and the use of a physically consistent methodology based on model physics constraints. However, because of the use of data from an ever-changing observation system, not least after the introduction of several new or improved satellite sensors over the last few decades, the uncertainty regarding the existence and magnitude of climate trends in reanalysis results is still considerable (Bengtsson et al., 2004; Thorne et al., 2005; de Padua and Ahn, 2024; Tarek et al., 2021). In addition, some parameters of great importance to the earth's radiation balance are not yet fully assimilated from observations. This concerns, in particular, cloudiness and cloud properties (Yao et al., 2020). Furthermore, the reanalysis dependency on physical constraints from the current numerical weather prediction (NWP) model means that the results are not completely independent, since model physics cannot be considered as perfectly describing the real atmosphere/earth system (as pointed out by Roebeling et al., 2025).

With this background, the value of a long time series of single-sensor observations or measurements for climate studies would still be high. This concerns, in particular, climate data records (CDRs) from satellite platforms, where several of them now cover considerably longer periods than the standard World Meteorological Organization (WMO) climatological 30-year period. The Advanced Very High Resolution Radiometer (AVHRR), operating on board polar satellites since 1978, provides the longest available time series of observations from meteorological satellite imagery. The third edition of the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) Climate Monitoring Satellite Application Facility (CM SAF) CDR, with cloud parameters, surface albedo, surface radiation, and top of atmosphere (TOA) radiation products from the AVHRR (CLARA-A3; see Karlsson et al., 2023), covers more than four decades of AVHRR observations. Neither AVHRR radiances nor AVHRR-derived cloud and radiation products have yet been assimilated in reanalysis datasets, which is an additional argument for their value as an independent observation dataset. However, the last AVHRR instrument was launched with the EUMETSAT satellite Metop-C in 2018. Thus, the AVHRR era will soon be over.

This paper investigates methods to extend the CLARA CDR with data from the AVHRR successor – the Visible Infrared Imaging Radiometer Suite (VIIRS) – which is now operational on the current polar meteorological satellites from NOAA. If these methods are successful, the CLARA CDR can be extended by at least 2–3 decades. The paper studies two approaches to simulate AVHRR radiances from VIIRS, using spectral band adjustment factors (SBAFs): (1) linear regression and (2) a method that uses a multilayer perceptron (MLP) neural network. As a further test of success besides ordinary radiance-to-radiance comparisons, the simulated radiances are used to produce the CLARA cloud properties, which are then validated using independent cloud observations from the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) on board the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite (Winker et al., 2009).

Section 2 describes the methodological background and the datasets used. The methodology is then described in detail in Sect. 3, followed by results in Sect. 4. Further analysis and discussions are presented in Sect. 5, with conclusions given in Sect. 6.

2 Methodological background and selected datasets

2.1 Introduction to spectral band adjustment methods

Adjusting measurements after introducing a slightly modified or new sensor version poses a long-standing challenge that has received considerable attention over the years. Most well known are the activities of the Global Space-based Inter-Calibration System (GSICS; WMO, 2025), where the primary goal is to ensure a homogeneous behaviour of measurement time series from a particular sensor or spectral channel. We call this adjustment “inter-calibration” (Chander et al., 2013a), and the purpose here is to provide a homogenous data record without artificial discontinuities. The fidelity and uncertainty in climate data records from the earth observations project (FIDUCEO) emphasized the difference between homogenized and harmonized datasets (Giering et al., 2019), with relevance for the CDR compilation. Harmonized data would imply corrections to a measurement based on high-quality reference measurements, thus providing the best possible estimation of the measured radiance. This correction would still allow differences to a similar instrument having slightly different spectral responses. However, for a CDR, which should allow for climate trend estimation, homogenized data seemingly should be the best choice. Homogenized data for a CDR means that measurements are corrected with respect to one particular sensor in the measurement series instead of to one high-quality reference sensor. On the other hand, this could also lead to sensor accuracy violation (if the various sensors have significant differences in spectral response). Thus, there are pros and cons of both spectral adjustment methods, and any of them should be applied with caution. Another important aspect is that radiance differences between two sensors might be caused by additional factors other than differences in spectral responses, e.g. radiance biases or calibration errors.

When focusing on the current problem to simulate AVHRR from VIIRS radiances based solely on spectral response differences, no spectral adjustment methodology will ever be able to simulate the AVHRR channels perfectly, since some parts of the spectrum covered by another AVHRR-heritage sensor channel are simply not observed by the corresponding AVHRR channel (and vice versa). However, if channel differences are not very large, corrections may be sufficient, depending on the intended applications. Piontek et al. (2023) estimated that linear SBAFs can explain more than 80 % of the variance, but the efficiency depends on the selected channels.

For many years, the standard methodology to handle these spectral band adjustments has been to calculate SBAFs. These can be derived from direct inter-comparisons of spatio-temporally collocated measurements from the two sensors (e.g. as described by Meirink et al., 2013). Relations based on SBAFs can be either linear (Chander et al., 2013b) or non-linear and sometimes more complicated, involving more channels than the targeted spectral channel (Villaescusa-Nadal et al., 2019; Claverie, 2023). When collocations are not possible, spectrometer data can be used for calculations, most often relying on data from the SCanning Imaging Absorption spectroMeter for Atmospheric CHartographY (SCIAMACHY, Bovensmann et al., 1999) or the Infrared Atmospheric Sounding Interferometer (IASI; Blumstein et al., 2004) for meteorological applications. Hyperspectral observations are then convolved with the narrow-band spectral response function (SRF) to calculate the SBAFs (Piontek et al., 2023). The NASA Satellite Cloud and Radiation Property Retrieval System (SatCORPS) SBAF tool is a comprehensive and widely used web-based tool based on this technique, providing SBAFs for a wide range of sensors and satellites (NASA, 2016; Scarino et al., 2016).

2.2 The challenge: bridging differences between the spectral channels of AVHRR and VIIRS

Table 1 lists the AVHRR channels simulated in this study. Notice that our reference sensor is the third version of this sensor (AVHRR/3), as carried by the NOAA-19 satellite. The choice of the AVHRR on NOAA-19 as our reference sensor is natural, since we want to replace the loss of NOAA-19 observations in the afternoon orbit after 2012, due to orbital drift. Section 2.5 provides an even stronger motivation for choosing NOAA-19. It should also be mentioned that the reference radiances for NOAA-19 AVHRR should be considered as harmonized data, since their quality and evolution over time has been optimized for this particular AVHRR sensor by a method described by Heidinger (2018).

In this study, we are not interested in simulating AVHRR channel 3A, as shown in Table 1. The reason is that satellites carrying the VIIRS sensor follow an afternoon orbit, a sun-synchronous path with a daytime Equator crossing shortly after noon. For all earlier satellites used in the CLARA-A3 CDR, where the AVHRR is in a similar orbit to VIIRS, only AVHRR channel 3B was available (active). The AVHRR observations have a swath width of 2600 km, and the horizontal resolution is approximately 1.1 km at the nadir. However, it is much coarser (approximately 6 km) at the swath edges. Cracknell (1997) provides more details on the AVHRR imager.

Table 1 Main AVHRR/3 sensor spectral characteristics. Note that AVHRR channel 3A (marked in italics) is not subject to spectral conversion here (see text for explanation).

Table 2 gives the complete set of medium-resolution channels (M-channels) of the VIIRS imager (described in more detail by Hillger et al., 2013). The swath width is 3000 km, the horizontal resolution is 750 m at the nadir and only slightly less (1.6 km) at the swath edges due to an oversampling scanning technique that is different from the AVHRR. The AVHRR-heritage channels are marked in bold numbers in Table 2. In theory, AVHRR channel 2 may be simulated using a combination of channels M6 and M7. However, saturation problems with channel M6 (as reported by Cao et al., 2013) produce unrealistic measurements, making it unsuitable for this purpose.

Table 2 Main spectral characteristics of the medium-resolution (M) channels of the VIIRS sensor. The AVHRR-heritage channels are marked in bold. Notice that the channels marked in bold italics are not used (see text for explanation).

For a better visualization of the differences between the two sensors' channels, we can study the differences in spectral responses, as illustrated in Fig. 1. It is clear that there are indeed substantial differences for most channels, except possibly for AVHRR channels 4 and 5. However, the different wavelength scales at the x axes in the plots in Fig. 1 tend to exaggerate differences for some channels (e.g. AVHRR channel 1) and underrate differences for other channels (e.g. AVHRR channel 2). AVHRR channel 2 shows the most significant difference, with a broader spectral coverage than VIIRS AVHRR-heritage channels (M6 and M7).

2.3 VGAC – reduced-resolution VIIRS data

The CLARA-A3 CDR is based on the archived global AVHRR dataset stored in a format called Global Area Coverage (GAC), with a horizontal resolution of approximately 4 km (Kidwell, 1991). Extending CLARA-A3 with VIIRS-derived products requires resampling VIIRS data to an equivalent horizontal resolution. This process benefits from a resampled VIIRS dataset already developed at NOAA (Knapp et al., 2019). This format is called VIIRS Global Area Coverage (VGAC), and VIIRS data in this format are currently available for almost the entire Suomi NPP record and some years of the NOAA-20 data record. The horizontal resolution is 3.9 km, and the resampling procedure (e.g. radiance averaging) is improved compared to the original GAC format for the AVHRR. The VGAC data have already been tested for use in CDR production (Wang et al., 2023; Seo et al., 2025). For this study, we used Suomi NPP VGAC data from 2012, 2013, and 2019, as well as the NOAA-20 VGAC data from 2019.

Figure 1 Comparison of spectral responses for AVHRR (NOAA-19) and AVHRR-heritage channels of VIIRS (Suomi NPP). Spectral response curves of AVHRR channels are given in blue to be compared with the response curves from VIIRS channels in red. The corresponding AVHRR-heritage channel notations (M5, M6, M7, M8, M12, M15, and M16) are provided at their central wavelengths along the x axis. The grey curves give the atmospheric transmittance for reference.

2.4 Selected approach

This study initially tested various SBAF relations, primarily sourced from NASA (2016). Results were acceptable for most AVHRR channels, but for some channels (especially channel 3B at 3.7 µm), we encountered problems in using the VIIRS-based simulations. For example, night-time cloud detection significantly overestimated low-level cloud amounts. The cloud detection method used (CMAPROB, described by Karlsson et al., 2020) is a probabilistic method using all AVHRR channels. AVHRR channel 3B is considered the most crucial channel for this method's performance, especially at night. Only minor deviations from original AVHRR channel 3B radiances significantly affect the results at night. The encountered problems were likely caused by the limitation of IASI not observing radiances for wavelengths shorter than 3.62 µm. Since AVHRR channel 3B and VIIRS band M12 spectral responses both allow for significant contributions at wavelengths shorter than 3.62 µm (see Fig. 1), this limitation can be substantial, especially since this affects in particular the contribution from reflected solar radiation, which rapidly increases with decreasing wavelengths. An effort to describe these contributions using radiative transfer model (RTM) calculations was applied in the SatCORPS tool, but this was made using assumptions, making results more uncertain.

Due to the uncertainties encountered for the NASA-derived SBAFs for this channel, we decided to proceed by calculating SBAFs from collocated AVHRR- and VIIRS-observed radiances. In practice this means that the derived spectral band adjustments might be composed of more than just the effects of differences in spectral responses, since we cannot separate these effects from other effects (e.g. radiance biases or calibration errors) when doing collocations.

2.5 Selected collocation and validation datasets

Since the VIIRS on the Suomi NPP satellite was already launched in 2011 (with useful data delivered from January 2012 onwards), collocations with the AVHRR measurements have been possible for more than a decade (Fig. 2). However, the only satellite allowing nearly simultaneous overpasses covering the entire globe was NOAA-19. In 2012 and 2013, NOAA-19 had nearly the same orbital configuration as the Suomi NPP, with a daytime Equator crossing overpass time near 01:30 pm (i.e. local solar time of 13:30 in Fig. 2). After 2013, the orbital drift of the NOAA-19 satellite gradually restricted available collocations to higher latitudes. Notice also that for later satellites carrying the VIIRS (i.e. NOAA-20 launched in 2018 and NOAA-21 launched in 2022), these satellites still have fixed Equator crossing times at 01:30 pm. Consequently, no further global collocations with AVHRR were possible.

Since NOAA-19 and the Suomi NPP had slightly different orbital altitudes and thus slightly different orbital periods, simultaneous observations (in this case, limited to within 2 min) occurred approximately every 15th day. Considering some data losses for both satellites, this resulted in a total collocation dataset of 115 Suomi NPP orbits and corresponding NOAA-19 orbits for the 2 years. At most, pixel data were separated by 3 km and maximum satellite zenith angles of up to 15° to avoid too much influence from directional effects. Collocations were calculated using nearest neighbour matching with the Pyresample module in the Pytroll software package (Raspaud et al., 2018). Table 3 lists the training and validation datasets used.

The linear SBAF regression methods utilized the entire training dataset (dataset 1 and 2). For the neural network (NN) approach, training dataset 1 was used for the actual training, and dataset 2 was used as a “during training validation dataset” to decide when to stop the training. The radiance validation dataset was used to evaluate the performance of all SBAF approaches.

For the NN training, special handling of AVHRR channel 3B for very low brightness temperatures (BTs) had to be applied. The original AVHRR data have a rather poor radiometric resolution at very low temperatures, while the corresponding data from the M12 channel of VIIRS are much improved compared to the AVHRR data. The poor radiometric resolution for AVHRR in this channel often led to missing data or a very large spread of realized BTs. It was decided not to let the network learn this type of irregular behaviour of the AVHRR. Thus, data points with missing channel 3B data or data with BTs below 220 K were slightly modified using a simple relation between the VIIRS and AVHRR BTs. Before training, the AVHRR BTs for these pixels were set to the original VIIRS M12 BTs plus the difference between AVHRR channel 4 and VIIRS M15 BTs, hereby assuming that the difference in BTs for the 11 µm channels should be the same as the difference for the 3.7 µm channels. This would stop the network from trying to learn these deficiencies of AVHRR channel 3B for temperatures colder than 220 K.

In addition to validating the VGAC-simulated AVHRR radiances, inspecting the performance of certain CLARA products derived from these radiances proved necessary. As noted earlier, achieving a perfect simulation of the AVHRR radiances is theoretically impossible. Therefore, it was important to check if the derived products were good enough to comply with the CLARA product requirements. Thus, we produced the CLARA-A3 cloud products from the simulated AVHRR radiances. Comparisons with the original NOAA-19 products (cloud property inter-comparison dataset in Table 3) and cloud datasets from the CALIPSO satellite (cloud product validation datasets in Table 3) were used to investigate them. More details on cloud product validations are given in Sect. 3.3.

Figure 2 Local solar times at Equator observations for all AVHRR-carrying NOAA satellites from TIROS-N to NOAA-19 and EUMETSAT's METOP A/B/C satellites. Shown are all data that were used for the CLARA-A3 CDR processing. The figure shows ascending (northbound) Equator crossing times for afternoon satellites (NOAA-7 to NOAA-19) and descending (southbound) Equator crossing times for morning satellites (NOAA-12 to NOAA-17 and METOP A/B/C). Corresponding night-time observations take place 12 h earlier/later. Take note that the Suomi NPP has a stable orbit with the Equator crossing time at 13:30 LT.

Table 3 Description of the training and validation datasets used. The table shows the number of selected Suomi NPP and NOAA-20 orbits being compared with the corresponding AVHRR radiances and cloud products; alternatively, (for cloud product validation) the number of orbits being validated with CALIPSO cloud products. See text for details.

3 Detailed description of applied spectral band adjustment methods and validation procedures

3.1 Spectral band adjustment (SBA) methods

We have tested two different SBA methods for the simulation of NOAA-19 AVHRR radiances from Suomi NPP VIIRS radiances.

1. SBAs derived from linear regression

2. SBAs derived from a multilayer perceptron (MLP) neural network

Method 1 is the classical regression method, often used in inter-calibration applications relating two nearby spectral channels, in which measurements can be collocated during an overlapping period. The two training datasets described in Table 3 were merged and used to estimate the regression parameters. The method has been applied with two different configurations.

•  

  Linear-1a: linear regression based on all training samples for individual channels

•  

  Linear-1b: linear regression separating results for day, night, and twilight

Configuration Linear-1b accounts for the fact that some channels might behave differently during the day and night. This concerns, in particular, AVHRR channel 3B at 3.7 µm, which exclusively measures thermally emitted radiation at night but both thermally emitted and reflected solar radiation during the day. We have defined night as solar zenith angles (SZAs) above 89°, twilight for SZAs between 80 and 89°, and day as SZAs below 80°.

Appendix A provides linear regression parameters for both methods.

Method 2 (hereafter described as the NN-based method or SBA-NN) explores whether multichannel information from VIIRS can be used to simulate individual AVHRR channel radiances, taking illumination conditions into account (in the same way as for the Linear-1b method). The NN-based method was trained on all training samples, allowing dependence on multiple input channels for each target channel, separated by night, twilight, and daytime conditions. The following section further describes this method and its training.

3.2 Definition and training of the SBA-NN

This study used multilayer perceptrons of the type quantile regression neural networks (QRNNs; Pfreundschuh et al., 2018; Cannon, 2011). These have been successfully used to retrieve cloud top height parameters from polar satellite imagery (Håkansson et al., 2018) in the EUMETSAT Nowcasting Satellite Application Facility (NWC SAF) project. The resulting cloud top height products were also used later in CLARA-A3. Separate networks were trained for day, night, and twilight conditions. As input to the AVHRR training, the network had individual channels (as reflectances and brightness temperatures), channel differences (i.e. brightness temperature differences, or BTDs), and channel ratios from VIIRS (e.g. M10 reflectance divided by M7 reflectance). As truth to train against, the same variables for the AVHRR were used (Table 4). However, the channels at 1.6 and 8.5 µm are not available on NOAA-19 AVHRR, which were consequently used only as input to the network. We omitted the variables concerning the reflective channels during night-time conditions. The reason for using primarily channel differences and ratios between channels in the training, rather than just the original set of all individual channels, is that many of the downstream applications for deriving CLARA products (e.g. cloud mask, cloud optical thickness, effective radii, and surface albedo) rely heavily on relations between two or more AVHRR channels. Thus, simulating these relations between channels as closely as possible is important.

The networks were trained for three distribution quantiles: 16 %, 50 %, and 84 %, which estimate the retrieval error. The sequential model from Keras/Tensorflow (Joseph et al., 2021) defined the networks, with the MLP settings detailed in Table 5. The code for the training and the resulting networks is available on GitHub (https://github.com/foua-pps/sbafs_ann, last access: 14 July 2025). Table C1 in Appendix C provides the details on the final networks.

The data underwent thinning before training. The input data range was calculated and divided into 10 equal bins for each variable, from which 1000 data points were randomly selected per bin. Bins with fewer than 1000 points were supplemented with additional random selections to reach exactly 400 000 data points for the night-time network and 700 000 for each daytime and twilight network. Data thinning aimed to give rare but important data points – such as cloud-free snow-covered surfaces and hot deserts – larger representation in the training data.

Table 4 Training variables for the MLP network.

Table 5 MLP network settings.

3.3 Evaluation methods

The results for the simulated radiances are primarily evaluated by inter-comparing radiances in scatter plots and calculating appropriate scores of radiance agreement. This evaluation is based on the radiance validation dataset in Table 3.

An equally important way of evaluating the results is to investigate the impact on some central CLARA CDR cloud products to verify that the products derived from the simulated AVHRR radiances fulfilled the essential CLARA-A3 CDR product requirements. A favourable condition for the chosen period was the possibility of collocating VIIRS and AVHRR orbits with observations from the CALIPSO satellite (also having Equator crossing times near to 01:30 pm). Consequently, cloud products derived from VIIRS-simulated AVHRR data could be efficiently validated using CALIPSO–CALIOP cloud products (Winker, 2016). Thus, the validation results for the cloud parameter cloud fractional cover (CFC in %), cloud top height (CTH, in metres), and cloud phase (CPH, meaning the percentage of liquid phase cloud tops) could be derived. The validation methods were the same as those described by Karlsson and Håkansson (2018).

The processing of the CLARA CDR cloud products is based on the software package for the Polar Platform system (PPS), originally developed in the EUMETSAT NWC SAF project (https://www.nwcsaf.org/web/guest/home, last access: 14 July 2025). PPS enables cloud product processing for a wide range of imagers. The shift from one sensor to another is generally dealt with by adjusting pre-calculated cloud detection thresholds, atmospheric corrections, and other adaptations from mainly RTTOV simulations utilizing each sensor's spectral response functions.

We used a variety of validation scores depending on the investigated parameters (all scores described in detail by Karlsson and Håkansson, 2018). For the cloud mask evaluation, we first converted cloud probabilities to a binary cloud mask using the probability threshold of 50 %. Then, we calculated the overall cloud fractional cover (CFC) to be compared with the CFC calculated from the CALIPSO cloud products. The mean error (bias in percentage points − %), the dimensionless hit rate, and the Kuipers scores were then calculated. Since the CALIOP cloud lidar on CALIPSO is more sensitive to cloud occurrence than AVHRR, the Kuipers and hit rate scores were also calculated for CALIOP observations, with the thinnest clouds excluded (i.e. excluding contributions from cloud layers with cloud optical thicknesses < 0.20). For CTH, the cloud top of the topmost cloud layer detected by CALIOP was used as the validation reference. Computed validation scores for CTH were mean error and mean absolute error (both in metres). Also here, the effect of removing the thinnest clouds from the CALIPSO dataset was studied but now using a slightly relaxed cloud optical thickness threshold of 0.4, claiming that the prospect of retrieving a cloud top height requires that clouds are also detected with reasonable confidence.

Notice that the Suomi NPP–CALIPSO matchups for the years 2012–2013 (i.e. basically the cloud product validation dataset for the Suomi NPP in Table 3) are independent of the AVHRR–Suomi NPP matchups (i.e. the two training datasets for the Suomi NPP) in the same period. More clearly, the likelihood of simultaneous nadir observations from all three platforms is extremely low.

In addition to evaluating the performance of cloud products for 2012 and 2013, we also made the same validation effort based on CALIPSO data for all months in 2019. The purpose was mainly to choose a period with more independent data, well separated in time from the 2 years when SBAF methods were initially derived. Thus, we wanted to see if the results were still valid for the Suomi NPP data 6 years later. In addition, we also wanted to see if the next VIIRS sensor on NOAA-20, with slightly different spectral responses for some involved channels, would produce results with similar quality. This validation effort used data from cloud validation datasets from 2019, as shown in Table 3.

As a final confirmation that simulated CLARA-A3 cloud products would perform satisfactorily overall, we also made inter-comparisons between the original and simulated AVHRR cloud property products for the NOAA-19/Suomi NPP matchups in the year 2012. Cloud property products COT, CRE, LWP, and IWP could not be validated in the CALIPSO validation study. Comparing them with the original AVHRR products indicates their validity, even though the validation dataset (i.e. the cloud property inter-comparison dataset in Table 3) partly includes cases used for training the methods. The analysis also computes agreement scores similar to those defined for the radiance inter-comparisons.

Figure 3 Radiance inter-comparisons for AVHRR channels 1 (0.6 µm, a–c) and 2 (0.9 µm, d–f). Original AVHRR vs VIIRS reflectances (%) are shown in (a) and (d). AVHRR vs VIIRS-simulated reflectances are shown in (b), (c), (e), and (f). (b) and (e) show results from the Linear-1b method, while (c) and (f) show simulated results from the SBA-NN method.

4 Results

4.1 Results for simulated AVHRR radiances

Figure 3 shows scatterplots with results of the simulated AVHRR reflectances for the two visible AVHRR channels (channels 1 and 2; see Table 1). For the linear regressions, we only show results of version Linear-1b, since the two linear versions give almost identical results for the visible channels. Results are compared to the original reflectances from AVHRR and VIIRS. It is clear that original channel reflectances are highly correlated in both sensors, although VIIRS reflectances are generally slightly higher than AVHRR reflectances. The relatively large spread around the identity line is largely explained by remaining but small collocation errors in space and time. Furthermore, sampling differences in how radiances are averaged in AVHRR GAC and VIIRS VGAC fields of view (FOV) can also contribute to the scatter.

Simulations with the linear method improve the agreement (i.e. bring results closer to the identity line) in Fig. 3 for both channels but appear to create overcompensated results for high reflectances. The NN-based method delivers the best results, effectively removing most of the off-diagonal deviations observed in the linear method.

Figure 4 Radiance inter-comparisons for AVHRR channel 3B (3.7 µm, a–c), channel 4 (11 µm, d–f), and channel 5 (12 µm, g–i). The figure shows merged results for all daytime categories. Original AVHRR vs VIIRS brightness temperatures (K) are shown in (a), (d), and (g). AVHRR vs VIIRS-simulated brightness temperatures are shown in (b), (c), (e), (f), (h), and (i). (b), (e) and (h) show results from the Linear-1b method, while (c), (f), and (i) show simulated results from the SBA-NN method.

As seen in Fig. 1, the spectral responses for VIIRS and AVHRR for the infrared channels 4 and 5 are very similar, which is also verified by the results in the middle and bottom panels of Fig. 4. Original and simulated results for the two AVHRR channels are here more or less identical. Some deviations are, however, seen for AVHRR channel 3B (upper panel in Fig. 4), and the NN-based method is somewhat better than the linear method for reducing these deviations.

Although AVHRR simulations for the infrared and shortwave infrared channels appear to agree well with the original AVHRR radiances in Fig. 4, we must emphasize that for some CLARA applications (like cloud detection), the quality of the simulated AVHRR channel differences is very important. Similarly, estimating surface albedo in CLARA requires accurate inter-channel relations (i.e. reflectance ratios) between the two visible channels. Figure 5 closes in on those features.

The close-up of differences and ratios in Fig. 5 reveals deviations from the identity line for original and simulated results. The brightness temperature differences for the three original infrared and shortwave infrared channels are relatively large. The linear approach somewhat improves the agreement, but the NN-based method can almost completely remove the differences. The linear method cannot improve the agreement for the reflectance ratio, while the NN-based method provides an excellent agreement.

Figure 5 Results for selected channel differences and ratios. The figure shows merged results for all three daytime categories. (a)–(c) show results for original and simulated brightness temperature differences (K) between AVHRR channels 4 and 5 (11–12 µm). (d)–(f) show results for original and simulated brightness temperature differences (K) between AVHRR channels 4 and 3B (11–3.7 µm). (g)–(i) show results for original and simulated reflectance ratio between AVHRR channels 2 and 1 (0.9/0.6 µm). Original AVHRR and VIIRS relations are shown in (a), (d), and (g); results for the Linear-1b method in (b), (e), and (h); and results for the NN-based method in (c), (f), and (i).

Figure 6 summarizes the quality (uncertainty) of simulated results, i.e. the difference distribution between simulated and real AVHRR results for the studied methods based on the same datasets as in Figs. 3, 4, and 5. Notice that the figure also shows results for the previously illustrated channel differences and ratios and results for the Linear-1a method (not shown in Figs. 4 and 5). For reference, the figure also shows the original deviation between the AVHRR and VIIRS channels.

We immediately notice that the SBA-NN method is superior to all other methods by showing the highest frequency at the zero-difference level (dark purple curves in Fig. 6). This method creates no secondary peaks, and the distribution is clearly non-Gaussian (i.e. the grey distribution indicated in Fig. 6 is Gaussian, with the same bias and standard deviation). Notice, in particular, that this method has excellent results for channel differences and ratios (lowest row in Fig. 6).

Results in Fig. 6 clearly show that AVHRR channel 3B simulations require separate handling of day and night measurements. If the simulation method is not different between day and night (i.e. as in Linear-1a), the error difference clearly peaks outside the zero level (e.g. green curves in the leftmost plots in the middle and bottom rows in Fig. 6).

Tables B1–B4 in Appendix B summarize all results for the selected overall statistical measures. Appendix B also provides results subdivided for day, night, and twilight conditions.

Figure 6 Frequency distributions of differences between simulated and real AVHRR radiances and brightness temperatures. For reference, original AVHRR and VIIRS channel deviations are also shown (dotted green curve: no SBA in the legend). (a)–(e) show results for all five AVHRR channels, while (f)–(h) show results for BTDs (channel 4 to channel 5 and channel 4 to channel 3B) and the reflectance ratio (channel 2 divided by channel 1).

4.2 CALIPSO-based validation of derived CLARA cloud products for 2012 and 2013

Tables 6–8 show results from the CALIPSO–CALIOP validation of the three CLARA-A3 cloud products based on VIIRS-VGAC-simulated AVHRR radiances. Section 3.3 describes the general validation setup and the validation scores used.

The tables contain validation results for eight different validation setups.

1. NOAA-19 CLARA-A3

   Achieved validation results in a previous validation of the CLARA-A3 products for products generated from the NOAA-19 AVHRR instrument over 2012–2013. These results are included, since this study aims to produce results from VIIRS-VGAC data compatible with earlier NOAA-19 AVHRR-based products.

2. PPS VIIRS

   Validation results based on uncorrected AVHRR-heritage channels of VIIRS-VGAC data in 2012–2013. Notice that this category means that PPS cloud products have been produced in the VIIRS environment (i.e. applying pre-calculated cloud detection thresholds, atmospheric corrections, and other adaptations using spectral responses of the five AVHRR-heritage channels of VIIRS).

3. VGAC no SBA

   Similar to the previous category but is now based on PPS cloud products produced in the AVHRR environment (i.e. applying pre-calculated cloud detection thresholds, atmospheric corrections, and other adaptations using spectral responses of the original five NOAA-19 AVHRR channels). This environment is also used for all following categories.

4. VGAC Linear-1a

   Validation results based on the VIIRS-VGAC simulation of AVHRR channels based on the Linear-1a method in 2012–2013.

5. VGAC Linear-1b

   Validation results based on the VIIRS-VGAC simulation of AVHRR channels based on the Linear-1b method in 2012–2013.

6. VGAC SBA-NN

   Validation results based on the VIIRS-VGAC simulation of AVHRR channels based on the SBA-NN method in 2012–2013.

7. VGAC SNPP 2019 SBA-NN

   Validation results based on the Suomi NPP VIIRS-VGAC simulation of AVHRR channels based on the SBA-NN method in 2019.

8. VGAC NOAA-20 2019 SBA-NN

   Validation results based on the NOAA-20 VIIRS-VGAC simulation of AVHRR channels based on the SBA-NN method in 2019.

The tables also show the original requirements for the three cloud products in the CLARA-A3 CDR in the rightmost columns. Products generated from VIIRS-VGAC-simulated data should also fulfil these requirements. In the CLARA-A3 evaluation, very thin clouds detected by CALIPSO–CALIOP were removed based on COT thresholds of 0.2 and 0.4 for CFC and CTH, respectively. Karlsson and Håkansson (2018) suggested this CFC COT threshold after studying the effects of COT thresholding during the CLARA-A2 CDR CFC validation exercise. They found the best overall validation scores when excluding clouds with COT lower than this threshold. This COT thresholding was later used to define the CLARA-A3 CFC requirement (Table 6, rightmost column). Here, using different CFC and COT thresholds (Tables 6 and 7) is motivated by wanting to discard the thinnest clouds when validating the CTH product. These clouds are always the most difficult to deal with for any CTH retrieval, i.e. the thinner the clouds, the more challenging it becomes to compensate for semi-transparency effects. The COT threshold for CTH validation is more arbitrarily chosen. It should be higher than 0.2, although not drastically, since cloud detection efficiency increases rapidly for COTs larger than 0.2 (see Karlsson and Håkansson, 2018). A reasonable threshold of 0.4 was found to remove some of these thin cloud uncertainties. The threshold should not be too high to give justice to all semi-transparency correction efforts, i.e. it would not make sense to only look at opaque clouds. To highlight improvements more clearly, Tables 6 and 7 show both COT thresholded results used for CLARA-A3 requirements and original results that include optically thin clouds. Relying only on COT thresholded results would overlook important improvements to the overall CDR, particularly for the CFC product.

Table 6 Validation scores for cloud fractional cover (CFC): bias [%], Kuipers score, and hit rate (both [–]). See text and Sect. 3.3 for details.

Table 7 Validation scores for cloud top height (CTH): bias and mean absolute error (both [m]). Total number of used orbits and samples are given in Table 6. See text and Sect. 3.3 for details.

Table 8 Validation scores for cloud phase (CPH, here the fraction of liquid clouds): bias [%], Kuipers score, and hit rate (both [–]). Total number of used orbits and samples are given in Table 6. See text and Sect. 3.3 for details.

If comparing results in columns “PPS VIIRS” and “VGAC no SBA” for CFC in Table 6, we notice that to process VIIRS data for AVHRR-heritage channels in the correct PPS VIIRS environment generally gives better results than when processing these data in the AVHRR PPS environment. It proves that there is indeed a need to apply spectral band adjustments to properly use VIIRS-based data in the AVHRR PPS environment. Some signs of this is also seen in the CPH results in Table 8, while it is more difficult to judge the changes seen for the CTH results in Table 7.

Regarding the CFC results, it is clear that all methods (except Linear-1a, which actually worsens results further) perform well, indeed much better than the CLARA-A3 requirements and very close to the achieved validation results for CLARA-A3. The SBA-NN method has the best overall scores for the VIIRS-VGAC simulations validated against all CALIPSO-detected clouds. A closer look at the results (not shown here) reveals that clearly better results are mainly explained by superior CFC performance during night conditions.

Results for CTH in Table 7 are also somewhat similar for all methods, but it is difficult to draw conclusions based exclusively on the bias parameter. As Håkansson et al. (2018) pointed out, the cloud top distribution is largely bi-modal with peaks for low-level and high-level clouds. Since determining high-level cloud tops is much more difficult than for low-level cloud tops (i.e. high-level clouds are predominantly semi-transparent), the actual distribution of low-level and high-level clouds in the validation dataset has therefore great importance for the bias results. Consequently, the error structure is generally non-Gaussian, making the bias parameter inappropriate as a measure of uncertainty. A more reliable uncertainty parameter here is the mean absolute error. We notice that all results based on simulated AVHRR data are in line with or even slightly better than the AVHRR reference results from CLARA-A3.

For the CPH product results in Table 8, the differences in the results between the various products are even smaller, so no method can be determined as clearly standing out compared to any other method.

It is encouraging to see that the method found to best simulate AVHRR radiances according to the previous section (the SBAF-NN method) also appears to perform well based on the Suomi NPP and NOAA-20 data from 2019. Only a minor degradation appears visible for NOAA-20, which has a VIIRS sensor with slightly different spectral channel responses compared to the Suomi NPP.

4.3 Inter-comparing original AVHRR-based and VIIRS-VGAC-simulated cloud physical products for 2012

Only some of the CLARA-A3 cloud products can be validated using CALIPSO–CALIOP. The cloud physical products (CPP) of CLARA-A3 consist not only of the CPH product (validated in the previous section) but also the CWP product, subdivided into cloud liquid water path (LWP) and ice water path (IWP). COT and CRE are needed for their generation. Even if we cannot easily validate these products from independent data, comparing the original AVHRR-based products to the VIIRS-VGAC-generated products in the used AVHRR–VIIRS collocation dataset is possible. Here, we show such inter-comparisons based on all collocated data for 2012. Notice that since the CPP products are only derived during the daytime, the only difference between the Linear-1a and Linear-1b methods is that the latter also has a distinction between twilight and day.

Figure 7 Frequency distributions of CPP differences between products based on the VIIRS-simulated and real AVHRR radiances. For reference, also products based on uncorrected VIIRS radiances are shown (dotted green curve: no SBA in the legend).

Table 9 Mean absolute difference (MAD) for the various spectral band adjustment methods for CPP products based on simulated data versus products based on the original AVHRR data. Minimum MADs for each CPP product are shown in bold numbers.

Figure 7 shows frequency distributions of the difference between VIIRS-simulated CPP products and original AVHRR-based CPP products for all SBA methods. The figure also shows resulting frequencies for the case where no spectral band adjustments are applied (i.e. the no SBA case). Table 9 summarizes the resulting mean absolute deviations (MADs) for the different cases.

It is clear from Fig. 7 that if no SBA corrections are made, the difference distributions of the two fundamental parameters COT and CRE (i.e. fundamental for the calculation of LWP and IWP) are not symmetrically distributed around the zero-difference value. In particular, the CRE difference distribution appears biased, with a secondary peak for negative differences (i.e. underestimated CREs compared to the AVHRR values). SBA corrections improve the results and especially the SBA-NN method, which produces a well-defined narrow peak centred at the zero-difference level. This is also reflected in Table 9, showing minimum MADs for the SBA-NN method.

5 Discussion

5.1 AVHRR radiance simulations

Our simulations show that radiance simulations based on linear regression methods can only partially remove differences between the original AVHRR and the AVHRR-heritage VIIRS channels. Although correlations between channels are generally high (especially for the infrared channels), there are obvious remaining non-linear features that linear methods cannot handle properly. In contrast, the SBA-NN method handles these non-linear features more accurately when simulating radiances. This includes channel differences and ratios during network training, which are particularly important for downstream applications and which further improve results.

One very important finding in our study is that for the 3.7 µm channel (AVHRR channel 3B), which measures both emitted thermal and reflected solar radiation during daytime, it is crucial to separate radiance simulations for daytime and night-time conditions. It is well known that radiances in this channel behave very differently during night and day. However, it is also clear that small differences in spectral responses between the original AVHRR channel 3B and the AVHRR-heritage channel M12 of VIIRS lead to clearly different daytime behaviour of the two channels. This is a consequence of the small difference in central wavelengths shown in Tables 1 and 2. The lower central wavelength value in M12 means that this channel is slightly more sensitive to reflected solar radiation, which results in a larger reflectance contribution to the observed brightness temperature in this channel. Thus, any relation deduced from purely night-time measurements (with only thermal emissions) will fail if applied to daytime measurements. From this, we also need to emphasize that, even if previously shown results for cloud screening in Table 6 indicate that good results could be achieved without any spectral band adjustments of VIIRS heritage channels (i.e. staying with the VIIRS processing of AVHRR-heritage channels without any adaptations to the AVHRR data), any retrieval that is highly dependent on AVHRR channel 3B needs to make spectral band adjustments on the corresponding VIIRS channel.

Central to this study was the definition of the AVHRR–VIIRS collocation dataset for 2012 and 2013. The fact that the NOAA-19 and Suomi NPP had very similar, but not identical, orbits caused some concern, since it led to an uneven global sample extraction. The small orbital differences led to the largest orbital track separations in the tropics, resulting in large (less favourable) angular differences. To avoid too much influence of angular differences, which could lead to differences from anisotropic reflection and atmospheric absorption effects, we set a maximum viewing angle of 15° for both sensors. This value was a compromise between the wish of getting samples for all latitudes and still limiting the effects of differences due to anisotropic reflection and atmospheric absorption. It allowed samples to be obtained from all latitudes, but the number of resulting samples also became an increasing function of latitude. Initially, we tried even stricter limits on viewing angles, but this resulted in only a few samples from very warm surfaces at low latitudes, thereby hampering the neural network training.

5.2 Resulting cloud masks, cloud top heights, and cloud phases

The CLARA-A3 CDR's cloud detection method depends heavily on radiances in AVHRR channel 3B. As mentioned in the previous section, the difficulties in handling this channel properly clearly affected the validation results based on the CALIPSO–CALIOP cloud products. Using the radiance simulation Linear-1a method resulted in serious errors in night-time cloud masking, even if daytime results were acceptable. Night-time results clearly improved with the Linear-1b method but even more so with the NN-based method.

The performance of the SBA-NN cloud mask was slightly better than the original results from NOAA-19 in the CLARA-A3 CDR. However, it should be noticed that the previous validation of the CLARA-A3 products allowed slightly larger time differences between the AVHRR and CALIPSO observations (i.e. 5 min instead of 2 min), which probably led to slightly degraded results for CLARA-A3 compared to using a maximum 2 min time difference in both datasets.

Results for the other two CALIPSO-examined cloud products did not show a large variation and the existing variation was mostly linked to the cloud mask quality.

Results for these three cloud products, based on the different AVHRR simulations and on the original AVHRR and uncorrected VIIRS radiances, generally did not differ very much in the achieved validation results (at the least, if not including, night-time results for the Linear-1a method). In fact, they all reside well within the requirements previously set for CLARA-A3. We conclude that, for these products, the AVHRR-heritage channels of VIIRS, corrected or not, contain enough information to provide results close to the original AVHRR-based products. It is also encouraging that the good results are stable enough to be repeated in 2019 for the Suomi NPP satellite. Results are also good for the VIIRS sensor on NOAA-20 in 2019, although with some degradation compared to the Suomi NPP. We expect this degradation to some extent due to differences in the spectral responses between the two VIIRS sensors. However, the differences remain small enough to avoid the need for radiance corrections, as results still meet the CLARA-A3 product requirements.

5.3 Resulting cloud physical parameters

The inter-comparison of the original AVHRR-based CPP products and products based on VIIRS-simulated radiances showed that very accurate AVHRR radiance simulations are more important here than for the previously discussed cloud products. This concerns the COT parameter (which depends largely on AVHRR channel 1) and the CRE parameter (which depends largely on AVHRR channel 3B). A simulation of visible and short-wave infrared radiances should preferably be subdivided into twilight and daytime categories. However, the most important for the simulation of channel 3B is that the simulation method must be derived exclusively from daytime data.

The success of the SBA-NN method compared to the linear methods reveals something very interesting: the neural network appears capable of applying different radiance correction factors to different objects in simulated channel 3B images. Notice that only objects able to reflect solar radiation in channel 3B should have a correction factor that differs from the one applied during night-time (when only thermally emitted radiation is measured). Furthermore, this correction factor should also vary depending on the reflection efficiency of the object, i.e. the correction factor should be dependent on the object's reflectance. Any linear correction method will fail here, since the correction factor is practically always constant, regardless of the object in the image. The objects reflecting the most in this channel are typically thick water clouds. Since the linear regression will produce a nearly constant average correction factor (only slightly moderated by an offset parameter), the correction factor in this channel is likely too small for thick water clouds and too large for darker objects with low reflectance (e.g. snow-covered surfaces). This is one explanation for underestimating simulated liquid cloud CRE values, which leads to underestimated LWP values. Only the SBA-NN method can correct for this effect.

Together, Fig. 8 and Table 10 illustrate the capability of the SBAF-NN method to generate different SBAs in AVHRR channel 3B for different objects in a VIIRS-VGAC-simulation of AVHRR data over the Greenland area from 21 July 2012. Crosses mark locations in the scene with different cloud and surface types, as interpreted by the Polar Platform system (PPS) cloud type product (Dybbroe et al. 2004). Table 10 shows the resulting SBA corrections for AVHRR channel 3B for the Linear-1b and the SBA-NN methods in all the marked points. Specifically, notice the different SBA corrections for SBA-NN for highly reflecting objects (X₂, X₄, and X₇) compared to those for weakly reflecting or non-reflecting objects (X₃, X₅, and X₆). This means that the NN is capable of identifying the different behaviour of different objects, thus making a kind of implicit object type identification when assigning which SBAs to use.

These results show the advantage of the NN approach in absorbing information, not only from the closest AVHRR-heritage channel but also from a larger set of VIIRS channels for making it possible to give different objects different effective SBA corrections. For example, notice that snow surfaces, which should hardly be SBA corrected at all according to Table 10, are known to have very low reflectance in AVHRR channels 3A and 3B. Thus, when also including channel M10 in the training process, this could help in identifying snow surfaces with higher confidence than if solely using channel M12 in this spectral region. Another example is the inclusion of VIIRS channel M14 in the training process, a channel that is not available at all on AVHRR. This channel is partly affected by water vapour absorption, and this might potentially be useful for getting a better treatment of the simulation of brightness temperature differences between AVHRR channels 4 and 5, generally affected by both water vapour absorption and differences in cloud transmissivities.

Figure 8 Excerpt of a VIIRS-VGAC-simulated AVHRR orbit from the Suomi NPP on 21 July 2012 (with first orbit scanline at 15:16 UTC). The scene shows Greenland, the Labrador Sea, and the adjacent Canadian islands (with southern direction upwards) in a colour composite (left) based on VIIRS-simulated AVHRR channels 3B, 4, and 5 and a corresponding PPS cloud type classification (right). SBAF corrections for the Linear-1b and SBA-NN methods for the marked positions are given in Table 10.

Table 10 Resulting SBA corrections (i.e. adjustment factor compared to uncorrected VIIRS channels) for simulation methods Linear-1b and SBA-NN for markup points in Fig. 8.

5.4 Prospects for remaining CLARA-A3 products

We have shown that NOAA-19 AVHRR radiances can be simulated from Suomi NPP VIIRS-VGAC radiances and that they can be used to produce cloud products with the same or even better quality than the original products included in the CLARA-A3 CDR. Whether the other CLARA-A3 products (i.e. surface albedo, surface radiation, and TOA radiation) can also be reproduced successfully from VIIRS-VGAC data remains to be investigated. Preliminary tests based on the SBA-NN radiance simulations have been performed for all products, with promising results. However, more extensive validations need to take place, which is outside this paper's scope. Nevertheless, there is no reason to believe in the failure of the remaining products if basic radiances and cloud products are produced with high quality.

6 Conclusions

AVHRR radiances from the NOAA-19 satellite have been successfully simulated from Suomi NPP VIIRS-VGAC radiances in 2012–2013, when observations from the two satellites could be collocated efficiently in both time and space. Two methods based on linear regression for each channel and one method based on an MLP neural network have been tested. The latter was trained using all AVHRR-heritage channels on VIIRS plus a few additional channels. Special attention was given to day and night differences, while constraints on channel differences and channel ratios were applied to the neural network.

The neural network approach achieved the best results for all individual channels. We found it crucial to separate daytime from night-time results when simulating AVHRR channel 3B at 3.7 µm. The small spectral response differences between AVHRR channel 3B and VIIRS M12 channel, leading to a smaller effective wavelength for the VIIRS channel, meant that different simulation methods had to be used for day and night. Furthermore, radiance corrections during the daytime had to depend on the actual object reflectance to be realistic. Only the neural network approach was able to achieve this.

Selected NOAA-19 cloud products (cloud mask, cloud top height, and cloud phase) in the CLARA-A3 CDR were produced from VIIRS-VGAC-simulated radiances based on the same retrieval methods as used when compiling CLARA-A3. The resulting products were validated using CALIPSO–CALIOP cloud products and compared to the original CLARA-A3 products. Product qualities agreed well with the original CLARA-A3 products and are clearly within product requirements. In addition, the cloud microphysical products COT, CRE, LWP, and IWP were well reproduced when based on the neural network-simulated radiances.

In order to check the validity of the derived spectral adjustments on a longer time scale, it is suggested to regularly repeat the validation efforts on derived cloud products based on new observations from active sensors on EarthCARE (Illingworth et al., 2015) and similar missions in the future. In addition, regular checks of simultaneous nadir observations (SNOs) at high latitudes between VIIRS and IASI (and its successors) can help in deducing the stability of the infrared band spectral responses. The possibility of making similar checks for VIIRS and METimage visible bands seems unfortunately not possible from the Tropospheric Ozone Monitoring Instrument (TROPOMI; Veefkind et al., 2012), due to very limited spectral coverage of AVHRR-heritage channels. The Ocean Color Instrument (OCI) on the PACE (Plankton, Aerosol, Cloud, ocean Ecosystem; Gorman et al., 2019) mission and the EMIT (Earth Mineral dust source Investigation) sensor aboard the International Space Station offer potential sources (Connelly et al., 2021) of suitable reference measurements. Sentinel-3's Ocean and Land Colour Instrument (OLCI; Donlon et al., 2012) can also provide reference data. While OLCI is not fully hyperspectral, it includes 21 bands within the 0.4–1.0 µm range and operates with a 10:00 am Equator overpass time.

CLARA-A3 will be complemented and extended with VIIRS-VGAC-based products to cover the period 1979–2024 (46 years), and this edition will be named CLARA-A3.5. Interestingly, for the future, the same radiance simulation approach could be applied on radiances from the upcoming METimage sensor on the EPS-SG satellites, with an expected first launch in August 2025. This satellite will have nearly the same Equator crossing time as the current Metop-C satellite with the last AVHRR sensor on board. This would enable collocations in the same way as the currently used collocations of NOAA-19 and Suomi NPP data. If AVHRR radiances can be successfully simulated from METimage data, the CLARA CDR can be extended by several decades based on measurements from VIIRS and METimage sensors. Finally, the possibility to also use MODIS-based AVHRR data simulations can be considered to improve observational coverage for some earlier years in the CLARA CDR (e.g. for 1999–2001 when the NOAA-16 orbital drift was considerable). However, the highest priority is to secure a CDR extension with VIIRS and METimage data.

Appendix A: Coefficients for linear regression methods

Table A1 Linear regression parameters for the Linear-1a method based on all data (i.e. merged training datasets 1 and 2 in Table 3). For details on channel wavelengths for the two sensors, see Tables 1 and 2.

Table A2 Linear regression parameters for the Linear-1b method based on all data (i.e. merged training datasets 1 and 2 in Table 3) but restricted to day conditions. For details on channel wavelengths for the two sensors, see Tables 1 and 2.

Table A3 Linear regression parameters for the Linear-1b method based on all data (i.e. merged training datasets 1 and 2 in Table 3) but restricted to twilight conditions. For details on channel wavelengths for the two sensors, see Tables 1 and 2.

Table A4 Linear regression parameters for the Linear-1b method based on all data (i.e. merged training datasets 1 and 2 in Table 3) but restricted to night conditions. For details on channel wavelengths for the two sensors, see Tables 1 and 2.

Appendix B: Detailed score statistics for radiance and image feature simulations

Table B1 Radiance validation scores (for quantity VIIRS–AVHRR) for day (4 307 482 samples). Shown are values of median error (median), mean absolute error (MAE), interquartile range (IQR), mean error (bias), root-mean-squared error (RMSE), and regression parameters (slope, offset, and correlation) for the simulation of each AVHRR channel and each channel combination and for every tested simulation method. Individual best scores are highlighted in bold numbers.

Table B2 Radiance validation scores (for quantity VIIRS–AVHRR) for twilight (852 881 samples). Shown are values of median error (median), mean absolute error (MAE), interquartile range (IQR), mean error (bias), root-mean-squared error (RMSE), and regression parameters (slope, offset, and correlation) for the simulation of each AVHRR channel and each channel combination and for every tested simulation method. Individual best scores are highlighted in bold numbers.

Table B3 Radiance validation scores (for quantity VIIRS–AVHRR) for night (5 758 113 samples). Shown are values of median error (median), mean absolute error (MAE), interquartile range (IQR), mean error (bias), root-mean-squared error (RMSE), and regression parameters (slope, offset, and correlation) for the simulation of each AVHRR channel and each channel combination and for every tested simulation method. Individual best scores are highlighted in bold numbers.

Table B4 Radiance validation scores (for quantity VIIRS–AVHRR) for all cases (10 918 476 samples but for visible channels and features 5 160 363 samples). Shown are values of median error (median), mean absolute error (MAE), interquartile range (IQR), mean error (bias), root-mean-squared error (RMSE), and regression parameters (slope, offset, and correlation) for the simulation of each AVHRR channel and each channel combination and for every tested simulation method. Individual best scores are highlighted in bold numbers.

Appendix C: Specification of derived MLP networks for day, night, and twilight

Table C1 Configuration file names for finally chosen MLP networks.