Emissivity retrievals using Rk,BG observed in neighboring pixels are a priori more robust than when this radiance is computed using a model. As discussed in G13, no biases were detected in V3 in the former case. However, when the ocean surface background radiances were computed using the model, median Δεeff12-k at εeff,12∼0 was clearly negative, down to ∼-0.015 for the 12–08 pair, which translated into significant low biases of the ice clouds microphysical indices at small emissivity (see Fig. 5 of G13 and Fig. 1c). This was due to channel-dependent biases in the computed radiances, which could be assessed independently by comparing observations and computations in clear-sky conditions. Consequently, the modeling of Earth surface radiance has been revisited in V4, as presented and evaluated in Sect. 3.3.

Large emissivities are typically found in so-called opaque clouds that fully attenuate the CALIOP signal. Importantly, “opaque” means opaque to CALIOP, that is, cloud visible optical depth typically larger than 3 in V4 (Young et al., 2018) or effective emissivity expected to be larger than about 0.8. In opaque ice clouds, V3 εeff,12 was rarely larger than 0.95 (G12), and median Δεeff12-k was minimum around εeff,12=0.95 rather than 1. Both suggested that εeff,12 was systematically too small and therefore that the cloud radiative temperature was underestimated. In other words, observations in opaque ice clouds tended to be warmer than the computed blackbody temperatures by about 5 K (see Fig. 8 in G12), while this systematic positive bias was not observed for opaque warm water clouds. A similar contrast between ice and water clouds was also reported by Hu et al. (2010) when comparing IIR observations and mid-cloud temperatures. Stubenrauch et al. (2010) reported that for high opaque ice clouds, the radiative height determined by the Atmospheric Infrared Sounder (AIRS) on board the Aqua satellite is on average lower than the altitude of the maximum CALIOP 532 nm attenuated backscatter by about 10 % to 20 % of the CALIOP apparent thickness. The warm bias between radiative temperature (Tr) and the centroid temperature Tc used in V3 was explained theoretically in G15. The Tr-Tc difference was found between 0 and +8 K for semi-transparent single-layered clouds and increased with cloud emissivity and geometric thickness, in agreement with previous studies (Stubenrauch et al., 2013, and references therein). Underestimating Tr (and therefore TOA TBB) yields underestimates in εeff,12 and the microphysical indices. Note that Heidinger et al. (2010) infer cirrus radiative height from suitable pairs of channels using a range of expected values of βeff as a constraint. The problem here is reversed and is instead to estimate Tr in order to infer microphysical indices. The determination of Tr in ice clouds implemented in V4 is presented and discussed in Sect. 3.4.

The background radiance from the surface is computed using the FASRAD model fed by horizontally and temporally interpolated temperature, water vapor, and ozone profiles and skin temperatures. These ancillary data are from the MERRA-2 reanalysis products in V4. In V3, differences between observed and computed brightness temperatures (BTDoc) in clear-sky conditions over oceans exhibited latitudinal and seasonal variations for all channels (G12), which appeared to be related to variations in the water vapor profiles near the surface to which the IIR window channels are the most sensitive. The water vapor absorption coefficients were updated in V4, to take advantage of the advances in atmospheric spectroscopy over the last decade (Rothman et al., 2013). Using MERRA-2 sea surface temperature and atmospheric profiles, the model was tuned to minimize the residual sensitivity of BTDoc to the column-integrated water vapor path (IWVP). This assessment was carried out in V4 pristine clear-sky conditions, i.e., when no layers were detected anywhere in the column or if the column included only low semi-transparent non-dust aerosols in which no single-shot cleared clouds were detected within the IIR pixel (Sect. 2). Systematic biases remained for each channel, even at night where the clear-sky mask is a priori the most accurate because of the increased CALIOP nighttime signal-to-noise ratio. Nighttime BTDoc was on average equal to -0.5 K at 08.65 µm, -0.35 K at 10.6 µm, and -0.2 K at 12.05 µm. These biases were explained by the combination of possible errors in the model, in the ancillary data, and in the calibration. We chose to reconcile observations and computations by using a new set of surface emissivity values (see Table 1) with no attempt to include surface temperature variations as reported from airborne measurements (Newman et al., 2005). The derived surface emissivity values used in V4 are close to 0.98 on average. It is noted that to save computation time, the contribution of the clear-sky downwelling radiance reflected by the surface is not included in the operational FASRAD model. Because the surface emissivity values are close to 1, the subsequent impact on their derived values is not significant.

As an illustration, median BTDoc is shown in Fig. 2a vs. IWVP derived from MERRA-2 for each IIR channel, both in V4 (solid lines) and in V3 (dashed lines). Overplotted in green is the median MERRA-2 sea surface temperature (Ts). The results are shown for 6 months of nighttime data in 2006 (from July through December) between 60∘ S and 60∘ N to ensure that the dataset is not contaminated by sea ice. The number of clear-sky IIR pixels used for this analysis is plotted in Fig. 2b. Even though the ancillary data are from GMAO GEOS 5.10 in V3 for this time period and from MERRA-2 in V4, the differences between V3 and V4 are mostly due to the changes in the radiative transfer model. The amplitude of the variations of median BTDoc with IWVP is drastically reduced in V4 compared to V3 and the inter-channel differences are significantly smaller. Between IWVP of 1 and 5 g cm-2, where most of the samples are found, V4 median BTDoc is between -0.2 and 0.2 K for the three channels. Using the V4 surface emissivities compensates for a residual 10–12 inter-channel BTDoc bias of -0.15 K and a residual bias of -0.3 K for the 08–12 pair. In contrast, the V3 median 10–12 and 08–12 inter-channel biases were up to -0.7 and -1.8 K, respectively, at IWVP of 5 g cm-2.

In order to assess the errors in the computed background radiances used in the effective emissivity retrievals (Rk,BG; see Eq. 1) and in the corresponding computed brightness temperatures (Tk,BG), we analyzed distributions of BTDoc for different latitudes and seasons. Figures 3 and 4 show probability density functions (PDFs) of BTDoc at 12.05 µm, noted BTDoc (12), and of the 10–12 and 08–12 inter-channel BTDoc differences, noted BTDoc (10–12) and BTDoc (08–12), respectively. The results are for 2 months in opposite seasons, namely January 2008 (Fig. 3) and July 2006 (Fig. 4), with computations from V4 (solid lines) and from V3 (dashed lines). The data are split into four 30∘ latitude bands between 60∘ S and 60∘ N, for both night (blue) and day (red). Statistics of the V4 differences (median, mean, standard deviation, and mean absolute deviation) are reported in Table 2 for the four latitude bands and globally (i.e., 60∘ S–60∘ N).

BTDoc (12) is overall less latitude dependent in V4 than in V3 due to the reduced bias related to IWVP in V4, and the width of the distributions is reduced. The V4 global standard deviations are similar for nighttime (0.8 K) and daytime (0.9 K) data. Mean V4 BTDoc (12) is larger for daytime than nighttime data at any latitude by 0.2 K on average. As mentioned earlier, the V4 clear-sky mask is expected to be more accurate at night than during the day. Undetected absorbing clouds would decrease the brightness temperature of the observations and therefore BTDoc (12), and a larger fraction of undetected clouds for daytime data would yield smaller daytime BTDoc (12) and not larger values as observed here. A similar finding was reported in Garnier et al. (2017) for both IIR and MODIS, suggesting that these differences are not due to calibration issues. The computations used a different model, namely the 4A-OP radiative transfer model (Scott and Chédin, 1981), and ancillary data were from the ERA-Interim reanalysis (Dee et al., 2011). It is unclear whether the small but systematic day vs. night differences are due to the V4 clear-sky mask or other reasons.

Again, the inter-channel differences are drastically reduced in V4 compared to V3, especially for the 08–12 pair of channels. In V4, the absolute values of the mean inter-channel differences are smaller than 0.1 K globally. The worst cases are in July 2006 at 30–60∘ N (Fig. 4), where mean BTDoc (10–12) and BTDoc (08–12) are equal to -0.15 and -0.26 K, respectively. The global standard deviations are around 0.31–0.35 K, notably smaller than 0.8–0.9 K found for BTDoc (12), because common biases due to errors in sea surface temperature cancel out. Keeping in mind that the random noise at warm temperature is 0.15–0.2 K (G12) in each channel, the standard deviations around 0.31–0.35 K can be largely explained by the random noise in the observed temperatures, which is estimated to be 0.2–0.3 K. Thus, the analysis of these inter-channel distributions shows that the uncertainty in computed Tk,BG can be taken identical in all channels. Based on the standard deviations in BTDoc (12), the random error ΔTBG is set to the conservative value ±1 K for all channels.

Again, the presence of clouds that were detected at single-shot resolution and later cleared from the 5 km layer product is forbidden in the V4 clear-sky mask. The impact of this refinement in V4 is illustrated in Fig. 5, which compares the BTDoc histograms in V4, in which single-shot clouds are specifically excluded, and in pseudo-clear-sky conditions (i.e., which contain at least one single-shot cloud) over oceans between 60∘ S and 60∘ N in January 2008. When cleared clouds are present (light blue and orange), median and mean BTDoc (12) are smaller by 1.3 and 2.2 K, respectively, and a marked negative tail down to about -8 K is observed, because these cleared clouds have a fairly large optical depth and are often colder than the surface. In this example, the fraction of IIR pixels that see at least one cleared cloud in the column is 35 % at night and 22 % for daytime data. The larger nighttime fraction is likely related to the fact that the probability for CALIOP to detect a cloud at single-shot resolution is larger at night due to the larger daytime background noise, so the probability that these clouds are cleared from the product is larger at night. The mean and median values of the inter-channel BTDoc are barely impacted, showing that the cleared clouds induce a similar bias in the three IIR channels.

Both in V3 and in V4, the first step into the computation of the radiative temperature is to determine the centroid altitude, Zc, of the cloud system. The centroid altitude of each layer is reported in the CALIOP 5 km layer product, together with the 532 nm integrated attenuated backscatter (hereafter IAB) of each layer. IAB is corrected for the molecular contribution and for the attenuation resulting from the overlying layers, noted T2overlying. Following the rationale presented in Appendix B, the centroid altitude of a multi-layer cloud system composed of N layers is computed as 4Zc=∑l=1l=NZc(l)⋅IAB(l)⋅T2overlying(l)∑l=1l=NIAB(l)⋅T2overlying(l). For single-layer cases (N=1), Zc is obviously the centroid altitude reported in the CALIOP data product. For multi-layer cases, the cloud system is seen as an equivalent single layer characterized by Zc given in Eq. (4), whose top and base altitudes are the top of the uppermost layer and the base of the lowermost layer, respectively. The approach is the same as that in V3, except that, because of an error in the computation of T2overlying in the V3 IIR algorithm, estimates of Zc could be too low by up to several kilometers in V3 multi-layer cases.

In V3, the radiative temperature (Tr) was set to the centroid temperature (Tc) for any cloud system. The approach is the same in V4, except when all the layers are classified as ice by the V4 ice–water phase algorithm (Avery et al., 2020). In the latter case, Tr is derived from Tc and parameterized functions, as presented and illustrated in the next section.

As demonstrated in G15, the radiative temperature Tr(k) in channel k is the brightness temperature associated with the centroid radiance of the attenuated infrared emissivity profile within the cloud. For a cloud containing a number, n, of vertical bins, i, of resolution δz, with i=1 to i=n from base to top, this centroid radiance can be written as a function of radiance Rk(i) of bin i and CALIOP particulate (i.e., cloud) extinction coefficient, αpart(i), as 5Rk=∑i=1i=n1-e-αpart(i)⋅δz/r⋅Rk(i).e-∑j=i+1j=n+1αpart(j)⋅δz/rεeff,k. The term αpart(i)⋅δz/r in Eq. (5) is the absorption optical depth in bin i. The ratio, r, of CALIOP optical depth to IIR absorption optical depth is taken equal to 2 (G15). The radiance Rk(i) is determined from the thermodynamic temperature in bin i.

On the other hand, Tc is the temperature at the centroid altitude of the attenuated 532 nm backscatter coefficient profile, which is written as a function of altitude Z(i) of bin i and αpart(i) as 6Zc=∑i=1i=nZ(i)⋅βpart(i)+βmol(i)⋅e-2∑j=ij=nηαpart(j)+αmol(j)⋅δz∑i=1i=nβpart(i)+βmol(i)⋅e-2∑j=ij=nηαpart(j)+αmol(j)⋅δz. In Eq. (6), βpart(i) is the cloud particulate backscatter in bin i, αmol(i) and βmol(i) are the molecular extinction coefficient and backscatter, respectively, and η is the ice cloud multiple scattering correction factor (Young et al., 2018, and references therein).

Using V3 CALIOP extinction and backscatter profiles in semi-transparent ice clouds, the Tr-Tc difference was found to increase with both cloud optical depth and geometric thickness (G15).

Because the CALIOP extinction profiles are not used in the IIR operational algorithm, the approach in V4 was to establish parameterized correction functions, Tr(k)-Tc, for each channel k, and to correct the initial estimate Tc that was used in V3 as Tr(k)=Tc+[Tr(k)-Tc]. These correction functions were derived offline from the statistical analysis of a series of simulated extinction and attenuated backscatter profiles. In order to reproduce the variability associated with the various possible shapes of the extinction profiles, we chose to use actual V4 CALIOP profiles (8000 profiles were used) rather than synthetic profiles. These initial CALIOP profiles were derived from single-layered semi-transparent clouds classified with high confidence as randomly oriented ice (ROI) by the V4 ice–water phase algorithm (Avery et al., 2020). Each CALIOP extinction (and backscatter) profile was scaled to simulate several pre-defined optical depths corresponding to several pre-defined effective emissivities using r=2, and the attenuated backscatter profile was simulated by applying the required attenuation to the simulated total (molecular and particulate) backscatter profile. The simulations of Tr(k) using Eq. (5) and of Tc using Eq. (6) were carried out for εeff,k ranging between 0.1 (or τa,k=0.1; see Eq. 2) and 0.99 (or τa,k=4.6). Variations of Tr(k)-Tc with η between 0.5 and 0.8 were also analyzed in order to cover the range of temperature-dependent values used in V4 (G15; Young et al., 2018). Variations with η were not discussed in G15 because η was taken constant and equal to 0.6 in V3.

The Tr(k)-Tc differences were examined against the “thermal thickness” of the clouds, that is, the difference between the temperatures at cloud base (Tbase) and at cloud top (Ttop). Overall, 90 % of the CALIOP profiles used for this analysis had Tbase-Ttop between 10 and 50 K. The median relative difference (Tr-Tc)/(Tbase-Ttop) was found to vary linearly with Tbase-Ttop, as illustrated in Fig. 6a for channel 12.05 µm using η=0.6. Figure 6b and c show that the intercepts (a0) and the slopes (a1) of the regression lines vary with cloud absorption optical depth τa,12. Furthermore, the Tr-Tc differences increase with η because the CALIOP signal is attenuated more quickly when less multiple scattering (i. e., larger η) contributes to the backscattered signal. As a result, both a0 (Fig. 6b) and a1 (Fig. 6c) increase as η is increased from 0.5 to 0.8. Finally, the mathematical expression for the correction implemented in V4 is 7Trk-Tc=a0τa,k,η,k×Tbase-Ttop+a1τa,k,η,k×Tbase-Ttop2, where the letter k refers to the IIR channel. The corrections derived from Fig. 6a are shown in Fig. 6d. For a given value of Tbase-Ttop, Tr-Tc increases with εeff,12 until εeff,12=0.7–0.8 and is maximum for εeff,12=0.8–0.99 (or τa,12 between 1.6 and 4.6), where it represents 10 % to 25 % of the cloud thermal thickness. We find that Tr(k) is slightly larger at 10.6 µm than at 12.05 µm, by less than 0.3 K in the worst case, and somewhat larger at 08.65 µm than at 12.05 µm but always by less than 1 K (not shown). Because at this stage of the algorithm, the final value of τa,k is still unknown, τa,k in Eq. (7) is the initial V3 value derived by taking Tr=Tc. No correction is applied when the initial emissivity is found to be larger than 1.

The errors in the ice cloud radiative temperature corrections were assessed by comparing Tr derived directly using the CALIOP extinction profiles with Tr derived from Eq. (7). The statistics obtained from the same 8000 CALIOP profiles as above are provided in Table 3, for both the Tr-Tc correction and the correction error, for channel 12.05 µm. These statistics are provided for εeff,12 equal to 0.2, 0.6, and 0.99, and using η equal to the extreme values 0.5 and 0.8. The median and mean correction errors are smaller than 0.25 K and significantly smaller than the median and mean corrections, which are found between 0.8 and 5 K. The standard deviations of the correction errors are between 0.66 and 1.2 K at η=0.5 and between 0.7 and 1.75 K at η=0.8, while their mean absolute deviations are smaller than 1.25 K. These quantities represent the estimated random error in the cloud radiative temperature correction resulting from the variability in the shape of the extinction profiles.

Again, the maximum corrections are for clouds having initial effective emissivities in the 0.8–0.99 range and are similar for emissivities larger than 0.9. This range of initial emissivities is found for clouds that are opaque to CALIOP. It is noted that the corrections are a priori underestimated for opaque clouds. Because the CALIOP signal does not penetrate to the true base of opaque layers, the reported base is instead an apparent one, and so Tbase–Ttop is a priori too small. Figure 7 illustrates the impact of the correction applied to V4 opaque ice clouds classified as high-confidence ROIs for IIR channel 12.05. The apparent thermal thickness (Fig. 7a) is larger at night (blue) compared to day (red), as already mentioned in Young et al. (2018). In this example, nighttime and daytime mean ± standard deviation of Tbase–Ttop are 28 ± 13 K and 21 ± 9 K, respectively. Similarly, the Tr-Tc corrections shown in Fig. 7b are larger at night. The discontinuities around Tr-Tc=0 in Fig. 7b are due to pixels with initial emissivity larger than 1 for which no correction is applied, which occurs more often at night. The smaller daytime apparent thickness is explained by the larger background noise in CALIOP daytime measurements, which increases the difficulty in accurately locating cloud boundaries. Consequently, both Tc and Tr are a priori more accurate at night. Figure 7c shows the Tr-Ttop (solid lines) and Tc-Ttop (dashed lines) differences relative to the apparent Tbase-Ttop. After correction, the nighttime mean ± standard deviation of (Tr-Ttop) / (Tbase-Ttop) is 0.48 ± 0.15. This result is fully consistent with Stubenrauch et al. (2017), who report that the radiative cloud height derived from AIRS is, on average, at mid-distance between the CALIOP cloud top and cloud apparent base in high opaque clouds at night. Because the IR absorption above ice clouds is usually weak, Tr is close to TOA TBB. For reference, the dotted lines in Fig. 7c represent (Tm-Ttop) / (Tbase-Ttop), where Tm is the measured brightness temperature (here T12,m). Tm represents the warmest possible value for TBB∼Tr if all clouds had effective emissivity equal to unity. At night, Tm is always located within the apparent cloud, at 61 % from the top as compared to 48 % for Tr. The Tm-Tr difference represents the maximum possible bias in the estimation of Tr and is equal to 1.5 K on average. For daytime data, both Tr and Tm are lower in the apparent cloud than at night, and even below (Tm>Tbase), which is at least in part due to the smaller daytime apparent thickness. Further evaluation will be carried out in the future using extinction profiles and true cloud base altitudes derived from the CloudSat radar.

With the introduction of corrections to the cloud radiative temperatures for ice clouds, V4 emissivities and microphysical indices now depend on the CALIOP ice–water phase classification. However, for optically very thin ice clouds, the corrections are typically smaller than 1 K, and, furthermore, these small corrections induce little changes in the final effective emissivity and microphysical indices (see Fig. 1). Thus, microphysical indices can be considered independent of the CALIOP ice–water phase at small emissivities, typically smaller than 0.3.

In the case of opaque liquid water clouds, the observed brightness temperature (Tm) is, on average, close to the TOA TBB inferred from Tc. This indicates that the temperature at the centroid altitude is a good proxy for Tr, which is why no change was implemented in the V4 algorithm for liquid water clouds. Nevertheless, significant differences between V4 and V3 can arise from differences in the meteorological data. This is illustrated in Fig. 8, which shows PDFs of Tm-TBB at 12.05 µm in V4 (solid lines) and in V3 (dashed lines) for opaque water clouds having identical centroid altitudes in V3 and in V4. These clouds are classified as water with high confidence by the V4 ice–water phase algorithm and they are the only layer detected in the column. The V3 - V4 differences are due mainly to the different temperature profiles (GMAO GEOS 5.10 in V3 and MERRA-2 in V4), yielding different values of Tc for an identical centroid altitude, and to a smaller extent to the changes in the water vapor profiles and in the FASRAD model. The V4 differences are -0.6±2.2 K at night and 0.12 ± 2.7 during the day. The larger fraction of negative Tm-TBB differences in V3, from -2 down to -10 K, has been traced back to cases with strong temperature inversions near the top of the opaque cloud, which seem to be better reproduced in MERRA-2.