In this study an MLP, a feed-forward artificial neural network, is used. An MLP consists of three major units; (1) the input layer, (2) the output layer and (3) the hidden layer(s). The input layer holds as many neurons as input variables and the output layer as many neurons as desired output variables. The hidden layer(s) hold an arbitrary number of additional neurons distributed over an arbitrary number of hidden layers. All connections between the neurons within the MLP are in the forward direction (input layer → hidden layer(s) → output layer). Connections backward or within a layer are forbidden . The value of a neuron is calculated by processing the output from the preceding neurons connected to that neuron and the corresponding weights through an activation function. These non-linear functions allow the ANN to solve complex problems with a limited number of neurons. A generic structure of an MLP is illustrated in Fig. . In addition to the input and hidden neurons, a constant bias neuron is commonly added to the input and hidden layers in order to give the MLP more flexibility during the training.

When the MLP is given a vector of input data it uses the connection weights and possible biases to estimate the vector of output data. Thus, it is crucial that the weights and bias neurons are assigned correct values.

The weights are tuned by training the MLP, which is done with a teacher–trainer approach, more known as supervised training. A commonly used training algorithm is the back-propagation algorithm. The most essential steps in the back-propagation algorithm are explained below, but for the curious reader the algorithm as a whole is well explained in .

Using back-propagation the network is fed with a set of training examples where the vector of input variables as well as the vector of expected output variables are known. From the training input data the MLP estimates its own output data using the current weights. From the vector of estimated output and the corresponding vector of expected output the total error E (squared difference) is calculated. The error is then propagated backwards through the MLP and used to update each weight using gradient descent in such a way that the total error is minimised. Each weight is updated using the following equation: wij*=wij-ηδEδwij=wij-Δwij, where wij and wij* are the old and new values for a weight connecting the two neurons i and j. δEδwij describes how much a change in wij affects the total error E. To adjust how aggressive the weight updates should be, a learning rate η is multiplied with δEδwij before the weight update. A larger learning rate means larger changes in the weights and thus a faster training. This can, however, lead to an oscillation of the total error around a minimum solution. With a small learning rate the total error will not oscillate around a minimum solution, but the training is slower and the risk of getting stuck in local minima is higher. By introducing a momentum term α, possible oscillations in the iterative search for the minimum error are attenuated, and this allows for a larger learning rate. The momentum makes use of the previous update of the corresponding weight in order to get a weighted sum of the current and previous error gradients. The momentum term is added to the second term on the right-hand side of Eq. () such that Δwijk=ηδEδwij+αΔwijk-1, where k represents the kth update of the weight wij, meaning that Δwijk-1 is the previous update of weight wij .

To find the minimum total error between the estimated and expected output vectors for a complex problem and tune the weights accordingly, a large training dataset is required. Training an MLP is an iterative process, where each training example is presented to the ANN multiple times until a satisfying result has been achieved. With common ANN terminology the training completes one iteration every time the weights are updated and one epoch when all training examples contained in the training dataset have been presented to the ANN. The amount of iterations required for one epoch does therefore depend on the amount of training examples the ANN is given for every update of the weights, i.e. the batch size. With stochastic gradient descent (sometimes referred to as momentum stochastic gradient descent, when the momentum term is used) the weights are updated for each training example (batch size=1), whereas for full batch gradient descent the weights are updated using all training examples at once (batch size=N, where N is the total number of training examples). Stochastic gradient descent leads to a noisy error gradient whereas the full batch gradient descent requires more computational power to converge. With mini-batch gradient descent an intermediate number of training examples is used for each weight update (1<batch size<N).

While in recent years very potent new learning methods that are based on back-propagation were developed, stochastic gradient descent is still the most used method due to its simplicity and robustness .

Brightness temperatures from all thermal channels of SEVIRI except for the ozone channel at 9.7 µm are used. The brightness temperatures are calculated according to . The ozone channel is excluded because its sensitivity peaks in the stratosphere, where no cirrus clouds are present, and because of its strong annual cycle due to the ozone variability . Channels with significant solar contribution are excluded in order to have the same conditions and similar performance during both day and night. Alongside the single brightness temperatures, CiPS works pixel by pixel and takes advantage of the information from nearby pixels by utilising the regional maximum brightness temperatures for the three window channels centred at 8.7, 10.8 and 12.0 µm. The regional maximum temperature is identified for each pixel as the maximum temperature within a 19×19 pixels (≈57×57 km2 at nadir) large box centred at the pixel under consideration. The idea with the regional maximum brightness temperature is to estimate the temperature that SEVIRI would observe for a cirrus-covered pixel if the pixel was cirrus-free. This is done by assuming that at least one of the 361 pixels within the box is not covered by a cirrus cloud . The corresponding cirrus-free temperature is useful for both the detection of cirrus clouds and the retrieval of the cirrus properties since it provides information about the up-welling radiation from the surface or lower water clouds. The box size of 19×19 pixels is chosen such that the region is small enough to reduce the risk of unrepresentative maximum temperatures over inhomogeneous surfaces (e.g. coast lines) but large enough to increase the chance of capturing a representative cirrus-free pixel.

For the classification ANNs (CCF and OPF) the regional average brightness temperatures for the two water vapour channels centred at 6.2 and 7.3 µm are used as well. The regional averaged brightness temperatures are calculated for each pixel as the boxcar average temperature within a 19×19 pixels large box centred at the pixel under consideration. A homogeneous area with cold temperatures indicates the presence of a thick cirrus cloud. The combination of a single temperature and the corresponding regional average for the water vapour channels provides information about high cloud structures useful for the detection of cirrus clouds .

With CiPS we introduce modelled data from the ECMWF ERA-Interim re-analysis dataset to the list of input variables. The surface skin temperature Tsurf is strongly related to the thermal radiation emitted by the Earth and thus related to the brightness temperatures observed by SEVIRI. This information helps to account for the radiation emitted by the surface which is partly transmitted in the satellite direction through thin cirrus. It also helps the ANNs to distinguish between cirrus clouds and cold surfaces like Greenland and Antarctica. The temporal resolution of 6 h and spatial resolution of 0.125∘ is used.

Along with the data provided by SEVIRI and ECMWF, additional auxiliary datasets are used. The latitude provides valuable information about the geographical location with respect to the global circulation convergence and divergence zones (e.g. the ITCZ, subsidence regions and the polar front) which strongly affect the presence and properties of cirrus clouds. Considering the SEVIRI viewing zenith angle, the SEVIRI pixel size and slant path length are implicitly accounted for. Two flags indicating the presence of surface water and permanent ice/snow are included to gain additional information about the observed surface type. Due to the seasonal variations in the global circulation and the presence of cirrus clouds the day of the year (DOY) is used. The DOY is converted to two variables, sin(2πDOY/365) and cos(2πDOY/365), in order to remove the hard transition from 31 December to 1 January. Two variables are used to avoid the repeating pattern of sine or cosine alone.

For this time period all quality-controlled CALIOP data within the SEVIRI field of view are identified and collocated with single SEVIRI pixels in time and space. Due to the different viewing geometries of SEVIRI and CALIOP, the same cloud seen by SEVIRI and CALIOP at the same time appears to be located at two different positions. The magnitude of this displacement depends on the viewing angle and the altitude of the cloud layer. This effect has been corrected for using the latitude, longitude and cloud top altitude from CALIOP (parallax correction) to project ice clouds to the SEVIRI grid. The cirrus properties from CALIOP are spatially collocated with SEVIRI observations from the pixel having the largest overlap with the 5 km CALIOP orbit segment. The data are temporally collocated by identifying the SEVIRI observation that has the smallest difference in acquisition time compared to CALIOP. With a temporal resolution of 15 min for SEVIRI, the maximum difference in acquisition time between SEVIRI and CALIOP is 7.5 min.

When collocating SEVIRI and CALIOP observations with the purpose of training an ANN one must consider two aspects. (1) Even though the 5 km average of CALIOP point measurements fits the spatial resolution of SEVIRI (3×3 km2 at nadir and approx. 4×5 km2 in mid-latitudes) quite well in the along-track direction, the two observations differ largely in scale in the across-track direction as the footprint of CALIOP is approx. 70 m wide at the Earth's surface. Consequently the 5 km CALIOP orbit segment is representative only for a relatively small fraction of a SEVIRI pixel. This will induce inevitable errors and lead to imperfect information used to train the ANN. This is especially relevant for partial cloud cover, where CALIOP may observe a cloud-free area in an otherwise cloud-covered SEVIRI pixel. If the error from imperfect collocations is random, this will have a limited effect on the ANN. Only if there is a recurrent systematic difference as a result of the different spatial scales this will lead to biased retrievals . (2) Although cirrus clouds leave their mark on both SEVIRI and CALIOP measurements in a similar way, SEVIRI does not share CALIOP's possibility of discerning vertically separated ice clouds, liquid water clouds and aerosols. Consequently SEVIRI should not be expected to discern the signal from liquid water clouds and aerosols when retrieving the IOT as effectively as CALIOP.

The ECMWF surface temperatures are spatially collocated with the satellite observations using nearest neighbour. For the temporal collocation, the ECMWF re-analysis data are linearly interpolated between the ECMWF 6 h time steps and the satellite acquisition time.

The full collocated dataset, covering the entire SEVIRI disc and a time period of almost 6 years, contains close to 50 million collocations. Of those collocations, 80 % are used to create the four datasets required for the training of the four ANNs contained in CiPS. For the CCF ANN, both cirrus-free collocations and collocations with transparent and opaque cirrus clouds are included in the training dataset. Collocations with no cirrus cloud present are excluded from the training datasets used to train the OPF ANN as well as the CTH and IOT/IWP retrieval ANNs, since those networks will be applied only on pixels identified as cirrus-covered by the CCF ANN. Furthermore, the IOT/IWP ANN is trained only with collocations containing transparent cirrus clouds, where the CALIOP signal was not saturated such that the true, rather than the apparent, IOT and IWP could be retrieved. Figure  shows the relative number distributions of the IOT, IWP and CTH retrieved by CALIOP. It is clear that the collocation dataset is unbalanced in several aspects. The IOT and IWP have exponential distributions with very few thicker cirrus clouds. Similarly there are comparably few low and high cirrus clouds available and the CTH distribution has two peaks, corresponding to mid-latitudes and tropics. To improve the end performance for those rare points the unbalance of the training datasets is reduced “by hand”. For the cirrus detection and IOT/IWP ANNs, four duplicates of all cirrus clouds with IOTCALIOP≥1.0 have been added to the training datasets. Similarly four duplicates of all cirrus clouds with CTHCALIOP>17 km or CTH CALIOP<5 km have been added to the CTH training dataset. For the opacity classification ANN, four duplicates of all opaque cirrus clouds have been added to the training dataset. This approach does not introduce any new information that the ANNs can learn from but does increase the weight of the added points during the training. Adding too few duplicates has a negligible effect whereas too many duplicates give the added points too strong an impact during the training. By testing different numbers, four duplicates are seen to yield the best results for all ANNs. Furthermore, the IOT and IWP are transformed to their logarithmic counterparts before the training (IOT*=log10(IOT),IWP*=log10(IWP/1gm-2)). Finally, the single input variables are normalised to have zero mean and unit variance and the output data are scaled to fit the ranges of the activation functions (Sect. ) used by the ANNs.

The remaining 20 % of the collocation dataset is used for validation. Half of these data are used to create internal validation datasets that are used to monitor the error against independent data during the training in order to avoid overfitting (see Sect. ) and to determine training meta-parameters, ANN structures (see Sect. ) and classification thresholds (see Sect. ). The internal validation datasets have been filtered in the same manner as the training datasets but have not been balanced by adding duplicates of selected points. The second half of the validation data are used for the final validation of CiPS (and COCS) presented in Sect. . These final validation data are not used for any purpose during the development and training of CiPS. With common ANN terminology the internal and final validation data are usually referred to as validation and test data respectively.

The CCF of CiPS and COCS and the OPF of CiPS are evaluated as a function of the geographic position. This aspect is interesting due to the very different meteorological conditions present on the SEVIRI disc. Figure a and b show the gridded FAR (Eq. ) for the CCF of CiPS and COCS, respectively, over 5∘×5∘ boxes, using the V3 CALIOP products as reference.

As mentioned in Sect.  the average FAR for the CiPS cirrus detection is 3.9 %. The FAR is sensitive to the frequency of the events, meaning that over regions where the natural probability of cirrus presence is high, a single false alarm will have a larger impact on the total FAR than over regions where the natural probability of cirrus presence is low. Although the FAR of CiPS is relatively homogeneous across the SEVIRI disc, this effect can be observed with higher FARs along the ITCZ and lower FARs over the Sahara, for example.

COCS has an equally low FAR over arid regions but has a clearly higher FAR in general. In particular over icy surfaces like Greenland and Antarctica, COCS overestimates the cirrus presence, with FARs up to approx. 90 %. But for high latitudes in general, the FAR of COCS remains higher than CiPS. In the polar regions (latitude≥65∘ N/S) the average FAR is 33 % for COCS and 5.3 % for CiPS. Also over Europe the FAR of CiPS is clearly lower. Furthermore, COCS strongly overestimates the cirrus presence around the sub-satellite point of SEVIRI. For viewing zenith angles smaller than 15∘, COCS has an average FAR of 23 %. This deficiency is not shown by CiPS, which has an average FAR of 8.5 % for the same area. Furthermore, a false alarm of COCS has IOTCOCS≥0.1, whereas a false alarm of CiPS can have an IOTCiPS down to 0.0, i.e. IOTCiPS>0.0.

Due to the high probability of cirrus cloud presence along the ITCZ, the effect of the higher FAR of CiPS over this region is small, since a high cirrus probability prevents false alarms from occurring. Figure c and d show the total number of false alarms/positives NFP by CiPS and COCS, respectively, i.e. the total number of cirrus-free points in the validation dataset (approx. 3.3 millions) that are falsely classified as cirrus. Again the numbers are calculated over 5∘×5∘ boxes. Even if the probability of having a false alarm by CiPS is higher than the average FAR along the ITCZ (Fig. a), the absolute number of false alarms is just as high as for most regions across the SEVIRI disc (Fig. c). Looking at NFP by COCS (Fig. d), more false alarms are observed at high latitudes (especially over icy surfaces), over Europe and around the sub-satellite point.

The FAR can easily be optimised by reducing the number of detected cirrus clouds (see Fig. ). Thus it is necessary to simultaneously look at the performance in cirrus detection alongside the false alarm analysis. A reduced POD would be a natural effect if the FAR is reduced, but despite the low FAR of CiPS the POD remains high. Figure  shows the POD of CiPS, again in comparison to COCS. The POD is a function of IOTCALIOP and within each IOTCALIOP interval the POD given by Eq. () is calculated, using the V3 CALIOP products as reference. For a better visualisation the POD is presented with a logarithmic scale for IOTCALIOP<1.0 and with a linear scale for IOTCALIOP≥1.0. For cirrus clouds with IOTCALIOP>1.0, CiPS and COCS perform similarly. A strong difference is instead seen for the thin cirrus clouds, where CiPS detects more cirrus clouds compared to COCS. For example at IOTCALIOP=0.1, CiPS detects 71 % of the cirrus clouds and COCS 43 %. A higher POD for thin cirrus clouds is an important improvement when studying contrail cirrus or the cirrus life cycle for example. Figure  only presents the results for the transparent cirrus clouds where the CALIOP laser was not saturated. For the opaque cirrus clouds the average POD is 98 % for both CiPS and COCS. The geographical dependency of POD is clearly anti-correlated with the geographical dependency of the FAR, meaning that CiPS has its highest and lowest POD over regions where the natural probability of cirrus presence is high and low respectively. Apart from that, the POD of CiPS is homogeneous across the SEVIRI disc.

Figure  shows the FAR of the CiPS OPF, again over 5∘×5∘ boxes, using the V3 CALIOP products as reference. Since the OPF is a new variable introduced with CiPS, the results cannot be compared to COCS. As mentioned in Sect.  the average POD and FAR is 71 and 4.0 % respectively. Both quantities are relatively homogeneous across the SEVIRI disc, but the risk of falsely classifying a transparent cirrus cloud as opaque is slightly lower in the tropical regions (latitude<30∘ N/S).

Figure  shows two density scatter plots, with CTHCALIOP on the horizontal axes and CTHCiPS (Fig. 9a) and CTHCOCS (Fig. 9b) on the vertical axes. The colour shows the normalised relative frequency, which is the relative frequency normalised to the interval 0–1. Along with the scatter plots the MPE and MAPE (Eqs.  and ) of CiPS and COCS with respect to CALIOP as a function of CTHCALIOP are shown (Fig. 9c). CiPS and COCS are validated using their own respective cirrus flags, meaning that CTHCiPS is validated using the cirrus-covered points that CiPS detects, whereas CTHCOCS is validated using those cirrus-covered points that COCS detects. Using a common cirrus flag (i.e. those cirrus-covered points that both CiPS and COCS detect) shows marginal differences, with slightly reduced errors for CiPS, as a result of the reduced amount of very thin cirrus that only CiPS detect, for which the CTH is more difficult to accurately estimate.

With CiPS the CTH is retrieved with a higher accuracy compared to COCS, especially for high and low cirrus clouds. The correlation between CALIOP and CiPS is 0.90. For CALIOP and COCS, the correlation coefficient is 0.82.

The MPE shows that CiPS overestimates and underestimates the CTH of the lowest and highest cirrus clouds, respectively, even if the errors are smaller than for COCS. From 8 to 15 km the MPE is close to zero, meaning that the CTH retrieval by CiPS is unbiased in this CTHCALIOP range. The MAPE shows that the average magnitude of the CiPS error is 10 % or less for cirrus clouds having a CTH above 8 km. Furthermore, the MAPE clearly shows the better accuracy of CiPS. For example, for cirrus clouds with a CTHCALIOP between 4 and 5 km, representing mid-level clouds with icy tops, the MAPE is 38 % for CiPS. For COCS the corresponding number is 107 % with solely overestimated values (MAPE=MPE). This is mainly an effect of the CTH filtering used for COCS (Sect. ), which excluded cirrus clouds with a CTHCALIOP<4.5 km from the training dataset, leading to strong overestimations of lower values. Furthermore, this type of low cirrus/icy clouds are found in the polar regions (see Fig. b), where the retrieval conditions for SEVIRI are more challenging with larger viewing zenith angles and pixel sizes.

The CTH has a strong latitude dependency and the CiPS results shown in Fig.  are not representative for all latitudes. Figure a shows the MPE of the CTHCiPS retrievals with respect to CALIOP as a function of CTHCALIOP and the latitude. Figure b shows the corresponding occurrences of the points that make up the statistics shown in Fig. a. Please remember that the validation dataset is a random subset of CALIOP data collected over a time period of almost 6 years and hence represents the natural latitudinal distribution of cloud top heights.

The MPE shows a clear latitude dependency and in contrast to Fig. c, where CiPS is shown to have no bias (MPE ≈ 0) between 8 and 15 km, we see that the CTHCALIOP limit when CiPS starts to over- and underestimate the CTH increases towards the Equator. At higher latitudes (e.g. over Europe), we see that CiPS is more likely to underestimate the CTH also for lower CTHCALIOP around 11–14 km, with an increasing bias towards higher latitudes. Similarly the CTHCiPS for cirrus clouds with CTHCALIOP<13 km is more likely to be overestimated along the ITCZ, with increasing errors towards the Equator. From Fig. b it is clear that the situations with higher errors and stronger biases (|MPE|≳20 %) are comparably rare and that CTHCiPS is unbiased for the more frequent combinations of CTHCALIOP and latitude.

Note the difference between the CiPS CTH retrieval and standard ones e.g., where the determination of cloud top height requires the knowledge of the appropriate vertical temperature profile from NWP (numerical weather prediction) models, while CiPS only requires the surface skin temperature from a NWP along with the SEVIRI brightness temperatures and auxiliary data.

Figure  shows again two density scatter plots, now with IOTCALIOP on the horizontal axes and IOTCiPS (Fig. 11a) and IOTCOCS (Fig. 11b) on the vertical axes. As before the colour shows the normalised relative frequency. Again the MPE and MAPE (Eqs.  and ) of CiPS and COCS with respect to CALIOP as a function of IOTCALIOP is shown in the right panel. Only transparent cirrus clouds, where CALIOP was not saturated, are included here. The two algorithms are validated using their respective cirrus cloud flags (as explained above for the CTH). This is not 100 % true for the IOTCOCS scatter plot, however, where all points with a retrieved IOTCOCS>0.0 are included. Instead the black grid on top of the scatter plot illustrates the area where COCS does not detect any cirrus clouds as a result of the COCS cirrus detection threshold at IOTCOCS=0.1 (Sect. ). A relatively large scatter is observed for both algorithms. CiPS shows a better correlation with the CALIOP retrievals though. The correlation between CiPS and CALIOP is 0.65, whereas the correlation between COCS and CALIOP is 0.61. Furthermore, CiPS shows higher frequencies along the 1–1 line down to IOTCALIOP≈0.09, but also below this value the correlation between CALIOP and CiPS is evident. Only below IOTCALIOP=0.04 does the correlation get lost.

For a better visualisation of the lower IOT range, where most points are located, the density scatter plots have logarithmic axes. This does, however, visually reduce the errors, so for a quantitative evaluation attention should be paid to Fig. c showing the MPE and MAPE of CiPS and COCS with respect to CALIOP. The MPE and MAPE are functions of IOTCALIOP and again the results are presented using a logarithmic scale for IOTCALIOP<1.0 and a linear scale for IOTCALIOP≥1.0. From the MAPE the low accuracy of CiPS for sub-visual cirrus clouds becomes evident. For IOTCALIOP<0.03, we also see that MAPE=MPE, meaning that CiPS entirely overestimates the IOT in this region. For COCS, the same is observed for IOTCALIOP<0.1 as a direct effect of the inability of COCS to detect cirrus clouds with an IOTCOCS<0.1. The opposite is observed for thicker cirrus clouds (IOTCALIOP≳2.0), where both CiPS and COCS entirely underestimate the IOT (MAPE=-MPE). With CiPS the IOT can be retrieved with a MAPE of 50 % or less for cirrus clouds with 0.35≲IOTCALIOP≲1.8. Similarly the MAPE of the retrieved IOTCiPS is 100 % or less for cirrus clouds with IOTCALIOP>0.07 and 230 % or less down to sub-visual cirrus clouds (IOTCALIOP>0.03). The corresponding MAPEs for the IOT retrieved by COCS within the same IOTCALIOP intervals are 59, 290 and 720 %. A MAPE of 100 % might seem high, but one should keep in mind that this translates into small absolute errors for such thin cirrus clouds. For the lower IOTCALIOP range, a similar scatter is observed between IOTCALIOP and modelled IOT from infrared radiances for thin cirrus clouds in .

Figure  shows the density scatter plot with IWPCALIOP on the horizontal axis and IWPCiPS on the vertical axis (Fig. 12a) together with the MPE and MAPE (Eqs.  and ) of CiPS with respect to CALIOP as a function of IWPCALIOP (Fig. 12b). Please note that again the density scatter plots have logarithmic axes and the errors are presented using logarithmic scale for the thinner cirrus clouds (IWPCALIOP<10.0 gm-2) and with linear scale for the thicker ones (IWPCALIOP≥10.0 gm-2). Since the IWP is not retrieved by COCS, no additional results are shown here for comparison. Again only transparent cirrus clouds are included.

The scatter between IWPCiPS and IWPCALIOP is very similar to that between IOTCiPS and IOTCALIOP. This is not surprising since the IWC from CALIOP is retrieved from the measured extinction coefficients using a parametrisation. The correlation between CiPS and CALIOP is, however, slightly lower for the IWP retrieval (0.59) compared to the IOT retrieval. This is also expected since possible deficiencies in the CALIOP IWC parameterisation will make it more difficult for the ANN to learn the relationship between the input data and the IWP. Nevertheless, these results show that the ANN is capable of reproducing this relationship in a good way. With CiPS the IWP can be retrieved with a MAPE of 100 % or less for cirrus clouds with IWPCALIOP>1.7gm-2 and 200 % or less down to IWPCALIOP≈0.7gm-2. Please note that deviations of 100 % are common even when microwave information is considered e.g.even if their error measure is different from ours.

In contrast to the CTHCiPS retrieval, CiPS shows a stable performance for the IOT and IWP retrievals across all latitudes (not shown here). The only anomaly observed is that the CiPS retrieval errors for thin to sub-visual cirrus are lower over convergence zones like the ITCZ, where they are mostly found .

As expected and as seen in Figs. , and , CiPS is not able to perfectly model the CALIOP cirrus properties using the SEVIRI, ECMWF and auxiliary data. There are several sources of error that add to the final performance of CiPS. Most importantly CALIOP and SEVIRI have different sensitivities to cirrus clouds. This is especially clear for thin to sub-visual cirrus clouds where CALIOP is able to accurately retrieve the top height and optical properties. Such faint cirrus leave a considerably weaker or no mark on the SEVIRI observations though, making it difficult to inversely determine the cirrus properties. Similarly the CTH is not necessarily defined equally by CALIOP and SEVIRI, as CALIOP is able to discern thinner icy layers at the cloud top that may appear as “invisible” to SEVIRI. Also for thicker cirrus clouds where both CALIOP and SEVIRI (thermal observations) approaches the point of saturation, the different sensitivities lead to ambiguous collocations. When an ANN is trained with a set of different output values that correspond to approximately the same input data as a result of the lower sensitivity, the ANN will not be able to model an accurate relationship. The reason for this is that the input vector contains no information on how the difference in sensitivity affects the target values. This can be regarded as an unknown hidden variable. This weakness is not specific to ANNs but applies to all regression models minimising the squared error. When such a set of incomplete input data (in the sense that there is a strong hidden variable) is given to the final ANN, it will output a conservative mean value that can be understood as an average over the distribution of the most likely solutions weighted by their probability. The larger the difference in sensitivity is, the higher the variance within the distribution of the most likely solutions will be, leading to larger retrieval errors. Throughout most of the output data range this error will be random. But, obviously, the distribution of the most likely solutions cannot be centred around the extreme values leading to systematic over- and underestimations of low and high output values when a conservative mean value is calculated. This effect increases towards the extreme values as the desired output value is skewed towards the edge of the distribution of the most likely solutions. This effect is clearly seen in Figs. c and c where low and high IOTCALIOP/IWPCALIOP are over- and underestimated respectively. This is to some extent also seen for the CTHCiPS retrieval in Fig. c, especially for low CTHCALIOP. Due to the randomness of the effects a lower sensitivity introduces, adding information about the magnitude of the sensitivity to the input vector is not likely to improve this situation. The larger CTHCiPS retrieval errors observed for low clouds can also be attributed to the smaller temperature contrast with respect to the surface temperature and thus the weaker radiative signal that those clouds have compared to higher cirrus clouds. Another source of error that amplifies the effect discussed above is the risk that there are additional variables relevant for finding an accurate relationship that are not represented by the vector of input data.

As discussed in Sect. , imperfect collocations as a result of the different spatial scales of CALIOP and SEVIRI together with partial cloud cover or spatially inhomogeneous clouds will further add to the retrieval errors. In a situation where CALIOP observed a small optically thin area of an otherwise optically thick cirrus inside a SEVIRI pixel, CiPS is likely to overestimate IOTCALIOP and IWPCALIOP. Similarly if CALIOP observed a small optically thick area of an otherwise optically thin cirrus inside a SEVIRI pixel, CiPS is likely to underestimate IOTCALIOP and IWPCALIOP.