Airborne ice particle ingestion into the engines of high-altitude jets is a serious aviation hazard in certain conditions. A number of power loss events caused by ice ingestion have occurred since 1990 (Lawson et al., 1998), which prompted the Federal Aviation Administration to issue Airworthiness Directive 2013-NM-209-AD (Federal Aviation Administration, 2020); 2013-NM-209-AD requires the flight manuals of Boeing models 747-8, 747-8f, and 787-8 to advise the attending flight crew of potential ice particle icing conditions at high altitudes and prohibits flight operation in those conditions. Such regulations create the need for instruments that alert flight crews whenever aircraft enter dangerous environments, which can include sub-visible and radar-undetected ice clouds. The Optical Ice Detector (OID) is an onboard, short-range cloud lidar that is under development by Collins Aerospace for the detection and characterization of hazardous cloud environments (Ray et al., 2009; Halama et al., 2010; Ray and Anderson, 2015). As with any cloud lidar, the OID measures backscatter, which depends mainly on the number and size of liquid droplets and ice crystals.

The objective is to evaluate the 2015 OID instrument using backscatter coefficient measurements and backscatter coefficients derived from particle size distributions obtained from state-of-the-art cloud probe observations. The evaluation uses backscatter coefficient uncertainty obtained by summing each size channel's uncertainty, which is derived using both particle concentration and channel width. Quantitative comparisons of OID backscatter to research-grade cloud probes is a major step in the deployment process of a commercial, onboard lidar system to alert flight crews of dangerous ice concentration conditions. Additionally, such comparisons enable the evaluation of backscatter theory, the processing of measurements, and the interpretation of observations.

Since the early 1990s, there have been over 240 ice ingestion-related incidents involving commuter and large transport aircraft (Mason et al., 2006). Many of these aircraft incidents occurred at altitudes greater than 3000 m a.m.s.l. (above mean sea level) (Bravin and Strapp, 2019) and near convective clouds (Haggerty et al., 2018). Before the early 2000s, ice particles in convective clouds were believed not to adhere to cold engine components and hence were not considered a safety hazard. However, power loss incident frequency continued to increase as the number of high-altitude flights increased. Analysis of 46 power loss events above 3000 m a.m.s.l. (Mason et al., 2006) indicated that aircraft would gradually lose power, with some aircraft experiencing total engine shut down. Once the aircraft descended to below 3000 m a.m.s.l., most failed engines were restarted, and normal engine performance was restored. However, in some cases, engine damage was sustained.

In 2002, a transport aircraft with dual Rosemount Icing Detectors experienced engine power loss without the presence of supercooled liquid water (Mason et al., 2006). Post-event analysis concluded that ice ingestion-related shutdowns can take place in environments consisting entirely of ice crystals, not simply in environments with supercooled liquid water. This aircraft hazard from ice particles causing engine power loss was termed “ice particle icing” (Mason et al., 2006) or “ice crystal icing” (Haggerty et al., 2018) to differentiate it from “icing” of surfaces from the impaction and accretion of supercooled liquid droplets. In 2013, an event occurred above 10 km that caused permanent damage to engine compressors and prompted an Airworthiness Directive issuance (Federal Aviation Administration, 2020) on 27 November 2013.

Cirrus clouds can contain ice crystal concentrations high enough to impact the performance of aircraft engines (Gayet et al., 2012; Heymsfield, 1986) and pitot tubes (Jarvinen, 2013). Analysis of 22 mid-latitude cirrus clouds showed an average ice particle concentration of 1 cm-3, with some concentrations above 5 cm-3 (Lawson et al., 2006a). However, most cirrus clouds have concentrations from 0.01 to 1 cm-3 (Krämer et al., 2009). Typically, ice crystal concentration increases with higher updraft speed and the subsequent increase in supersaturation (Heymsfield and Miloshevich, 1993). Frey et al. (2011) found that outflow from developing mesoscale convective systems in Africa contained ice particle concentrations as high as 8.3 cm-3, with sub-visible cirrus having an average ice particle concentration of 0.01 cm-3. See Heymsfield et al. (2017) for additional information on cirrus cloud properties and processes.

Due to their potentially high concentrations and, often, the lack of visibility to pilots and onboard radar, cirrus clouds pose an aircraft flight safety hazard due to potential engine power loss. Mitigating this hazard requires alerting flight crews of dangerous ice particle concentrations using onboard instrumentation. Research-grade aircraft instruments for measuring ice particle concentration have been available since the 1970s (Baumgardner et al., 2017). Optical array probes are a class of in situ instruments that measure individual particle size using the blockage of laser light (shadows) on an array of diodes (Fig. 1). The fundamental measurement is the two-dimensional shadow area of individual particles, which are generally used to obtain size distributions by deriving the particle diameter. However, the fundamental measurement of a cloud imaging probe is particle shadow area, where light is decreased by 50 % or more.

To avoid placing probes directly into the airstream, instruments have been developed that use flush-mounted windows which allow observations without altering the airflow around an aircraft. One such system is the droplet measurement technologies (DMT) backscatter cloud probe (BCP), which measures the backwards scattering of a continuous wave laser beam from cloud water droplets and ice crystals outside the aircraft (Beswick et al., 2014). The BCP has a sample volume of approximately 125 cm3 s-1 at an aircraft speed of 100 m s-1 and is located approximately 4 cm from the aircraft skin. The BCP has been deployed on commercial aircraft to measure the particle size distribution (Beswick et al., 2015). BCP measurements have been evaluated (Beswick et al., 2015) using comparisons with a cloud droplet probe (CDP) and a cloud and aerosol spectrometer (CAS) probe.

The OID is similar to the BCP in using an observation window; however, the OID uses a lidar with a conical sample volume that extends up to 10 m from the aircraft (Ray and Anderson, 2015). The OID sample volume is 4500 cm3 s-1 at 100 m s-1, with a viewing direction perpendicular to the forward motion of the aircraft (Fig. 2), and uses a circularly polarized wavelength of 905 nm and a randomly orientated, linearly polarized wavelength of 1550 nm. With the 905 nm wavelength, the OID is able to measure both the first Stokes parameter as well as the fourth Stokes parameter (Liou and Yang, 2016; van de Hulst, 1981). The fourth Stokes parameter is the focus of this study, and the 1550 nm wavelength channel is not used in the analysis. The OID's large sample volume, ability to observe in the unperturbed air ahead of the aircraft, and flush mounting are important for commercial applications in detecting hazardous aircraft environments. Additionally, the OID's larger sample volume compared to other aircraft instruments is advantageous for comparison with other remote sensing platforms (e.g., CALIPSO). Hence, a research-grade, OID-type instrument would be an important tool for linking small-scale microphysical cloud observations to large-scale remote sensing observations.

The OID has been deployed on the North Dakota Citation Research Aircraft (Delene et al., 2019) during several field projects, including a 2015 field project (CAPE2015) to study Florida thunderstorms (Schmidt et al., 2019). The Citation Research Aircraft has conducted several research projects to collect cloud microphysical observations using various instrumentation configurations (Skofronick-Jackson et al., 2014; Jensen et al., 2015; Delene, 2016). Multiple field projects have included OID measurements; however, this study analyzes data (Wagner and Delene, 2020a) from only CAPE2015 flights, which focused on measurements of cirrus cloud anvils from convective storms. An OID is mounted in the Citation Research Aircraft's fuselage to measure backscattered light from cloud particles slightly ahead of and along the span of the aircraft wing (Fig. 2). The OID's viewing port is fitted with anti-reflection coated class. A fan moves cabin air across the port window to prevent water and ice condensation on the inside surface. Interested readers are referred to Ray and Anderson (2015) for a thorough description of the OID instrument and how it obtains measurements, as only a brief summary is provided herein.

The OID measurement of the backscatter coefficient (β in units of km-1 sr-1) assumes a homogeneous cloud particle distribution over the sampling distance R. The backscatter coefficient is calculated by inverting the returned lidar power PR equation: 1PR=βGRe-2εR, where GR is a light collection efficiency as a function of particle range, ε is the extinction coefficient, and R is the range to the particles (Ray and Anderson, 2015). ε is derived by curve fitting the OID-measured attenuation and unattenuated backscatter (where the alpha parameter is the attenuation, and the y-intercept is the true unattenuated backscatter), allowing for ε to vary with the changing cloud conditions. As the sampling volume of the OID is relatively small, attenuation by ice crystals is low. Thus, a correction with the two-way transmissivity is unnecessary. While a complete OID error analysis is not yet available, the primary error source is likely the inversion of the range-resolved lidar signal for estimating extinction. For additional details regarding lidar retrievals, readers are encouraged to see Lolli et al. (2013). While the backscatter coefficient includes scattering from both molecules and cloud particles, the OID is not sensitive enough to measure molecular-only scattering (Ray and Anderson, 2015); therefore, it is assumed that all backscatter is from cloud particles.

The OID emits light pulses at a repetition rate of 20 kHz, with each pulse having a temporal width of 4 ns full width at half maximum (Halama et al., 2010). The 20 kHz measurements are aggregated to produce 5 Hz raw data. Raw data are averaged to match the 1 Hz external cloud probe (ECP) processed data. The 1 Hz frequency data has more uncertainty than 5- or 10 s-averaged data; however, 1 Hz data is able to detect rapid changes in cloud conditions that are important when monitoring environmental conditions from a fast-moving aircraft and for understanding cloud processes. While the OID transmits both 905 and 1550 nm laser light, absorption at 905 nm by water is approximately 1000 times less than at 1550 nm. Hence, using only the 905 nm wavelength simplifies analysis by eliminating the consideration of absorption.

The North Dakota Citation Research Aircraft is a Cessna model 550 fan-jet aircraft that has instruments to measure atmospheric state parameters such as temperature, relative humidity, and wind velocities as well as cloud particle size distribution using a set of in situ cloud probes. In situ cloud microphysical instrumentation (Fig. 2) includes a Stratton Park Engineering Company Incorporated (SPEC) two-dimensional stereo (2D-S) probe (Lawson et al., 2006b), a SPEC High-Volume Precipitation Spectrometer Version Three (HVPS3) probe (Kumjian et al., 2016), and a Sky Tech Research Incorporated Nevzorov probe (Korolev et al., 2013b). Optical array probes, such as the 2D-S and HVPS3, use a laser beam between probe arms that is directed onto an array of photodiodes to observe shadowing by passing cloud particles during flight (Fig. 1). The 2D-S has 128 diodes with a 10 µm resolution, and the HVPS3 has 128 diodes with a 150 µm resolution. Ice crystals and water droplets passing through the laser beam between the arms block light, causing reduced illumination on some of the individual elements of the photodiode array. Images are produced when at least one array element is “shadowed” (i.e., reduced in intensity by 50 % or more). This imaging threshold and its accuracy are further discussed in McFarquhar et al. (2017). The sample duration of the photodetector array is adjusted using the measured true airspeed so that images have correctly scaled pixels. When cloud particle concentrations are high, the 2D-S probe may not have sufficient time to off-load the array buffers as quickly as they are filled, which results in reduced probe activity (Lawson et al., 2006b). CAPE2015 has no “dead-time” issues, as the particle concentration is low enough that the probe's activity is 100 % during all flights analyzed.

The Nevzorov probe is a hot-wire instrument with sensors to measure cloud liquid and total (liquid + ice) water content (Korolev et al., 1998). The total water content (TWC) sensor uses a conical receiver to collect both liquid water droplets and ice crystals, while the liquid water content (LWC) sensor uses a round wire to collect droplets but very few ice particles. Both sensors have corresponding reference sensors that are exposed to the airflow but not to cloud particles. The additional power required to maintain constant temperature for particle sensors compared to reference sensors is directly related to water mass. Mass measurements and aircraft true airspeed are used to determine LWC and TWC. Ice water content (IWC) is not directly measured but rather calculated by subtracting LWC from TWC. Nevzorov probe data processing uses multiple linear regressions of the measured static pressure and indicated airspeed to obtain a per-flight calibration (Schwarzenboeck et al., 2009), and an automatic baseline correction ensures that the out-of-cloud water content is zero (Delene et al., 2019). The uncertainty of Nevzorov measurements increases with hydrometeor size, because large ice crystals may bounce out of the conical receiver before complete evaporation occurs, while large water droplets may not evaporate completely before bouncing from the conical receiver or shedding from the round wire. While it is possible to use the OID measurement to derive IWC (Anderson and Ray, 2019), the 2015 dataset does not have a direct measurement of IWC. Hence, an IWC comparison would have more uncertainty than a backscatter coefficient comparison.

In recent years, efforts have been made to collectively document the uncertainties associated with optical cloud microphysical probes (Baumgardner et al., 2017). These uncertainties include the possibility of particles being counted in the wrong size channel (Korolev et al., 1991; Baumgardner and Korolev, 1997), sizing uncertainty for ice particles outside of the focal volume (Connolly et al., 2007), approximation of irregular ice particles as spheres (Wu and McFarquhar, 2016), and particle coincidence (Cooper, 1988; Lance, 2012; Johnson et al., 2014). These uncertainties are accounted for as much as possible in the data processing and analysis methodology used here. Additionally, splashing of water droplets and the shattering of ice crystals can cause measurement errors; hence, anti-shattering probe tips and data processing methods have been employed to minimize such issues.

The 2D-S and HVPS3 images are processed to obtain particle concentrations using the sample volume (SV) given by 2SV=DOF⋅w⋅TAS⋅t, where DOF is the depth of field, w is the effective width of the photodiode array, TAS is the aircraft's true airspeed measured by a pitot tube, and t is elapsed time (McFarquhar et al., 2017). The depth of field (Fig. 1) is the region along the laser beam where particles are sufficiently within focus to create a clear shadow on a photodiode array (Korolev, 2007). DOF⋅w is the sample area, while TAS⋅t provides the third dimension in determining SV. The nose gust boom-measured TAS and left-wing pitot tube (Fig. 2) measured TAS agree within 3 % for the CAPE2015 field project.

Quality control conducted during the CAPE2015 field project involved instrument maintenance and performance checks. Instrument maintenance included cleaning the outside windows on all optical probes before each flight. Performance checks included reviewing the voltages of the first and last (edge) photodiodes in the cloud probe arrays and ground testing instruments with spray water to ensure correct performance. To enable timely post-flight data review, the open source Airborne Data Processing and Analysis (ADPAA) software package (Delene, 2011) is used to automatically process and visualize data after each flight. Even with rigorous instrument quality control and assurance, dataset problems can still occur. For example, the liquid cloud droplet size spectrum measured by a DMT CDP is inconsistent (i.e., a factor of 10 low) compared to the 2D-S size spectrum in the overlapping region. Misalignment of the CDP laser discovered after the CAPE2015 project makes the validity of the CDP measurements questionable; therefore, only the 2D-S and HVPS3 instruments are used to create a combined size spectrum.

All CAPE2015 two-dimensional optical array probe data are automatically processed using the ADPAA code (Delene et al., 2020a) that interfaces with the System for Optical Array Probe Data Analysis Version 2 (SODA2) software package (Bansemer, 2020). Even when using Korolev anti-shatter tips, bursts of particles over a short sampling period are likely to be shattering artifacts (Field et al., 2006); therefore, SODA2's shattering artifact rejection methodology is used for all CAPE2015 data processing. SODA2 processing of the CAPE2015 2D-S and HVPS3 images uses the center-of-mass-in method (Heymsfield and Parrish, 1978) to obtain particle concentrations. An ADPAA script merges 2D-S horizontal-arms size channels (5 to 1000 µm) and HVPS3 (1000 to 30 000 µm) size channels obtained with SODA2 processing to create a combined particle size distribution ranging from 5 to 30 000 µm, which is used for the analysis (Table 1). The 2D-S vertically oriented arm measurements are not used in the analysis; however, the two orientations have similar particle size distributions (see Supplement). The CAPE2015 data have been previously processed and analyzed (Wagner, 2019) using the full reconstruction method (Heymsfield and Parrish, 1978); however, the center-of-mass-in method is used here, since HVPS3 measurements are available to cover the larger particle size range, and the observed cirrus clouds do not have particles large enough to be excluded by center-of-mass-in HVPS3 processing. Particle diameters are determined using the area-equivalent method, where the diameter is that of a circle having an area equal to the pixel area contained within an imaged particle. Analysis using another method for determining particle diameters, known as the fast-circle method, is shown in the supplemental material. In the fast-circle method, the diameter is of the smallest circle that encompasses the imaged particle entirely. The area-equivalent method is used herein, as the method works best for spheroids; however, there is no generally agreed upon method for calculating the effective diameter of clouds containing ice particles for many shapes. The area-equivalent and fast-circle are the two most commonly accepted methods, and the interested reader can compare the analysis herein to the analysis presented in the Supplement to see how the methods affect the OID comparison.

Cloud particles are typically much larger than the wavelengths (905 and 1550 nm) of the OID, especially particles in cirrus cloud anvils. The ratio of particle circumference to wavelength defines the light-scattering regime (geometric, Mie, or Rayleigh) and is known as the size parameter α, which is given by 3α=πDλ, where D is the cloud particle diameter, and λ is the wavelength of incident light (van de Hulst, 1981). Light scattering is in the geometric optics regime when the size parameter is greater than 100, in the Mie regime when the size parameter is between 0.1 and 100, and in the Rayleigh regime when the size parameter is less than 0.1 (Bohren and Huffman, 1983). Most size channels of the combined cloud probe spectrum are in the geometric scattering regime (Table 1). However, determining backscatter coefficients from particle size distributions using geometric optics methods produces inaccuracies due to a lack of higher order scattering terms (Yang and Liou, 1995; Zhou and Yang, 2015). Previous studies (Cairo et al., 2011) comparing in situ measured particle size distributions and backscatter measurements have used Mie theory despite it strictly only applying to spherical particles, since there is no upper size limit and can therefore be applied to the entire cloud probe size range. Applying Mie theory to particle size distributions obtained using the area-equivalent method for determining particle diameters produces larger backscatter coefficients by an average factor of 2 or more (Mishchenko et al., 1996). Applying more sophisticated methods would reduce the backscatter coefficient uncertainty; however, such methods require knowing not only the size distribution of all single crystal types within the sampled cloud but also how aggregates of these crystals scatter light. To avoid such complexity, Mie theory is used with an acknowledgment of the method's uncertainty.

Backscatter coefficients depend on particle scattering efficiency at 180∘ (backscatter efficiency), the number concentration, and the particle size (Bohren and Huffman, 1983). The backscatter coefficient equation (Zhang et al., 2015 p. 271) is given by 4βECP=∑i=1i=maxQbackiηiπri2, where βECP is the derived backscatter coefficient from external cloud probes (ECP) in units of km-1 sr-1; i is the particle size channel number (see Table 1); ηi is the number of particles in channel i; Qbacki is the backscatter efficiency for channel i; and ri is the area-equivalent particle radius in channel i. For water spheres, πr2 is the cross-sectional area (A); for irregular particles such as ice, A is the cross-sectional area imaged by the two-dimensional probes (see the ice crystal image insert in Fig. 1). The cross-sectional area is converted to a diameter (or radius) by determining the sphere that has a two-dimensional projected surface area equivalent to A. In either case, A is the particle area directly measured by the cloud imaging probe as the area of shadowed diodes obtained using SODA2 data processing. Backscatter efficiency indicates the effectiveness of a particle at scattering in the 180∘ direction regarding incident light; it is given by 5Qback=QextωP11π4π, where Qext is extinction efficiency, ω is single-scattering albedo, and P11π is the scattering phase function at 180∘. Backscatter efficiency (Fig. 3) is calculated using the Python-based software, MiePlot (Laven, 2020), which has a definition (see Graph types/Phase function in MiePlot program's help documentation) of Eq. (5). Backscatter efficiency calculations use 20 ∘C for the medium temperature, 905 nm for the incident light wavelength, and refractive indices of 1.3263+5.6×10-7i for water at 20 ∘C (Kedenburg et al., 2012) and 1.3031+5.6×10-7i for ice (Warren and Brandt, 2008). The refractive indices depend not only on incident wavelength but also on particle temperature and, therefore, density (Wesely, 1976). However, sensitivity tests using 0 and 10 ∘C water refractive indices show a difference of less than 1 % for particle diameters up to 105 µm. Since water at 4 ∘C has the highest density (Tanaka et al., 2001), testing both 0 and 10 ∘C provides backscatter efficiency results over a range of densities for water droplets. As ice particle density depends on multiple formation factors other than temperature, the effects of ice particle temperature are less relevant and likely have little impact. Correct utilization of MiePlot for the determination of water backscatter efficiency has been confirmed by obtaining results comparable to Fig. 61 of van de Hulst (1981) and Fig. 1 of Lolli et al. (2013).

Backscatter efficiency values are calculated using MiePlot for diameters distributed log-normally between 1 µm and 30 mm. Intervals within the smallest diameter channel (1 µm) are 0.0001 µm and increase to 3 µm intervals in the largest diameter channel (30 mm). Backscatter efficiencies are arithmetically averaged for all particle diameters within each channel (Shishko et al., 2020) in the combined particle size spectrum (Table 1), beginning at 5 µm and ending at 30 mm (Fig. 3). Averaging over channel intervals smooths out rapid changes in backscatter efficiencies with diameter. As a result, the backscatter efficiencies for water and ice are nearly constant (within approximately 10 %) for particle diameters up to 105 µm (the upper limit of size bin 10 in Table 1). Above 105 µm, water has a backscatter efficiency that increases above the isotropic backscatter efficiency (Zhang et al., 2015, p. 271) of 0.0796 (1/(4π)), while ice has a backscatter efficiency that decreases below the isotropic backscatter efficiency, indicating depression near the backscattering direction. Therefore, the water-to-ice difference increases by 2 orders of magnitude at larger sizes, despite water and ice only having a small difference in the real part of the refractive index. Also, there is a backscatter efficiency dip around the 10 000 µm diameter that is likely due to the destructive interference of surface waves (Hovenac and Lock, 1992). For more information about the effects of surface waves on light scattering, see Chýlek et al. (1980). While the backscatter efficiency can be related (Zhang et al., 2015, p. 271) to the lidar ratio (reciprocal of the backscatter-to-extinction ratio), the lidar ratio is not used to derive the OID backscatter coefficient (see discussion of Eq. 1); therefore, the only important consideration is using the same scattering phase function normalization for both the OID backscatter coefficient calculation and βECP. In both cases, the phase function (P) integrated from 0 to 180 is given by 6∫02πPθcos⁡θsin⁡θdθ=4π.

Uncertainty in the backscatter coefficient (δβECP) due to sizing and counting errors is derived using a weighted error propagation in the quadrature method (Berendsen, 2011): 7δβECP=∑i=1i=maxQbackiπri2δηi2+2ηiQbackiπriδri2, where δni is the concentration uncertainty for channel i, and δri is the particle radius uncertainty for channel i. Poisson statistics determine the absolute uncertainty in concentration: 8δηi=ηi⋅1Ni, where Ni is the counted particle number per channel (Horvath et al., 1990). Another example of applying Poisson statistics to cloud probe analysis can be found in Baumgardner et al. (2014). One half of the channel width is used as the particle radius uncertainty, which is a lower bound on the uncertainty. Additional factors could result in a larger uncertainty, such as particles outside the depth of field appearing too large and hence not within the correct size channel (O'Shea et al., 2019) or particles not being randomly oriented within the cloud probe measurement volumes.

The OID is undergoing development and is an immature instrument compared to two-dimensional array probes; hence, a full uncertainty analysis is not available. Conducting a full OID uncertainty analysis is beyond the scope of this paper; instead, the 1 Hz OID backscatter coefficient uncertainty (δβOID) is obtained from the standard deviation of the 5 Hz OID data. The standard deviation includes some natural variability, as the aircraft traverses different cloud conditions, in addition to random uncertainty; hence, the uncertainty is an upper bound. OID and ECP agreement is considered to occur when the measurement difference is less than 3 standard deviations (Berendsen, 2011), which is calculated using 9σ3=3δβOID2+δβECP2. Thus, a time series plot of the OID and ECP backscatter coefficient difference is useful to determine periods of measurement agreement and disagreement. The third standard deviation is used as a threshold in accordance with the three sigma rule (Pukelsheim, 1994), since the OID is in development and Eq. (9) does not include systematic errors (e.g., systematic error due to spherical particle assumptions).

OID backscatter coefficients are compared to 1 Hz backscatter coefficients derived from the external cloud probes (2D-S, HVPS3) using a case study analysis of four 60 s CAPE2015 flight segments, with each segment selected to represent a different meteorological condition. The cases (Table 2) are at four different temperatures (+7, +4, -33, and -46 ∘C), which provide two warm cloud cases and two cold cloud cases at constant flight heading and altitude (Fig. 4). The warm cases (Fig. 4a and b) are at approximately 3 km altitude over the ocean, just off the east coast of Florida. The warm cloud cases have small liquid droplets that are not observable on radar (Table 2). The +7 ∘C case occurred early in the flight while waiting for air traffic control climb permission and sampled developing cumulus clouds with very few drizzle-sized droplets. Similarly, the +4 ∘C case also occurred before ascending to sample cirrus clouds but sampled mature cumulus clouds with drizzle-sized droplets. As expected for these warm temperatures, the 2D-S images (Fig. 4a and b) show small spherical droplets. The cold cases (Fig. 4c and d) occurred above 9 km in cirrus cloud anvils. Melbourne radar observations have cloud bases at 1000 m a.m.s.l. (Table 2) for both cold cloud cases, which indicates that sampling is above the stratus rain region instead of the cirrus anvil outflow region of the storm. Cloud tops are located at 11 000 m a.m.s.l. for the -33 ∘C case and 15 000 m a.m.s.l. for the -46 ∘C case. The 2D-S images show mostly irregularly shaped particles indicative of ice crystals. The -33 ∘C case is warmer than the homogeneous freezing temperature (-38 ∘C) and has some small particles appearing round, which could be liquid or frozen cloud droplets. Therefore, the -33 ∘C case may contain liquid droplets; however, the -46 ∘C case, being much colder than the homogeneous freezing temperature, contains only ice particles.

Cloud probe images (2D-S and HVPS3) are processed and combined to create a 1 Hz ECP dataset, which is used to derive backscatter coefficients (Eq. 4) and uncertainty (Eq. 7). The absolute uncertainty is greatest for the +7 ∘C case (Fig. 5a), because the backscatter coefficient magnitude is 10 times the +4 ∘C case and 100 times the cold temperature cases. Figure 6a shows that OID and ECP backscatter coefficients agree (within 3 standard deviations) at all times for the +7 ∘C case. The greatest difference occurs at 10 s, where the ECP backscatter coefficient is over twice the OID's. Overall, the +7 ∘C case ECP backscatter coefficient is typically higher than the OID. The +7 ∘C water cloud contains high concentrations (Fig. 7a) of small droplets (i.e., 20 µm mean diameter), with the majority of the calculated backscatter due to droplets between 5 and 30 µm (Fig. 7d).

Figure 5b shows that the +4 ∘C case has ECP backscatter coefficients that vary with the OID backscatter coefficient with the smallest difference between the two of any of the four cases; however, the ECP backscatter coefficient is consistently lower. The OID and ECP backscatter coefficients agree within uncertainties (Fig. 6b) throughout the entire period. The difference changes little, because the particle concentration (Fig. 7a) and mean diameter (Fig. 7b) are approximately constant. Compared to the +7 ∘C case, the concentration is an order of magnitude less, and the mean diameter is greater, being approximately 50 instead of 20 µm. Many of the observed drops are from 30 to 70 µm (Fig. 7c), with the particle size distribution peaking at approximately 50 µm. Particles from 100 to 300 µm diameter are 100 times more plentiful than in the +7 ∘C case.

For the -33 ∘C case (Figs. 5c and 6c), the OID and ECP backscatter coefficients agree for the first 20 s. Similarly to the +4 ∘C case, the OID backscatter coefficient is consistently higher than the ECP backscatter coefficient. However, around 24 s, there is a sudden change in backscatter coefficient agreement that corresponds to an increase in mean particle size and concentration. Figure 7a shows particle concentration increasing from approximately 0.1 to 0.5 cm-3, and Fig. 7b shows the mean particle diameter increasing from 200 to slightly over 600 µm around the 24 s change. While both the OID and ECP backscatter coefficients increase at this time, the magnitude of the OID coefficient is significantly larger. For the last 30 s, the OID backscatter coefficient is 2 to 3 times the ECP backscatter coefficient, with disagreement exceeding 3 standard deviations. The particle size spectrum (Fig. 7c) and backscatter coefficient distribution (Fig. 7d) are much broader and flatter than in the warm cloud cases. Figure 7d indicates the majority of the backscatter coefficient is due to particles from 100 to 2000 µm in diameter.

The -46 ∘C case (Fig. 5d) also shows significant differences between the backscatter coefficients. For the first 48 s, the backscatter coefficient is in disagreement (Fig. 6d), with the ECP backscatter coefficient being consistently less than the OID's. However, both backscatter coefficients exhibit the same temporal trends. For the last 12 s of the -46 ∘C case, the backscatter coefficients are mostly in agreement. As Fig. 7b shows, the mean particle diameter is nearly constant at approximately 200 µm, with a decrease at the end of the period. Both the -46 and -33 ∘C case show 2D-S images with large, irregularly shaped particles (Fig. 4d and c, respectively). The -46 ∘C case has small particles that are irregular in shape, while the -33 ∘C case contains small particles that appear round.

There is a strong correlation (0.91) between βECP versus βOID, with the linear fit line being less than the one-to-one line over most of the data range (Fig. 8). The -33 ∘C case has higher OID backscatter coefficients compared to similar ECP backscatter coefficients from the -46 ∘C case. This difference at an ECP backscatter of 0.2 km-1 sr-1 (Fig. 8) is far greater than the calculated uncertainty, which indicates an unaccounted source of systematic error. Figure 9 shows that the Nevzorov probe TWC and backscatter coefficient have a power-law relationship. The backscatter–TWC relationship indicates that the OID instrument is sensitive to TWC over a wide range of cloud conditions, including clouds with TWC as low as 0.02 g m-3. The OID's sensitivity to small values of TWC (plot insert in Fig. 9) in optically thin ice clouds is an important characteristic for deployment on high flying aircraft. The OID has less scatter than the ECP for the backscatter–TWC plots, likely due to the larger sample volume of the OID. Figure 10 includes all observations greater than 0 and less than -20 ∘C for the four case study flights. Data between 0 and -20 ∘C are omitted to exclude strongly mixed-phase cloud conditions as the ECP data processing, and thus calculated backscatter coefficients, do not account for ice and liquid water backscatter efficiencies simultaneously. Including the whole flights, there is much more scatter between the ECP and OID backscatter coefficients (Fig. 10) than for the four 60 s flight segments (Fig. 8). Similar to the flight segment results in Fig. 8, the trend over the whole flights (Fig. 10) indicates that the ECP backscatter coefficients are lower than the OID coefficients. However, there is far more scatter between the OID backscatter coefficient and the Nevzorov probe TWC for cold cases (Fig. 10) (R2=0.43) than warm cases (R2=0.69). There appears to be two regimes in the OID backscatter–TWC plot (Fig. 10); however, the regression fits for water and ice conditions do a poor job of picking up on the two regimes due to the large amount of scatter in the 1 Hz data. Long time averages and restricting the aircraft sampling to level legs (as done for Figs. 8 and 9) seem necessary to obtain a robust relationship.

The +7 ∘C case has the largest backscatter coefficient and is the only case where the ECP-derived backscatter coefficient is larger than the OID measurement. This is due to high droplet concentrations saturating the OID above its upper response limit of 22 km-1 sr-1. The OID's limited dynamic range in a dense cloud of small water droplets is not a serious liability, since sensing ice particle concentration is the main measurement objective and since warm water clouds do not produce ice particle icing. Furthermore, environments containing severe conditions would not require such large concentrations to be flagged by the OID. Note that the cumulus cloud sampled in the +7 ∘C case has a large water content (Table 2), and the backscatter coefficients to water content comparisons indicate that the OID is able to measure over 1.0 g m-3 before saturation (Fig. 9).

Excluding the +7 ∘C case, ECP backscatter coefficients are consistently lower than those measured by the OID (Fig. 5b, c, d, and summarized in Fig. 8), which suggests a possible systematic error. The most likely systematic error relates to using Mie Theory and assuming that cloud particles are spherical ice scatterers. However, previous studies (Cairo et al., 2011) and theory (Mishchenko et al., 1996) would suggest that the ECP would be larger than the OID by a factor of 2 or more, while the results indicate lower ECP backscatter coefficients. There are several other possible sources of error that could produce a bias. Two-dimensional cloud imaging probes (Fig. 1) have several factors that can bias sizing and counting of particles. Particle sizing uses the area-equivalent method for all cases, which is appropriate for liquid droplets; however, this may not be the best sizing method to obtain the equivalent diameter for ice particles using Mie backscattering. The fast-circle method results in larger diameter ice particles and larger calculated backscatter coefficients from the ECP (see Supplement). Particle diameters less than 100 µm are difficult for the 2D-S to measure due to the small depth of field, which could result in lower backscatter coefficients than those measured by the OID. However, the 2D-S LWC is larger than the Nevzorov probe LWC measurements for the CAPE2015 field project. Since the OID backscatter coefficients are more reflective of the 2D-S LWC, the Nevzorov LWC may be low due to an inability to fully capture and/or evaporate the largest water droplets. OID measurements may have a bias despite no indication of malfunctions. Another possibility is the OID receiving backscatter from particles too small to be measured by the two-dimensional cloud imaging probes; however, Fig. 7d indicates low backscatter coefficient contribution from the smallest sized channels. Additional measurements with carefully calibrated instruments are necessary to resolve the discrepancy between the OID and ECP for ice-containing clouds; however, the OID and ECP measurements do agree within the calculated uncertainties for liquid clouds (Fig. 6).

There are other possible errors which affect both liquid water and ice water measurements. A bias in concentration could be due to an error in the airspeed used for determining the sampling volume (Eq. 2). However, the aircraft's airspeed error is estimated to be between 1 % and 3 % based on measurement comparisons, which is too small to account for the bias. Another error source may be coupling probes to the ambient airflow environment. Airflow deviations around instrument pylons can influence particle concentrations, even when probes are placed well in front of the wing's leading edge (Baumgardner, 1984). Pressure changes and streamlines alter particle flow around both wings and instruments, with small particles being most heavily affected (Spanu et al., 2020). This phenomenon results in lower measured concentrations of small particles compared to actual values in undisturbed air. Another possible concentration bias is in processing asynchronous 2D probe images to calculate the sample area. Errors in determining sample area are size dependent and larger for small particles (Korolev et al., 2013a); therefore, such errors are more important for the +7 and +4 ∘C cases.

A sample area error affects particle concentration, which has the larger uncertainty contribution. The +4 ∘C case (Fig. 9) has an average contribution to uncertainty (Eq. 7) of 0.4 km-1 sr-1 from particle concentration, while only 0.045 km-1 sr-1 from particle size. Similarly, the -46 ∘C case (Fig. 5d) has uncertainty of 0.24 km-1 sr-1 from concentration and only 0.03 km-1 sr-1 from particle size. Hence, the primary contributor to total uncertainty is fluctuation in concentration. Concentration uncertainty can be reduced by averaging over longer time periods; however, systematic differences between the OID and ECP would not change.

Bias in calculated sample volume cannot explain the change in agreement between OID and ECP backscatter coefficients that occurred between 20 and 30 s for the -33 ∘C case (Figs. 4c, 5c, and 6c), nor can it explain the overall disagreement in the -46 ∘C case. Onset of the -33 ∘C case discrepancy could be due to a change in the particle size spectrum (Fig. 11). Manually reviewing 2D-S and HVPS3 images indicates an increased number of larger particles between the first and last 20 s of the -33 ∘C case. This change in particle size distribution is believable, since there is a smooth decrease in concentration with increasing size. Furthermore, the combined 2D-S and HVPS3 particle size distribution is similar to the two-dimensional cloud (2D-C) probe (Fig. 2) distribution (Fig. 11), and performance checks conducted by the manufacturers after CAPE2015 found no measurement issues. Additionally, a manufacturer's review of the OID indicated no measurement issues. Therefore, both the OID and ECP measurements seem to be valid for the -33 ∘C case.

A notable difference between the -33 and -46 ∘C cases is LWC measured by the Nevzorov probe (Fig. 12). The Nevzorov probe measurement for the -46 ∘C case has near-zero LWC, as expected for a temperature below the threshold temperature for homogeneous freezing. However, the Nevzorov probe measurement indicates a mixed-phase cloud for the -33 ∘C case. While there are LWC data shown for the entire -33 ∘C case, LWC increases from an average of 0.022 to 0.027 g m-3 (∼30 % of the TWC) just after disagreement between backscatter coefficients begins. An abrupt increase in TWC also occurs at this time, which should result in both the OID and ECP backscatter coefficients increasing; however, while the increase does occur simultaneously (Fig. 5c), the discrepancy between backscatter coefficients after the increase is of interest. While the increase of LWC is small, the additional LWC would increase the difference between backscatter coefficients, since ECP processing assumes an ice backscatter efficiency, which is lower than water backscatter efficiency (Fig. 4). Furthermore, the increase in LWC likely corresponds to the particle size distribution increase in the 100 to 200 µm range (Fig. 11), which has a high contribution to the overall backscatter coefficient (Fig. 7d). It should be noted that, while the Nevzorov probe has a collection efficiency of nearly 1 for droplets with a diameter of less than 100 µm (Korolev et al., 1998), collection efficiency is approximately 0.5 for clouds with a mean particle volume diameter of 150 to 200 µm (Biter et al., 1987). During the disagreement period, there are significantly more particles with diameters from 150 to 200 µm (Fig. 11), which would require a greater, and not readily available, correction to Nevzorov probe LWC measurements.

Even at temperatures as low as -33 ∘C, supercooled liquid water droplets can still exist (Rosenfeld and Woodley, 2000). Supercooled liquid water droplets have even been observed by the same OID at temperatures as low as -30 ∘C in Anderson and Ray (2019). Similarly to Anderson and Ray (2019), such supercooled liquid water droplets are evident – but not proven – from the round particles seen in 2D-S images taken at the LWC measurement peak (Fig. 4c). Additionally, the Rosemount Icing Detector (Fig. 2) rod frequency (not shown) indicates the presence of LWC. The authors speculate that the presence of supercooled liquid water droplets in the -33 ∘C case results in larger OID backscatter coefficients due to higher (see Q ratios in Table 1) liquid water backscatter efficiency (Fig. 3). Calculating ECP backscatter coefficients using liquid water scattering efficiencies for particles less than 500 µm and ice scattering efficiencies for particles larger than 500 µm, results in ECP and OID agreement for the -33 ∘C case and would eliminate the systematic difference at the ECP backscatter coefficient of 0.2 km-1 sr-1 in Fig. 8. Accounting for mixed-phase conditions in ECP backscatter coefficients would require a methodology that is able to simultaneously utilize a particle size spectrum for both liquid water and ice, which is beyond the scope of this study. The authors would like to note that mixed-phase clouds are only one possibility, and further analysis is required.

While each of the previously mentioned sources of error increase uncertainty within the backscatter coefficient comparisons and should be acknowledged, their contributions are small relative to the assumption made of particles either being water or ice. Incorrect phase assumptions when determining the backscatter efficiencies used in calculating backscatter coefficients can have a significant impact on the results due to large differences in efficiencies between ice and water, particularly for particles over 140 µm in diameter (see Table 1). At a diameter of 1000 µm, incorrectly assuming water over ice (or vice versa) would impact the backscatter efficiency (and thus the backscatter coefficient) by 1500 %. This could be solved via testing in a closed system where particle phases can be known with certainty.

For ice particles specifically, the most significant uncertainty is likely the spherical particles assumption of Mie Theory and the corresponding processing method used to obtain the size of particles – i.e., area-equivalent (figures presented herein) versus the fast-circle method (figures presented in the Supplement). This is made evident in Fig. 10, where there is significantly less correlation between the OID results and the Nevzorov TWC for the cold cases as opposed to the warm cases. The area-equivalent method is best for spherical droplets, since the imaging probes directly measure area; however, the best processing method for ice particles is not clear, as a sphere that equivalently backscatters light – which likely is not an area-equivalent sphere obtained from one viewing angle – is required.

A backscatter coefficient comparison is conducted for both warm and cold cloud conditions using an in situ lidar (OID) and observations from externally mounted cloud particle probes (ECP). ECP backscatter coefficients are calculated using combined 2D-S and HVPS3 data over the 5–30 000 µm size range. ECP-derived backscatter coefficients are consistently lower than OID measurements for the +4, -33, and -46 ∘C cases, suggesting a systematic bias in either or both of the datasets (Fig. 5). However, the ECP-derived backscatter coefficient is higher than the OID measurement for the +7 ∘C case due to OID saturation. In the +7 and +4 ∘C cases, the particles that primarily contribute to the backscatter coefficient are in the 5–50 µm diameter range, while in the -33 and -46 ∘C cases, the particles that contribute the most are in the 200–3000 µm diameter range (Fig. 7d). ECP backscatter coefficients are less than 3 standard deviations from OID measurements, indicating agreement for the +7 ∘C case and +4 ∘C cases (Fig. 6). However, there is disagreement in the majority of the -46 ∘C case. The -33 ∘C case has disagreement for 36 out of 60 s, which is possibly due to using ice scattering efficiencies for all particles when small particles are potentially liquid. All case comparisons show qualitatively that backscatter coefficients are a suitable proxy for LWC and IWC as low as 0.02 g m-3 (Fig. 9). A comparison between OID and ECP backscatter coefficients to Nevzorov probe TWC for all times during four CAPE2015 flights (Fig. 10) has a correlation of 0.69 for warm cases but only 0.43 for cold cases. Considering the measurement uncertainties, the case study analysis, and overall flight comparison, the OID is a useful tool for detecting dangerous ice particle icing conditions, even at low concentrations.

The CAPE2015 field project analysis could be expanded to include additional field projects. Additionally, future work could include both the 905 and 1550 nm OID wavelengths. Using both wavelengths would allow two parameters of cloud particle size distributions to be determined, helping to obtain a TWC. Additionally, OID polarization measurements could distinguish the liquid–ice fraction of clouds (Yang et al., 2013). New instrumentation, such as the Particle Habit Imaging and Polar Scattering (PHIPS) probe (Schön et al., 2011) deployed during the CapeEx19 Florida field project, would be very useful in determining the fraction of liquid water in mixed-phase clouds. Quantifying the liquid water fraction of mixed-phase clouds would allow for creation of both liquid and ice 1 Hz particle size distributions, so liquid and ice scattering efficiencies could simultaneously be used to obtain the backscattering coefficient. PHIPS data would also aid in the identification of ice particle habits. Knowing the ice particle habits would allow for more accurate scattering calculations to be utilized (Yang et al., 2005).