Ground-based all-sky mid-infrared and visible imagery for purposes of characterizing cloud properties

Introduction Conclusions References


Introduction
Uncertainty in the characterization of clouds in general circulation models (GCMs) is one of the major causes of the broad range of future climate change predictions (Atmospheric System Research (ASR) Science and Program Plan, January 2010).Hemispherical Cloud Fraction (HCF), which is closely related to cloud fraction (a dominant modulator of radiative fluxes), has been an integral part of the observational dataset that feed these GCMs (Kassianov et al., 2005).HCF, however, has only been directly determined at the ARM sites during daytime hours utilizing the Total Sky Imager (TSI) (Long et al., 2001).Other indirect cloud fraction data products can be derived from surface radiometers and the statistical analysis of lidar and radar observations (Qian Introduction

Conclusions References
Tables Figures

Back Close
Full  , 2012).A direct means of determining nighttime HCF has been and remains a critical programmatic gap in ARM's observational dataset and is an important factor in understanding the life cycle of clouds, one of the central themes of the ASR program.
The mid-infrared (mid-IR) atmospheric window from 8-13 microns (µm) has long been known to hold great promise in closing this gap as well as providing other valuable ground-based cloud properties and atmospheric data (Shaw et al., 2005;Thurairajah and Shaw, 2005).A thermal IR imager has the distinct advantage of directly detecting emission from clouds, rather than relying on scattered light or obscured starlight, and is not hampered by the presence of the Sun or the Moon, thus providing consistent and reliable information under a wide variety of conditions.A major challenge for thermal imagers has been separating the effects of water vapor emission from that of cloud emission, particularly cirrus clouds (Brocard et al., 2011).The ASIVA's primary function is to provide radiometrically-calibrated imagery across the entire sky in the mid-IR.Figure 1 shows the clear-sky downwelling radiance as simulated using MODTRAN (Berk et al., 1999) for a standard mid-latitude summer atmosphere pointed at the zenith for 22 mm of precipitable water vapor (PWV), typical of conditions found at the ARM SGP site.Absorption and therefore thermal emission is dominated by water vapor at wavelengths less than 8 µm, by carbon dioxide at wavelengths greater than 13 µm, and by ozone near 9.5 µm.Water vapor absorption lines are present throughout this spectral interval but are least prevalent in the 10.2-12.2µm region.For this reason, a custom 10.2-12.2µm filter for optimizing clear-sky/cloud contrast was fabricated for the ASIVA instrument.The spectral response of this filter (shown in red) as well the 8.25-9.25 µm filter (shown in blue) used in this research are presented in Fig. 1.This paper will discuss the ASIVA instrument with particular emphasis on the calibration procedures that have been developed to improve mid-IR radiometric performance enabling the removal of water vapor emission.Infrared data analysis procedures that are being developed to characterize cloud properties with particular emphasis on determining HCF will also be discussed.In addition, HCF data from ASIVA's visible channel Introduction

Conclusions References
Tables Figures

Back Close
Full will be discussed and compared to that from the infrared channel as well as from the TSI.

Description of the ASIVA instrument
The ASIVA instrument (shown in Fig. 2) was deployed at the ARM SGP site from 21 May to 27 July 2009.This instrument was a prototype unit that featured an infrared camera subsystem consisting of a 320 × 256 uncooled microbolometer array sensitive to 8-14 µm radiation, a 180-degree (all sky) custom designed hard carbon coated waterproof lens, and a filter wheel which included the two IR filters whose response is shown in Fig. 1.The IR camera provided image data at 14-bit resolution and at a 30 Hz rate.Sixteen of these images were co-added to produce a single frame with an effective exposure of 0.53 s.Sixteen consecutive frames were then bundled to produce a 3-dimensional FITS file (data cube) that was stored to disk for future data analysis.acquired in the closed position.The hatch was then opened wherein one visible and two infrared sky images were acquired until the next data acquisition sequence.All data sequences were identical with the exception of toggling the exposure time for the visible camera between a daytime and nighttime setting.All data were stored to disk.
A collection of images acquired on 3 July 2009 at 13:32 UTC is shown in Fig. 3.An image from the ARM facility's TSI instrument is shown for comparison with ASIVA's visible camera.Note that the ASIVA visible image does not utilize a sun occulter and that its 10-bit resolution allows for better sensitivity near the sun.The ASIVA IR image is a single frame (0.53 s exposure) image and demonstrates that the sun's presence has almost no impact on the image.

Determination of instrument response coefficients
A somewhat more detailed discussion of the ASIVA's calibration procedure can be found in Klebe et al. (2012).In the interest of completeness, much of that discussion is repeated here.The ASIVA instrument incorporates a two-step calibration process in determining the spectral radiance for IR images.The first step in this calibration procedure is to determine the instrument response coefficients G λ for every pixel in the array in each IR filter.These coefficients are generated using Eqs.(1) and (2).
where I λ = Instrumental counts measured for the blackbody reference in a specific filter, λ = emissivity of the blackbody reference in a specific filter, and where t λ = system response as a function of wavelength for a specific filter.
The blackbody spectral density equation BB λ (T ) above assumes a wavelength λ given in units of microns (µm) and is integrated over the system response t λ (which includes the combined effects of filter/lens transmission and nominal IR detector sensitivity as a function of wavelength) for a particular temperature T in Kelvin.The emissivity λ of the blackbody depends on the coating and design of the calibration reference used to cover the IR lens.The emissivity is assumed to be constant for a given filter but can be adjusted from one filter to the next if necessary.A value of 0.95 was used for this study.
To eliminate instrumental offsets, G λ was determined by calculating a least squares linear fit to the I λ vs. λ • BB λ (T ) data for a range of temperatures.The built-in blackbody reference located inside the hatch cover was used to determine the instrumental response of the system and also to measure the fixed pattern components in the image during data acquisition.A temperature sensor was bonded within the blackbody reference and the temperature data were written into the FITS header when acquiring images.A heater was embedded in the blackbody reference and was used to control its temperature during calibration.
The calibration procedure was performed on 21 May 2009.During the procedure, the hatch was opened and the blackbody was heated to ∼ 80 • C and then allowed

Calibration of spectral radiance images
The sky's spectral radiance (F λ Sky ) for a given filter is then determined using Eq. ( 3).
where  2) for ambient temperature T Ref .
A useful quantity utilized in the analysis presented in Sect. 4 is the normalized spectral radiance F λ Sky given by Eq. ( 4).The normalized spectral radiance F λ Sky can be thought of as a proxy to the sky's average emissivity.F λ Sky is generally an underestimate of the true emissivity since the ambient temperature given by T Ref is nominally greater than the mean temperature of the emitting sky.
4 Cloud detection and hemispherical cloud fraction analysis

Verification of calibration procedures
The calibration procedures described in Sect. 3 provide the foundation for cloud detection and other cloud data products that can be derived from the ASIVA instrument.
As a verification of these procedures, ASIVA spectral radiance data were compared with the precisely calibrated data retrieved from the Atmospheric Emitted Radiance Interferometer (AERI) instrument available for the campaign period.The mean spectral radiance was determined by averaging the AERI spectral radiance data over the response of each of the two ASIVA IR channels as depicted in Fig. 5.The 8 min average Introduction

Conclusions References
Tables Figures

Back Close
Full AERI data were utilized, as this cadence was similar to the 5 min cadence used by the ASIVA instrument.Calibrated ASIVA data were then evaluated at the zenith, coincident with AERI's field-of-view.Comparison plots of AERI data with ASIVA data for the two daytime periods that will be highlighted in this paper are shown in Fig. 6.Agreement is very good (< 5 %) for the two daytime comparisons and are representative of the entire campaign dataset.Note that the agreement is good in both clear and cloudy circumstances.

Removal of clear-sky emission
The primary step in the cloud detection process is to remove the effects of clear-sky emission.This is done by employing the all-sky capabilities of the ASIVA instrument.One of the primary challenges of producing a robust cloud decision map is determining the clear-sky radiance in nearly 100 % cloudy conditions.This is accomplished by demanding that the chi-square value for the polynomial fit be less than some threshold (chosen to be 0.002 in this analysis) to ensure a strict goodness-of-fit criterion.If this criterion is not met, the previous polynomial equation that has met this criterion is used to define the clear-sky radiance.We have found this procedure to be very effective.A case in which the sky is nearly 100 % cloudy is illustrated in Fig. 8.Note that the fit denotes the proper clear-sky radiance for this image.

Hemispherical cloud fraction determination
After determining the clear-sky emission, cloud decision masks can be processed by applying the proper thresholds to the clear-sky subtracted images.Using the thresholds (in normalized radiance units) of 0.03 < thin cloud < 0.05 and opaque cloud ≥ 0.05, the HCF comparison plots derived for 21 July 2009 are shown in Fig. 9.
Agreement between TSI data retrieved from the ARM archive and the ASIVA IR data for this day is excellent.Note that the TSI instrument shows erroneous cloud fraction results at the beginning and end of the day.The low sun elevation angles prove more difficult for the TSI instrument.A more sophisticated analysis package is available (Long, 2010) to improve the TSI dataset and will be used for further comparison with ASIVA data.but that the TSI is insensitive to thin clouds in highly overcast conditions.Presently we are adopting a variable lower thin cloud threshold that is dependent on opaque cloud fraction to achieve better agreement with TSI data in both of these circumstances.
Basically the thin cloud threshold will be lowered in clearer sky conditions.Ultimately, we do not expect perfect agreement as the cloud decision analysis is fundamentally different between the IR and visible.However, the primary goal of this research has been to obtain as close agreement as possible between the TSI and ASIVA instrument.

Retrieval of HCF data product from ASIVA visible data
Retrieval of the HCF data product from ASIVA's visible channels uses the same analysis adopted by the TSI instrument.The first-order analysis involves taking the ratio of the red image to blue image and then setting appropriate opaque/thin/no cloud thresholds (Long et al., 2006).Second-order analysis requires taking into account the Sun's position in the sky and adopting a varying threshold depending on a pixel's relative position to the Sun.Second-order analysis is easily implemented in the ASIVA instrument but was not used in the analysis presented below as it was not used in the dataset retrieved from the ARM archive.Figure 12 shows the visible red/blue ratio images coincident with the IR images shown in Figs.7 and 10.Note that the images in Fig. 12 are at 90 • relative to those in Figs.7 and 10.The sun has been occulted in software (the larger circle) and is similar in size to the zone-of-avoidance utilized in the TSI cloud fraction analysis.In addition, a small circle is used to mask an artifact seen at the sun's position reflected through the zenith.This artifact (which can be seen in Fig. 3) is brought about by internal reflection within the fisheye lens.Since the ASIVA instrument does not require a sun occulter like that utilized in the TSI instrument, eliminating this part of sky is relatively minor in the analysis.Thresholds are then set to define the thin and opaque boundaries to provide the best agreement with the TSI instrument.for low sun angles without any further processing and better matches the HCF data retrieved from the ASIVA IR channel.Using these same threshold values to determine cloud fraction for the more challenging dataset of 25 May 2009, we obtain the results shown in Fig. 14.The agreement is remarkably good and demonstrates that the ASIVA visible subsystem is more than adequate reproducing TSI daytime functionality.Development of ASIVA's HCF data product is nearly complete and is currently being applied to the entire field campaign dataset.

Sky/cloud temperature and other potential ASIVA data products
As a radiometrically calibrated instrument the ASIVA has the potential of delivering many other data products that will be useful to the meteorological community.This section discusses the current research in this area.

Determination of sky/cloud temperature
Two temperature data products can be immediately derived from ASIVA's IR radiance data; brightness temperature and color temperature.The brightness temperature (for a given IR filter) of an image can be determined by equating the measured radiance with a blackbody whose temperature yields this same radiance.Brightness temperature images determined from the representative images of 25 May (surface temperature 304 K) and 21 July (surface temperature 297 K) are shown in Fig. 15.Note that the peripheries of the clouds indicate lower brightness temperature consistent with lower optical depth in these regions.
The color temperature can be inferred by taking the ratio of sky radiance images acquired in the 8.25-9.25 µm and 10.2-12.2µm filters and assigning this a temperature for which a blackbody yields this same radiance ratio.Color temperature has the distinct advantage of only being affected by differences in sky/cloud emissivity in the two Introduction

Conclusions References
Tables Figures

Back Close
Full filters but is insensitive to the total optical depth.For this reason we believe that color temperature will ultimately yield a better measure of the true temperature for optically thin clouds.Color temperature images are shown in Fig. 16.Both of the images in Fig. 16 show variations (both positive and negative temperature fluctuations) at the periphery of the clouds due to the motion of the clouds over the data acquisition period.This is particularly evident in the 21 July image as the clouds were very fast moving.Current ASIVA instruments now acquire 8.25-9.25 µm and 10.2-12.2µm image data in a much shorter time interval to combat this problem.Also note that the clear-sky color temperature is higher (one would expect lower temperatures) at the zenith due to sky emissivity differences in the two IR filters.Color temperatures in the optically thick regions of clouds shown in Fig. 16 are consistent with those of Fig. 15 and both indicate cloud temperatures 10-20 K below the ground temperature.To some degree, these color maps already provide an estimate of cloud temperature that may be very valuable to cloud modelers.To improve on the accuracy of this measurement, one will have to account for the intervening atmospheric absorption and emission in each of the filters.This is where knowledge of PWV (discussed in Sect.5.3) is required to provide additional information regarding the atmosphere's radiative properties.

Determination of sky/cloud emissivity
Perhaps one of the most powerful data products that can be derived using the temperature analysis outlined above is an accurate estimate of the emissivity of an image.By assuming that the color temperature is indeed a measure of the true mean temperature for an image, one can compute a blackbody radiance image from the color temperature image.By dividing the measured radiance by the blackbody radiance derived from the color temperature, one arrives at a measure of the emissivity of the sky. Figure 17 shows the results of this analysis.
The accuracy of this measure is somewhat hampered by the variations in emissivity between the two IR filters but can be corrected using knowledge of the PWV burden and the information it yields regarding the atmosphere's radiative properties.Ignoring Introduction

Conclusions References
Tables Figures

Back Close
Full the effects of cloud motion the images of Fig. 17 show cloud emissivity near unity for opaque clouds as one would expect.They also show the expected variations in clear-sky emissivity (i.e.lower emissivity at the zenith and higher emissivity near the horizon).

Determination of precipitable water vapor (PWV)
As discussed above, determination of PWV is important as it can provide valuable ancillary information in the analysis of other cloud property data products, in particular refining cloud temperature measurements.The basic analysis strategy is to compare the clear-sky envelope (described in Sect.4.2) with modeled data.The model data are constructed using a series of MODTRAN simulations that provide a parameterization of the normalized clear-sky downwelling radiance as a function of PWV evaluated at different elevation angles (i.e.airmass).Preliminary simulations have been run for each of ASIVA's filters using a standard mid-latitude summer atmosphere at sea level and a lapse rate extracted from a radiosonde dataset acquired for 21 July 2009.The simulated dataset is then best fit to the clear-sky envelope determined for a particular image thus providing an estimate of PWV. Figure 18 shows the result of this analysis for the 25 May 2009 and 21 July 2009 datasets.The PWV data are compared with that retrieved from the Microwave Radiometer (MWR), an instrument with accuracy of better than 1 mm PWV.The correlation is fairly good for the 21 July dataset (the day from which the radiosonde data was used in the MODTRAN analysis) but deviates significantly for the 25 May dataset.This suggests coincidental radiosonde data may be required to improve the accuracy of ASIVA's estimate of PWV.
Better accuracy in the determination of PWV may not be required in that this measure is only required to make second order corrections to data products such as the sky/cloud temperature images.If better accuracy is required, other instruments such as MWR could be used in a value added product (VAP).

Determination of cloud height
Cloud height can be estimated from ASIVA cloud temperature images by utilizing the altitude vs. temperature information retrieved from radiosonde data.Figure 19 shows the comparison of cloud height derived from ASIVA brightness temperature images at zenith with cloud height measurements retrieved from the ARM ceilometer (CEIL) operated at SGP during the campaign period.Only very opaque clouds (opaque cloud ≥ 0.3) were used in deriving cloud height measurements in these comparisons to insure that the clouds were optically thick and that their brightness temperature was a good estimate of the cloud temperature.Agreement is good for the 25 May dataset but varies significantly for the low clouds seen in the 21 July dataset.This is presumably due to the clouds being much cooler than where the lapse rate would place them.
In addition, the ASIVA instrument measures the mean temperature of the cloud to one optical depth.This will always be located at a higher altitude then the cloud base.

Conclusions
The ASIVA demonstrates considerable promise in providing a diurnal hemispherical Discussion Paper | Discussion Paper | Discussion Paper | et al.
Discussion Paper | Discussion Paper | Discussion Paper | The ASIVA visible camera subsystem featured a color progressive scan 2048 × 1536 CMOS detector array and a 180-degree off-the-shelf lens.The visible camera provided image data at 10-bit resolution and exposures up to 2 min in length.The instrument featured a unique hatch/radiation shield subsystem used for radiometric calibration.The hatch subsystem provided the following relevant features: (1) integrated the IR blackbody reference and visible dark reference into a single hatch mechanism.(2) IR blackbody reference and the visible reference remained in the same protected orientation (pointed downward) as the hatch mechanism was opened and closed.(3) Temperature sensor and a heater were embedded in the IR blackbody reference to provide in situ radiometric image calibration.(4) Radiation shield allowed the blackbody reference to equilibrate with the ambient air temperature.An observing script governed the data acquisition process which determined what filters and exposure times were to be used for each 5 min data sequence.Each sequence began with the hatch closed.An IR blackbody reference image (in each of the two filters) and a visible dark reference image (of the appropriate exposure) were Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | to passively cool down to near ambient temperature.During the cool-down period, the hatch was periodically closed (2 min intervals) to take calibration data in each of the two IR filters.Data were acquired in this fashion for approximately sixty minutes wherein the hatch stayed open for the majority of the time to prevent heating of the IR lens during the calibration procedure.The data were analyzed as described above and a calibration image file containing the G λ values for each pixel was created.An example of the calibration dataset used to establish G λ for a single central pixel for the 10.2-12.2µm filter is provided in Fig. 4. The response is very linear over a factor of two in radiance yielding great confidence in extrapolating to low radiance values as is done when observing clear skies with low PWV.Discussion Paper | Discussion Paper | Discussion Paper |

λ
Sky = Instrumental Counts measured for the sky image, I λ Ref = Instrumental Counts measured for the reference blackbody image, G λ = Instrument response coefficients derived from Eq. (1), and BB λ (T Ref ) = Integrated Blackbody Radiance derived from Eq. ( Screen / Esc Printer-friendly Version Interactive Discussion Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper |

Figure 7
Figure7illustrates the procedure in which the clear-sky normalized radiance is determined from the ASIVA IR image dataset.In Fig.7a, the normalized radiance pixel data are plotted as a function of airmass for the image shown in Fig.7b.The normalized radiance data are sorted into 29 airmass bins of roughly equal pixel count.The lower envelope of points in each airmass bin (shown as red squares) is fit to a 2ndorder polynomial equation (gold line) which identifies the clear-sky radiance.Even in this very cloudy image the lower envelop is still well defined (chi-square = 0.0002) and serves as an excellent representation of the underlying clear-sky emission.The clear-sky emission is then described as a function of airmass utilizing this polynomial equation and is then subtracted from the original normalized radiance image to yield the clear-sky subtracted image shown in Fig.7b.This image forms the basis of cloud fraction determination.A cloud/no cloud decision can be simply made by choosing a single threshold value, above which an individual pixel is determined to be cloudy.Two thresholds can be used to determine the presence of "thin" and "opaque" clouds, the criteria employed by the TSI instrument.The advantage of expressing the clearsky subtracted image in normalized radiance is that as mentioned it is related to the emissivity of the cloud and is largely independent of ambient temperature.
21 July was dominated by thick opaque clouds.The 25 May 2009 dataset, which provided a mix of thin and opaque clouds, represents more challenging conditions for cloud fraction analysis.Figure 10 illustrates the difficulty in determining thin clouds.The problem arises in that the thin cirrus clouds evident in Fig. 10b (deep blue color) are below the thin cloud threshold of 0.03 used in the analysis of the 21 July 2009 dataset.As can be seen in Fig. 11 below, much better agreement (in particular for thin cloud determination) with TSI data can be achieved by lowering this threshold to 0.016.However applying this analysis to the 21 July dataset would produce a larger fraction of thin clouds than shown in Fig. 9.It may be that these thin clouds are indeed present Discussion Paper | Discussion Paper | Discussion Paper | Figure 13 shows the cloud fraction analysis for 21 July 2009.In contrast to the TSI, the ASIVA visible channel performs very well Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper | Discussion Paper |