Articles | Volume 15, issue 17
https://doi.org/10.5194/amt-15-5159-2022
© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/amt-15-5159-2022
© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Extending water vapor measurement capability of photon-limited differential absorption lidars through simultaneous denoising and inversion
Willem J. Marais
CORRESPONDING AUTHOR
Space Science Engineering Center, University of Wisconsin–Madison, Madison, Wisconsin, USA
Matthew Hayman
National Center for Atmospheric Research, Boulder, Colorado, USA
Related authors
Jeffrey S. Reid, Robert E. Holz, Chris A. Hostetler, Richard A. Ferrare, Juli I. Rubin, Elizabeth J. Thompson, Susan C. van den Heever, Corey G. Amiot, Sharon P. Burton, Joshua P. DiGangi, Glenn S. Diskin, Joshua H. Cossuth, Daniel P. Eleuterio, Edwin W. Eloranta, Ralph Kuehn, Willem J. Marais, Hal B. Maring, Armin Sorooshian, Kenneth L. Thornhill, Charles R. Trepte, Jian Wang, Peng Xian, and Luke D. Ziemba
EGUsphere, https://doi.org/10.5194/egusphere-2025-2605, https://doi.org/10.5194/egusphere-2025-2605, 2025
This preprint is open for discussion and under review for Atmospheric Chemistry and Physics (ACP).
Short summary
Short summary
We document air and ship born measurements of the vertical distribution of pollution and biomass burning aerosol particles transported within the Maritime Continent’s monsoonal flows for 1000’s of kilometers, and yet still exhibit intricate patterns around clouds near the ocean’s surface. Findings demonstrate that, while aerosol transport occurs near the surface, there is heterogeneity in particle extinction that must be considered for both in situ observations and satellite retrievals.
Meloë S. F. Kacenelenbogen, Ralph Kuehn, Nandana Amarasinghe, Kerry Meyer, Edward Nowottnick, Mark Vaughan, Hong Chen, Sebastian Schmidt, Richard Ferrare, John Hair, Robert Levy, Hongbin Yu, Paquita Zuidema, Robert Holz, and Willem Marais
EGUsphere, https://doi.org/10.5194/egusphere-2025-1403, https://doi.org/10.5194/egusphere-2025-1403, 2025
Short summary
Short summary
Aerosols perturb the radiation balance of the Earth-atmosphere system. To reduce the uncertainty in quantifying present-day climate change, we combine two satellite sensors and a model to assess the aerosol effects on radiation in all-sky conditions. Satellite-based and coincident aircraft measurements of aerosol radiative effects agree well over the Southeast Atlantic. This constitutes a crucial first evaluation before we apply our method to more years and regions of the world.
Willem J. Marais, Robert E. Holz, Jeffrey S. Reid, and Rebecca M. Willett
Atmos. Meas. Tech., 13, 5459–5480, https://doi.org/10.5194/amt-13-5459-2020, https://doi.org/10.5194/amt-13-5459-2020, 2020
Short summary
Short summary
Space agencies use moderate-resolution satellite imagery to study how smoke, dust, pollution (aerosols) and cloud types impact the Earth's climate; these space agencies include NASA, ESA and the China Meteorological Administration. We demonstrate in this paper that an algorithm with convolutional neural networks can greatly enhance the automated detection of aerosols and cloud types from satellite imagery. Our algorithm is an improvement on current aerosol and cloud detection algorithms.
Matthew Hayman, Robert A. Stillwell, Adam Karboski, and Scott M. Spuler
EGUsphere, https://doi.org/10.5194/egusphere-2025-3523, https://doi.org/10.5194/egusphere-2025-3523, 2025
This preprint is open for discussion and under review for Atmospheric Measurement Techniques (AMT).
Short summary
Short summary
A new processing method for lidar data obtained from rapidly changing laser pulse lengths enables measurement of atmospheric water vapor from the ground up to 6 km. The technique blends all captured data to reveal hidden water vapor structures, especially near the surface. This solution offers continuous, high-resolution insights, key for improving weather forecasts. It showcases how flexible laser technology can enhance atmospheric observation.
Jeffrey S. Reid, Robert E. Holz, Chris A. Hostetler, Richard A. Ferrare, Juli I. Rubin, Elizabeth J. Thompson, Susan C. van den Heever, Corey G. Amiot, Sharon P. Burton, Joshua P. DiGangi, Glenn S. Diskin, Joshua H. Cossuth, Daniel P. Eleuterio, Edwin W. Eloranta, Ralph Kuehn, Willem J. Marais, Hal B. Maring, Armin Sorooshian, Kenneth L. Thornhill, Charles R. Trepte, Jian Wang, Peng Xian, and Luke D. Ziemba
EGUsphere, https://doi.org/10.5194/egusphere-2025-2605, https://doi.org/10.5194/egusphere-2025-2605, 2025
This preprint is open for discussion and under review for Atmospheric Chemistry and Physics (ACP).
Short summary
Short summary
We document air and ship born measurements of the vertical distribution of pollution and biomass burning aerosol particles transported within the Maritime Continent’s monsoonal flows for 1000’s of kilometers, and yet still exhibit intricate patterns around clouds near the ocean’s surface. Findings demonstrate that, while aerosol transport occurs near the surface, there is heterogeneity in particle extinction that must be considered for both in situ observations and satellite retrievals.
Luke Colberg, Kevin S. Repasky, Matthew Hayman, Robert A. Stillwell, and Scott M. Spuler
EGUsphere, https://doi.org/10.5194/egusphere-2025-1989, https://doi.org/10.5194/egusphere-2025-1989, 2025
This preprint is open for discussion and under review for Atmospheric Measurement Techniques (AMT).
Short summary
Short summary
Two methods were developed to measure the mixed layer height, an important variable for weather forecasting and air quality studies. An aerosol-based method and a thermodynamic method were tested using a lidar system that can measure vertical profiles of aerosols, humidity, and temperature. Each method performed best under different conditions. Together, they provide a path toward more reliable mixed layer height monitoring with a single instrument.
Meloë S. F. Kacenelenbogen, Ralph Kuehn, Nandana Amarasinghe, Kerry Meyer, Edward Nowottnick, Mark Vaughan, Hong Chen, Sebastian Schmidt, Richard Ferrare, John Hair, Robert Levy, Hongbin Yu, Paquita Zuidema, Robert Holz, and Willem Marais
EGUsphere, https://doi.org/10.5194/egusphere-2025-1403, https://doi.org/10.5194/egusphere-2025-1403, 2025
Short summary
Short summary
Aerosols perturb the radiation balance of the Earth-atmosphere system. To reduce the uncertainty in quantifying present-day climate change, we combine two satellite sensors and a model to assess the aerosol effects on radiation in all-sky conditions. Satellite-based and coincident aircraft measurements of aerosol radiative effects agree well over the Southeast Atlantic. This constitutes a crucial first evaluation before we apply our method to more years and regions of the world.
Robert A. Stillwell, Adam Karboski, Matthew Hayman, and Scott M. Spuler
EGUsphere, https://doi.org/10.5194/egusphere-2025-1288, https://doi.org/10.5194/egusphere-2025-1288, 2025
Short summary
Short summary
A method is introduced to expand the observational capability of a diode-laser-based lidar system. This method allows the lidar transmitter to change the laser pulse characteristics from one shot to the next. We use this capability to lower the minimum altitude of observation and also used it to observe clouds with higher range resolution.
John S. Schreck, Gabrielle Gantos, Matthew Hayman, Aaron Bansemer, and David John Gagne
Atmos. Meas. Tech., 15, 5793–5819, https://doi.org/10.5194/amt-15-5793-2022, https://doi.org/10.5194/amt-15-5793-2022, 2022
Short summary
Short summary
We show promising results for a new machine-learning based paradigm for processing field-acquired cloud droplet holograms. The approach is fast, scalable, and leverages GPUs and other heterogeneous computing platforms. It combines applications of transfer and active learning by using synthetic data for training and a small set of hand-labeled data for refinement and validation. Artificial noise applied during synthetic training enables optimized models for real-world situations.
Scott M. Spuler, Matthew Hayman, Robert A. Stillwell, Joshua Carnes, Todd Bernatsky, and Kevin S. Repasky
Atmos. Meas. Tech., 14, 4593–4616, https://doi.org/10.5194/amt-14-4593-2021, https://doi.org/10.5194/amt-14-4593-2021, 2021
Short summary
Short summary
Continuous water vapor and temperature profiles are critically needed for improved understanding of the lower atmosphere and potential advances in weather forecasting skill. To address this observation need, an active remote sensing technology based on a diode-laser-based lidar architecture is being developed. We discuss the details of the lidar architecture and analyze how it addresses a national-scale profiling network's need to provide continuous thermodynamic observations.
Willem J. Marais, Robert E. Holz, Jeffrey S. Reid, and Rebecca M. Willett
Atmos. Meas. Tech., 13, 5459–5480, https://doi.org/10.5194/amt-13-5459-2020, https://doi.org/10.5194/amt-13-5459-2020, 2020
Short summary
Short summary
Space agencies use moderate-resolution satellite imagery to study how smoke, dust, pollution (aerosols) and cloud types impact the Earth's climate; these space agencies include NASA, ESA and the China Meteorological Administration. We demonstrate in this paper that an algorithm with convolutional neural networks can greatly enhance the automated detection of aerosols and cloud types from satellite imagery. Our algorithm is an improvement on current aerosol and cloud detection algorithms.
Cited articles
Ahn, S. and Fessler, J.: Globally convergent image reconstruction for emission tomography using relaxed ordered subsets algorithms, IEEE T. Med. Imaging, 22, 613–626, https://doi.org/10.1109/TMI.2003.812251, 2003. a
ARM: Raman Lidar: Aerosol backscatter, scattering ratio, lidar ratio, extinction, cloud mask, and linear depolarization ratio derived from Thorson FEX code, ARM (Atmospheric Radiation Measurement) user facility [data set], https://doi.org/10.5439/1373934, 2022. a
Azzari, L. and Foi, A.: Variance stabilization in Poisson image deblurring, in: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), 18–21 April 2017, Melbourne, VIC, Australia, 728–731, https://doi.org/10.1109/ISBI.2017.7950622, 2017. a, b
Beck, A. and Tetruashvili, L.: On the Convergence of Block Coordinate Descent Type Methods, SIAM J. Optimiz., 23, 2037–2060, https://doi.org/10.1137/120887679, 2013. a, b
Boyd, S. and Vandenberghe, L.: Convex optimization, Cambridge university press, ISBN 9780521833783, 2004. a
Cinlar, E.: Introduction to stochastic processes, Dover Publications, ISBN 9780486497976, 2013. a
Donovan, D. P., Whiteway, J. A., and Carswell, A. I.: Correction for nonlinear photon-counting effects in lidar systems, Appl. Opt., 32, 6742–6753, https://doi.org/10.1364/AO.32.006742, 1993. a, b
Fessler, J. A.: Optimization Methods for Magnetic Resonance Image Reconstruction: Key Models and Optimization Algorithms, IEEE Signal Proc. Mag., 37, 33–40, https://doi.org/10.1109/MSP.2019.2943645, 2020. a
Geerts, B., Parsons, D., Ziegler, C. L., Weckwerth, T. M., Turner, D. D.,
Wurman, J., Kosiba, K., Rauber, R. M., McFarquhar, G. M., Parker, M. D.,
Schumacher, R. S., Coniglio, M. C., Haghi, K., Biggerstaff, M. I., Klein,
P. M., Jr., W. A. G., Demoz, B. B., Knupp, K. R., Ferrare, R. A., Nehrir,
A. R., Clark, R. D., Wang, X., Hanesiak, J. M., Pinto, J. O., and Moore,
J. A.: The 2015 Plains Elevated Convection At Night (PECAN) field project,
B. Am. Meteorol. Soc., 98, 767–786, https://doi.org/10.1175/BAMS-D-15-00257.1, 2016. a
Hayman, M. and Spuler, S.: Demonstration of a diode-laser-based high spectral
resolution lidar (HSRL) for quantitative profiling of clouds and aerosols,
Opt. Express, 25, A1096–A1110, https://doi.org/10.1364/OE.25.0A1096, 2017. a, b
Hayman, M., Stillwell, R. A., and Spuler, S. M.: Optimization of linear signal processing in photon counting lidar using Poisson thinning, Opt. Lett., 45, 5213, https://doi.org/10.1364/OL.396498, 2020. a, b, c
Hurst, D., Cullis, P., Hall, E., and Jordan, A.: GCOS Reference Upper-Air Network (GRUAN), GRUAN Lead Centre, DWD, https://www.gruan.org, last access: 26 August 2022. a
Jensen, M. P., Petersen, W. A., Bansemer, A., Bharadwaj, N., Carey, L. D., Cecil, D. J., Collis, S. M., Genio, A. D. D., Dolan, B., Gerlach, J., Giangrande, S. E., Heymsfield, A., Heymsfield, G., Kollias, P., Lang, T. J.,
Nesbitt, S. W., Neumann, A., Poellot, M., Rutledge, S. A., Schwaller, M.,
Tokay, A., Williams, C. R., Wolff, D. B., Xie, S., and Zipser, E. J.: The
Midlatitude Continental Convective Clouds Experiment (MC3E), B. Am. Meteorol. Soc., 97, 1667–1686, https://doi.org/10.1175/BAMS-D-14-00228.1, 2016. a
Kelley, C. T.: Iterative Methods for Optimization, Society for Industrial and Applied Mathematics, https://doi.org/10.1137/1.9781611970920, 1999. a
Lee, S. and Wright, S. J.: Implementing algorithms for signal and image reconstruction on graphical processing units, Optimization Online, 10, University of Wisconsin-Madison, https://optimization-online.org/2008/10/2131/ (last access: 2 September 2022), 2008. a
Marais, W. and Willett, R.: Proximal-Gradient methods for poisson image reconstruction with BM3D-Based regularization, in: 2017 IEEE 7th
International Workshop on Computational Advances in Multi-Sensor Adaptive
Processing (CAMSAP), 10–13 December 2017, Curacao, 1–5, https://doi.org/10.1109/CAMSAP.2017.8313128, 2017. a, b, c
Marais, W. J., Holz, R. E., Hu, Y. H., Kuehn, R. E., Eloranta, E. E., and Willett, R. M.: Approach to simultaneously denoise and invert backscatter and
extinction from photon-limited atmospheric lidar observations, Appl. Opt.,
55, 8316–8334, https://doi.org/10.1364/AO.55.008316, 2016. a, b, c, d, e, f, g
Müller, J. W.: Dead-time problems, Nucl. Instrum. Meth., 112, 47–57, https://doi.org/10.1016/0029-554X(73)90773-8, 1973. a
National Academies of Sciences, Engineering, and Medicine: The Future of Atmospheric Boundary Layer Observing, Understanding, and Modeling: Proceedings of a Workshop, The National Academies Press, Washington, DC, https://doi.org/10.17226/25138, 2018a. a
National Academies of Sciences, Engineering, and Medicine: Thriving on Our Changing Planet: A Decadal Strategy for Earth Observation from Space, The National Academies Press, Washington, DC, https://doi.org/10.17226/24938, 2018b. a
NCAR/EOL MPD Team: NCAR MPD data, Version 1.0, UCAR/NCAR – Earth Observing Laboratory [data set], https://doi.org/10.26023/MX0D-Z722-M406, 2020. a, b
Nehrir, A. R., Repasky, K. S., Carlsten, J. L., Obland, M. D., and Shaw, J. A.: Water Vapor Profiling Using a Widely Tunable, Amplified Diode-Laser-Based Differential Absorption Lidar (DIAL), J. Atmos. Ocean.
Tech., 26, 733–745, https://doi.org/10.1175/2008JTECHA1201.1, 2009. a, b, c
Newsom, R. and Sivaraman, C.: Raman Lidar Water Vapor Mixing Ratio and
Temperature Value-Added Products, Tech. rep., DOE Office of Science
Atmospheric Radiation Measurement (ARM) Program (United States),
https://doi.org/10.2172/1489497, 2018. a, b
NRC: Observing Weather and Climate from the Ground Up, National Academies
Press, Washington, D.C., https://doi.org/10.17226/12540, 2009. a
NRC: When Weather Matters, National Academies Press, Washington, D.C.,
https://doi.org/10.17226/12888, 2010.
a
NRC: Weather Services for the Nation, National Academies Press, Washington,
D.C., https://doi.org/10.17226/13429, 2012. a
Oh, A. K., Harmany, Z. T., and Willett, R. M.: Logarithmic total variation regularization for cross-validation in photon-limited imaging, in: 2013 IEEE International Conference on Image Processing, 15–18 September 2013, Melbourne, VIC, Australia, 484–488, https://doi.org/10.1109/ICIP.2013.6738100, 2013. a, b, c, d, e, f, g
Oh, A. K., Harmany, Z. T., and Willett, R. M.: To e or not to e in poisson image reconstruction, in: 2014 IEEE International Conference on Image
Processing (ICIP), 27–30 October 2014, Paris, France, 2829–2833, https://doi.org/10.1109/ICIP.2014.7025572, 2014. a
Roelofs, F., Janssen, M., Natarajan, I., et al.: SYMBA: An end-to-end VLBI synthetic data generation pipeline-Simulating Event Horizon Telescope observations of M 87, Astron. Astrophys., 636, A5, https://doi.org/10.1051/0004-6361/201936622, 2020. a
Schafer, R. W.: On the frequency-domain properties of Savitzky-Golay filters,
in: 2011 Digital Signal Processing and Signal Processing Education Meeting
(DSP/SPE), 4–7 January 2011, Sedona, AZ, USA, 54–59, https://doi.org/10.1109/DSP-SPE.2011.5739186, 2011. a
Spuler, S. M., Hayman, M., Stillwell, R. A., Carnes, J., Bernatsky, T., and Repasky, K. S.: MicroPulse DIAL (MPD) – a diode-laser-based lidar architecture for quantitative atmospheric profiling, Atmos. Meas. Tech., 14, 4593–4616, https://doi.org/10.5194/amt-14-4593-2021, 2021. a, b, c, d, e, f, g, h
Weckwerth, T. M., Wulfmeyer, V., Wakimoto, R. M., Hardesty, M. R., Wilson, J. W., and Banta, R. M.: NCAR-NOAA Lower-Tropospheric Water Vapor Workshop,
B. Am. Meteorol. Soc., 80, 2339–2357,
https://doi.org/10.1175/1520-0477(1999)080<2331:WOTOFE>2.0.CO;2, 1999. a
Willett, R. and Nowak, R.: Platelets: a multiscale approach for recovering edges and surfaces in photon-limited medical imaging, IEEE T. Med. Imaging, 22, 332–350, https://doi.org/10.1109/TMI.2003.809622, 2003. a
Wulfmeyer, V., Hardesty, R. M., Turner, D. D., Behrendt, A., Cadeddu, M. P., Di Girolamo, P., Schlüssel, P., Van Baelen, J., and Zus, F.: A review of the remote sensing of lower tropospheric thermodynamic profiles and its indispensable role for the understanding and the simulation of water and energy cycles, Rev. Geophys., 53, 819–895, https://doi.org/10.1002/2014RG000476, 2015. a
Xiao, D., Wang, N., Shen, X., Landulfo, E., Zhong, T., and Liu, D.: Development of ZJU High-Spectral-Resolution Lidar for Aerosol and Cloud: Extinction Retrieval, Remote Sens., 12, 3047, https://doi.org/10.3390/rs12183047, 2020. a, b
Short summary
For atmospheric science and weather prediction, it is important to make water vapor measurements in real time. A low-cost lidar instrument has been developed by Montana State University and the National Center for Atmospheric Research. We developed an advanced signal-processing method to extend the scientific capability of the lidar instrument. With the new method we show that the maximum altitude at which the MPD can make water vapor measurements can be extended up to 8 km.
For atmospheric science and weather prediction, it is important to make water vapor measurements...