Articles | Volume 7, issue 12
https://doi.org/10.5194/amt-7-4387-2014
© Author(s) 2014. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
https://doi.org/10.5194/amt-7-4387-2014
© Author(s) 2014. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
Research article
| 
11 Dec 2014
Research article |
| 11 Dec 2014
Regression models tolerant to massively missing data: a case study in solar-radiation nowcasting
I. Žliobaitė
CORRESPONDING AUTHOR
Aalto University, Department of Information and Computer Science, Espoo, Finland
Helsinki Institute for Information Technology (HIIT), Helsinki, Finland
J. Hollmén
Aalto University, Department of Information and Computer Science, Espoo, Finland
Helsinki Institute for Information Technology (HIIT), Helsinki, Finland
H. Junninen
Department of Physics, University of Helsinki, Helsinki, Finland
Related authors
No articles found.
M. Sipilä, N. Sarnela, T. Jokinen, H. Junninen, J. Hakala, M. P. Rissanen, A. Praplan, M. Simon, A. Kürten, F. Bianchi, J. Dommen, J. Curtius, T. Petäjä, and D. R. Worsnop
Atmos. Meas. Tech., 8, 4001–4011, https://doi.org/10.5194/amt-8-4001-2015, https://doi.org/10.5194/amt-8-4001-2015, 2015
Short summary
Short summary
Atmospheric concentrations of amines are poorly known mainly due to challenges related to their reliable high-sensitivity detection. We have created a method and instrument that is capable for detecting amines with lowest limit of detection of around 0.01 parts per trillion. Application of the instrument in the field study indicates that concentrations of dimethyl amine in a boreal forest site are below 0.03ppt, not enough to account for the observed new particle formation rates.
A. Kürten, S. Münch, L. Rondo, F. Bianchi, J. Duplissy, T. Jokinen, H. Junninen, N. Sarnela, S. Schobesberger, M. Simon, M. Sipilä, J. Almeida, A. Amorim, J. Dommen, N. M. Donahue, E. M. Dunne, R. C. Flagan, A. Franchin, J. Kirkby, A. Kupc, V. Makhmutov, T. Petäjä, A. P. Praplan, F. Riccobono, G. Steiner, A. Tomé, G. Tsagkogeorgas, P. E. Wagner, D. Wimmer, U. Baltensperger, M. Kulmala, D. R. Worsnop, and J. Curtius
Atmos. Chem. Phys., 15, 10701–10721, https://doi.org/10.5194/acp-15-10701-2015, https://doi.org/10.5194/acp-15-10701-2015, 2015
Short summary
Short summary
New particle formation (NPF) is an important atmospheric process. At cold temperatures in the upper troposphere the binary (H2SO4-H2O) and ternary (H2SO4-H2O-NH3) system are thought to be important for NPF. Sulfuric acid monomer (H2SO4) and sulfuric acid dimer ((H2SO4)2) concentrations were measured between 208 and 248K for these systems and dimer evaporation rates were derived. These data will help to better understand and predict binary and ternary nucleation at low temperatures.
K. Ruusuvuori, P. Hietala, O. Kupiainen-Määttä, T. Jokinen, H. Junninen, M. Sipilä, T. Kurtén, and H. Vehkamäki
Atmos. Meas. Tech., 8, 2577–2588, https://doi.org/10.5194/amt-8-2577-2015, https://doi.org/10.5194/amt-8-2577-2015, 2015
Short summary
Short summary
Ionization reagents suitable for accurate measurements of the atmospheric gas-phase amine vapour concentrations are needed. Based on computational results, acetone is a viable option for use as an ionization reagent in CI-APi-TOF measurements on atmospheric dimethylamine. However, comparison between the computational and experimental results revealed notable discrepancies. Further study is still required before the acetone CI-APi-TOF can be considered a viable option in practice.
A. P. Praplan, S. Schobesberger, F. Bianchi, M. P. Rissanen, M. Ehn, T. Jokinen, H. Junninen, A. Adamov, A. Amorim, J. Dommen, J. Duplissy, J. Hakala, A. Hansel, M. Heinritzi, J. Kangasluoma, J. Kirkby, M. Krapf, A. Kürten, K. Lehtipalo, F. Riccobono, L. Rondo, N. Sarnela, M. Simon, A. Tomé, J. Tröstl, P. M. Winkler, C. Williamson, P. Ye, J. Curtius, U. Baltensperger, N. M. Donahue, M. Kulmala, and D. R. Worsnop
Atmos. Chem. Phys., 15, 4145–4159, https://doi.org/10.5194/acp-15-4145-2015, https://doi.org/10.5194/acp-15-4145-2015, 2015
Short summary
Short summary
Our study shows, based on data from three atmospheric pressure interface time-of-flight mass spectrometers measuring in parallel charged and neutral molecules and molecular clusters, how oxidised organic compounds bind to inorganic ions (e.g. bisulfate, nitrate, ammonium). This ionisation is selective for compounds with lower molar mass due to their limited amount and variety of functional groups. We also found that extremely low volatile organic compounds (ELVOCs) can be formed immediately.
C. Rose, K. Sellegri, E. Asmi, M. Hervo, E. Freney, A. Colomb, H. Junninen, J. Duplissy, M. Sipilä, J. Kontkanen, K. Lehtipalo, and M. Kulmala
Atmos. Chem. Phys., 15, 3413–3428, https://doi.org/10.5194/acp-15-3413-2015, https://doi.org/10.5194/acp-15-3413-2015, 2015
S. Schobesberger, A. Franchin, F. Bianchi, L. Rondo, J. Duplissy, A. Kürten, I. K. Ortega, A. Metzger, R. Schnitzhofer, J. Almeida, A. Amorim, J. Dommen, E. M. Dunne, M. Ehn, S. Gagné, L. Ickes, H. Junninen, A. Hansel, V.-M. Kerminen, J. Kirkby, A. Kupc, A. Laaksonen, K. Lehtipalo, S. Mathot, A. Onnela, T. Petäjä, F. Riccobono, F. D. Santos, M. Sipilä, A. Tomé, G. Tsagkogeorgas, Y. Viisanen, P. E. Wagner, D. Wimmer, J. Curtius, N. M. Donahue, U. Baltensperger, M. Kulmala, and D. R. Worsnop
Atmos. Chem. Phys., 15, 55–78, https://doi.org/10.5194/acp-15-55-2015, https://doi.org/10.5194/acp-15-55-2015, 2015
Short summary
Short summary
We used an ion mass spectrometer at CERN's CLOUD chamber to investigate the detailed composition of ammonia--sulfuric acid ion clusters (of both polarities) as they initially form and then grow into aerosol particles, at atmospherically relevant conditions. We found that these clusters’ composition is mainly determined by the ratio of the precursor vapors and ranges from ammonia-free clusters to clusters containing > 1 ammonia per sulfuric acid. Acid--base bindings are a key formation mechanism.
M. Sipilä, T. Jokinen, T. Berndt, S. Richters, R. Makkonen, N. M. Donahue, R. L. Mauldin III, T. Kurtén, P. Paasonen, N. Sarnela, M. Ehn, H. Junninen, M. P. Rissanen, J. Thornton, F. Stratmann, H. Herrmann, D. R. Worsnop, M. Kulmala, V.-M. Kerminen, and T. Petäjä
Atmos. Chem. Phys., 14, 12143–12153, https://doi.org/10.5194/acp-14-12143-2014, https://doi.org/10.5194/acp-14-12143-2014, 2014
L. Rondo, A. Kürten, S. Ehrhart, S. Schobesberger, A. Franchin, H. Junninen, T. Petäjä, M. Sipilä, D. R. Worsnop, and J. Curtius
Atmos. Meas. Tech., 7, 3849–3859, https://doi.org/10.5194/amt-7-3849-2014, https://doi.org/10.5194/amt-7-3849-2014, 2014
M. Crippa, F. Canonaco, V. A. Lanz, M. Äijälä, J. D. Allan, S. Carbone, G. Capes, D. Ceburnis, M. Dall'Osto, D. A. Day, P. F. DeCarlo, M. Ehn, A. Eriksson, E. Freney, L. Hildebrandt Ruiz, R. Hillamo, J. L. Jimenez, H. Junninen, A. Kiendler-Scharr, A.-M. Kortelainen, M. Kulmala, A. Laaksonen, A. A. Mensah, C. Mohr, E. Nemitz, C. O'Dowd, J. Ovadnevaite, S. N. Pandis, T. Petäjä, L. Poulain, S. Saarikoski, K. Sellegri, E. Swietlicki, P. Tiitta, D. R. Worsnop, U. Baltensperger, and A. S. H. Prévôt
Atmos. Chem. Phys., 14, 6159–6176, https://doi.org/10.5194/acp-14-6159-2014, https://doi.org/10.5194/acp-14-6159-2014, 2014
Related subject area
Subject: Others (Wind, Precipitation, Temperature, etc.) | Technique: Remote Sensing | Topic: Data Processing and Information Retrieval
Global sensitivity analysis of simulated remote sensing polarimetric observations over snow
Improving the Gaussianity of radar reflectivity departures between observations and simulations using symmetric rain rates
On the temperature stability requirements of free-running Nd:YAG lasers for atmospheric temperature profiling through the rotational Raman technique
Limitations in wavelet analysis of non-stationary atmospheric gravity wave signatures in temperature profiles
A new non-linearity correction method for the spectrum from the Geostationary Inferometric Infrared Sounder on board Fengyun-4 satellites and its preliminary assessments
Determination of high-precision tropospheric delays using crowdsourced smartphone GNSS data
Unfiltering of the EarthCARE Broadband Radiometer (BBR) observations: the BM-RAD product
Variance estimations in the presence of intermittent interference and their applications to incoherent scatter radar signal processing
A clustering-based method for identifying and tracking squall lines
A multi-instrument fuzzy logic boundary-layer-top detection algorithm
Sensitivity of thermodynamic profiles retrieved from ground-based microwave and infrared observations to additional input data from active remote sensing instruments and numerical weather prediction models
Scale separation for gravity wave analysis from 3D temperature observations in the mesosphere and lower thermosphere (MLT) region
Estimating the refractivity bias of FORMOSAT-7/COSMIC-2 Global Navigation Satellite System (GNSS) radio occultation in the deep troposphere
High Spectral Resolution Lidar – generation 2 (HSRL-2) retrievals of ocean surface wind speed: methodology and evaluation
Dual adaptive differential threshold method for automated detection of faint and strong echo features in radar observations of winter storms
Noise filtering options for conically scanning Doppler lidar measurements with low pulse accumulation
Measuring rainfall using microwave links: the influence of temporal sampling
Drone-based photogrammetry combined with deep learning to estimate hail size distributions and melting of hail on the ground
Global scale gravity wave analysis methodology for the ESA Earth Explorer 11 candidate CAIRT
The High lAtitude sNowfall Detection and Estimation aLgorithm for ATMS (HANDEL-ATMS): a new algorithm for snowfall retrieval at high latitudes
Next-generation radiance unfiltering process for the Clouds and the Earth's Radiant Energy System instrument
Improved rain event detection in commercial microwave link time series via combination with MSG SEVIRI data
A directional surface reflectance climatology determined from TROPOMI observations
Investigation of gravity waves using measurements from a sodium temperature/wind lidar operated in multi-direction mode
Retrieval pseudo BRDF-adjusted surface reflectance at 440 nm from Geostationary Environmental Monitoring Spectrometer (GEMS)
An improved BRDF hotspot model and its use in VLIDORT for studying the impact of atmospheric scattering on hotspot directional signatures in the atmosphere
A multi-decadal time series of upper stratospheric temperature profiles from Odin-OSIRIS limb-scattered spectra
Thermal tides in the middle atmosphere at mid-latitudes measured with a ground-based microwave Radiometer
CALOTRITON: a convective boundary layer height estimation algorithm from ultra-high-frequency (UHF) wind profiler data
Enhancing consistency of microphysical properties of precipitation across the melting layer in dual-frequency precipitation radar data
Development of a HAMSTER: Hyperspectral Albedo Maps dataset with high Spatial and TEmporal Resolution
Profiling the molecular destruction rates of temperature and humidity as well as the turbulent kinetic energy dissipation in the convective boundary layer
Forward operator for polarimetric radio occultation measurements
Assessing atmospheric gravity wave spectra in the presence of observational gaps
Joint 1DVar retrievals of tropospheric temperature and water vapor from Global Navigation Satellite System radio occultation (GNSS-RO) and microwave radiometer observations
Mispointing characterization and Doppler velocity correction for the conically scanning WIVERN Doppler radar
Radar and environment-based hail damage estimates using machine learning
A new power-law model for μ–Λ relationships in convective and stratiform rainfall
Suppression of precipitation bias in wind velocities from continuous-wave Doppler lidars
Difference spectrum fitting of the ion–neutral collision frequency from dual-frequency EISCAT measurements
Performance evaluation of three bio-optical models in aerosol and ocean color joint retrievals
Observation of horizontal temperature variations by a spatial heterodyne interferometer using single-sided interferograms
Drop Size Distribution Retrieval Using Dual Frequency Polarimetric Weather Radars
Version 8 IMK–IAA MIPAS temperatures from 12–15 µm spectra: Middle and Upper Atmosphere modes
GNSS radio occultation excess-phase processing for climate applications including uncertainty estimation
Impact analysis of processing strategies for long-term GPS zenith tropospheric delay (ZTD)
Irradiance and cloud optical properties from solar photovoltaic systems
Single field-of-view sounder atmospheric product retrieval algorithm: establishing radiometric consistency for hyper-spectral sounder retrievals
Higher-order calibration on WindRAD (Wind Radar) scatterometer winds
On the polarimetric backscatter by a still or quasi-still wind turbine
Matteo Ottaviani, Gabriel Harris Myers, and Nan Chen
Atmos. Meas. Tech., 17, 4737–4756, https://doi.org/10.5194/amt-17-4737-2024, https://doi.org/10.5194/amt-17-4737-2024, 2024
Short summary
Short summary
We analyze simulated polarization observations over snow to investigate the capabilities of remote sensing to determine surface and atmospheric properties in snow-covered regions. Polarization measurements are demonstrated to aid in the determination of snow grain shape, ice crystal roughness, and the vertical distribution of impurities in the snow–atmosphere system, data that are critical for estimating snow albedo for use in climate models.
Yudong Gao, Lidou Huyan, Zheng Wu, and Bojun Liu
Atmos. Meas. Tech., 17, 4675–4686, https://doi.org/10.5194/amt-17-4675-2024, https://doi.org/10.5194/amt-17-4675-2024, 2024
Short summary
Short summary
A symmetric error model built by symmetric rain rates handles the non-Gaussian error structure of the reflectivity error. The accuracy and linearization of rain rates can further improve the Gaussianity.
José Alex Zenteno-Hernández, Adolfo Comerón, Federico Dios, Alejandro Rodríguez-Gómez, Constantino Muñoz-Porcar, Michaël Sicard, Noemi Franco, Andreas Behrendt, and Paolo Di Girolamo
Atmos. Meas. Tech., 17, 4687–4694, https://doi.org/10.5194/amt-17-4687-2024, https://doi.org/10.5194/amt-17-4687-2024, 2024
Short summary
Short summary
We study how the spectral characteristics of a solid-state laser in an atmospheric temperature profiling lidar using the Raman technique impact the temperature retrieval accuracy. We find that the spectral widening, with respect to a seeded laser, has virtually no impact, while crystal-rod temperature variations in the laser must be kept within a range of 1 K for the uncertainty in the atmospheric temperature below 1 K. The study is carried out through spectroscopy simulations.
Robert Reichert, Natalie Kaifler, and Bernd Kaifler
Atmos. Meas. Tech., 17, 4659–4673, https://doi.org/10.5194/amt-17-4659-2024, https://doi.org/10.5194/amt-17-4659-2024, 2024
Short summary
Short summary
Imagine you want to determine how quickly the pitch of a passing ambulance’s siren changes. If the vehicle is traveling slowly, the pitch changes only slightly, but if it is traveling fast, the pitch also changes rapidly. In a similar way, the wind in the middle atmosphere modulates the wavelength of atmospheric gravity waves. We have investigated the question of how strong the maximum wind may be so that the change in wavelength can still be determined with the help of wavelet transformation.
Qiang Guo, Yuning Liu, Xin Wang, and Wen Hui
Atmos. Meas. Tech., 17, 4613–4627, https://doi.org/10.5194/amt-17-4613-2024, https://doi.org/10.5194/amt-17-4613-2024, 2024
Short summary
Short summary
Non-linearity (NL) correction is a critical procedure to guarantee that the calibration accuracy of a spaceborne sensor approaches a reasonable level. Different from the classical method, a new NL correction method for a spaceborne Fourier transform spectrometer is proposed. To overcome the inaccurate linear coefficient from two-point calibration influencing NL correction, an iteration algorithm is established that is suitable for NL correction of both infrared and microwave sensors.
Yuanxin Pan, Grzegorz Kłopotek, Laura Crocetti, Rudi Weinacker, Tobias Sturn, Linda See, Galina Dick, Gregor Möller, Markus Rothacher, Ian McCallum, Vicente Navarro, and Benedikt Soja
Atmos. Meas. Tech., 17, 4303–4316, https://doi.org/10.5194/amt-17-4303-2024, https://doi.org/10.5194/amt-17-4303-2024, 2024
Short summary
Short summary
Crowdsourced smartphone GNSS data were processed with a dedicated data processing pipeline and could produce millimeter-level accurate estimates of zenith total delay (ZTD) – a critical atmospheric variable. This breakthrough not only demonstrates the feasibility of using ubiquitous devices for high-precision atmospheric monitoring but also underscores the potential for a global, cost-effective tropospheric monitoring network.
Almudena Velázquez Blázquez, Edward Baudrez, Nicolas Clerbaux, and Carlos Domenech
Atmos. Meas. Tech., 17, 4245–4256, https://doi.org/10.5194/amt-17-4245-2024, https://doi.org/10.5194/amt-17-4245-2024, 2024
Short summary
Short summary
The Broadband Radiometer measures shortwave and total-wave radiances filtered by the spectral response of the instrument. To obtain unfiltered solar and thermal radiances, the effect of the spectral response needs to be corrected for, done within the BM-RAD processor. Errors in the unfiltering are propagated into fluxes; thus, accurate unfiltering is required for their proper estimation (within BMA-FLX). Unfiltering errors are estimated to be <0.5 % for the shortwave and <0.1 % for the longwave.
Qihou Zhou, Yanlin Li, and Yun Gong
Atmos. Meas. Tech., 17, 4197–4209, https://doi.org/10.5194/amt-17-4197-2024, https://doi.org/10.5194/amt-17-4197-2024, 2024
Short summary
Short summary
We discuss several robust estimators to compute the variance of a normally distributed random variable to deal with interference. Compared to rank-based estimators, the methods based on the geometric mean are more accurate and are computationally more efficient. We apply three robust estimators to incoherent scatter power and velocity processing, along with the traditional sample mean estimator. The best estimator is a hybrid estimator that combines the sample mean and a robust estimator.
Zhao Shi, Yuxiang Wen, and Jianxin He
Atmos. Meas. Tech., 17, 4121–4135, https://doi.org/10.5194/amt-17-4121-2024, https://doi.org/10.5194/amt-17-4121-2024, 2024
Short summary
Short summary
The squall line is a type of convective system. Squall lines are often associated with damaging weather, so identifying and tracking squall lines plays an important role in early meteorological disaster warnings. A clustering-based method is proposed in this article. It can identify the squall lines within the radar scanning range with an accuracy rate of 95.93 %. It can also provide the three-dimensional structure and movement tracking results for each squall line.
Elizabeth N. Smith and Jacob T. Carlin
Atmos. Meas. Tech., 17, 4087–4107, https://doi.org/10.5194/amt-17-4087-2024, https://doi.org/10.5194/amt-17-4087-2024, 2024
Short summary
Short summary
Boundary-layer height observations remain sparse in time and space. In this study we create a new fuzzy logic method for synergistically combining boundary-layer height estimates from a suite of instruments. These estimates generally compare well to those from radiosondes; plus, the approach offers near-continuous estimates through the entire diurnal cycle. Suspected reasons for discrepancies are discussed. The code for the newly presented fuzzy logic method is provided for the community to use.
Laura Bianco, Bianca Adler, Ludovic Bariteau, Irina V. Djalalova, Timothy Myers, Sergio Pezoa, David D. Turner, and James M. Wilczak
Atmos. Meas. Tech., 17, 3933–3948, https://doi.org/10.5194/amt-17-3933-2024, https://doi.org/10.5194/amt-17-3933-2024, 2024
Short summary
Short summary
The Tropospheric Remotely Observed Profiling via Optimal Estimation physical retrieval is used to retrieve temperature and humidity profiles from various combinations of passive and active remote sensing instruments, surface platforms, and numerical weather prediction models. The retrieved profiles are assessed against collocated radiosonde in non-cloudy conditions to assess the sensitivity of the retrievals to different input combinations. Case studies with cloudy conditions are also inspected.
Björn Linder, Peter Preusse, Qiuyu Chen, Ole Martin Christensen, Lukas Krasauskas, Linda Megner, Manfred Ern, and Jörg Gumbel
Atmos. Meas. Tech., 17, 3829–3841, https://doi.org/10.5194/amt-17-3829-2024, https://doi.org/10.5194/amt-17-3829-2024, 2024
Short summary
Short summary
The Swedish research satellite MATS (Mesospheric Airglow/Aerosol Tomography and Spectroscopy) is designed to study atmospheric waves in the mesosphere and lower thermosphere. These waves perturb the temperature field, and thus, by observing three-dimensional temperature fluctuations, their properties can be quantified. This pre-study uses synthetic MATS data generated from a general circulation model to investigate how well wave properties can be retrieved.
Gia Huan Pham, Shu-Chih Yang, Chih-Chien Chang, Shu-Ya Chen, and Cheng Yung Huang
Atmos. Meas. Tech., 17, 3605–3623, https://doi.org/10.5194/amt-17-3605-2024, https://doi.org/10.5194/amt-17-3605-2024, 2024
Short summary
Short summary
This research examines the characteristics of low-level GNSS radio occultation (RO) refractivity bias over ocean and land and its dependency on the RO retrieval uncertainty, atmospheric temperature, and moisture. We propose methods for estimating the region-dependent refractivity bias. Our methods can be applied to calibrate the refractivity bias under different atmospheric conditions and thus improve the applications of the GNSS RO data in the deep troposphere.
Sanja Dmitrovic, Johnathan W. Hair, Brian L. Collister, Ewan Crosbie, Marta A. Fenn, Richard A. Ferrare, David B. Harper, Chris A. Hostetler, Yongxiang Hu, John A. Reagan, Claire E. Robinson, Shane T. Seaman, Taylor J. Shingler, Kenneth L. Thornhill, Holger Vömel, Xubin Zeng, and Armin Sorooshian
Atmos. Meas. Tech., 17, 3515–3532, https://doi.org/10.5194/amt-17-3515-2024, https://doi.org/10.5194/amt-17-3515-2024, 2024
Short summary
Short summary
This study introduces and evaluates a new ocean surface wind speed product from the NASA Langley Research Center (LARC) airborne High-Spectral-Resolution Lidar – Generation 2 (HSRL-2) during the NASA ACTIVATE mission. We show that HSRL-2 surface wind speed data are accurate when compared to ground-truth dropsonde measurements. Therefore, the HSRL-2 instrument is able obtain accurate, high-resolution surface wind speed data in airborne field campaigns.
Laura M. Tomkins, Sandra E. Yuter, and Matthew A. Miller
Atmos. Meas. Tech., 17, 3377–3399, https://doi.org/10.5194/amt-17-3377-2024, https://doi.org/10.5194/amt-17-3377-2024, 2024
Short summary
Short summary
We have created a new method to better identify enhanced features in radar data from winter storms. Unlike the clear-cut features seen in warm-season storms, features in winter storms are often fuzzier with softer edges. Our technique is unique because it uses two adaptive thresholds that change based on the background radar values. It can identify both strong and subtle features in the radar data and takes into account uncertainties in the detection process.
Eileen Päschke and Carola Detring
Atmos. Meas. Tech., 17, 3187–3217, https://doi.org/10.5194/amt-17-3187-2024, https://doi.org/10.5194/amt-17-3187-2024, 2024
Short summary
Short summary
Little noise in radial velocity Doppler lidar measurements can contribute to large errors in retrieved turbulence variables. In order to distinguish between plausible and erroneous measurements we developed new filter techniques that work independently of the choice of a specific threshold for the signal-to-noise ratio. The performance of these techniques is discussed both by means of assessing the filter results and by comparing retrieved turbulence variables versus independent measurements.
Luuk D. van der Valk, Miriam Coenders-Gerrits, Rolf W. Hut, Aart Overeem, Bas Walraven, and Remko Uijlenhoet
Atmos. Meas. Tech., 17, 2811–2832, https://doi.org/10.5194/amt-17-2811-2024, https://doi.org/10.5194/amt-17-2811-2024, 2024
Short summary
Short summary
Microwave links, often part of mobile phone networks, can be used to measure rainfall along the link path by determining the signal loss caused by rainfall. We use high-frequency data of multiple microwave links to recreate commonly used sampling strategies. For time intervals up to 1 min, the influence of sampling strategies on estimated rainfall intensities is relatively little, while for intervals longer than 5–15 min, the sampling strategy can have significant influences on the estimates.
Martin Lainer, Killian P. Brennan, Alessandro Hering, Jérôme Kopp, Samuel Monhart, Daniel Wolfensberger, and Urs Germann
Atmos. Meas. Tech., 17, 2539–2557, https://doi.org/10.5194/amt-17-2539-2024, https://doi.org/10.5194/amt-17-2539-2024, 2024
Short summary
Short summary
This study uses deep learning (the Mask R-CNN model) on drone-based photogrammetric data of hail on the ground to estimate hail size distributions (HSDs). Traditional hail sensors' limited areas complicate the full HSD retrieval. The HSD of a supercell event on 20 June 2021 is retrieved and contains > 18 000 hailstones. The HSD is compared to automatic hail sensor measurements and those of weather-radar-based MESHS. Investigations into ground hail melting are performed by five drone flights.
Sebastian Rhode, Peter Preusse, Jörn Ungermann, Inna Polichtchouk, Kaoru Sato, Shingo Watanabe, Manfred Ern, Karlheinz Nogai, Björn-Martin Sinnhuber, and Martin Riese
EGUsphere, https://doi.org/10.5194/egusphere-2024-1084, https://doi.org/10.5194/egusphere-2024-1084, 2024
Short summary
Short summary
We investigate the capabilities of a proposed satellite mission, CAIRT, for observing gravity waves throughout the middle atmosphere and present the necessary methodology for in-depth wave analysis. Our findings suggest that such a satellite mission is highly capable of resolving individual wave parameters and could give new insights into the role of gravity waves in the general atmospheric circulation and atmospheric processes.
Andrea Camplani, Daniele Casella, Paolo Sanò, and Giulia Panegrossi
Atmos. Meas. Tech., 17, 2195–2217, https://doi.org/10.5194/amt-17-2195-2024, https://doi.org/10.5194/amt-17-2195-2024, 2024
Short summary
Short summary
The paper describes a new machine-learning-based snowfall retrieval algorithm for Advanced Technology Microwave Sounder observations developed to retrieve high-latitude snowfall events. The main novelty of the approach is the radiometric characterization of the background surface at the time of the overpass, which is ancillary to the retrieval process. The algorithm shows a unique capability to retrieve snowfall in the environmental conditions typical of high latitudes.
Lusheng Liang, Wenying Su, Sergio Sejas, Zachary Eitzen, and Norman G. Loeb
Atmos. Meas. Tech., 17, 2147–2163, https://doi.org/10.5194/amt-17-2147-2024, https://doi.org/10.5194/amt-17-2147-2024, 2024
Short summary
Short summary
This paper describes an updated process to obtain unfiltered radiation from CERES satellite instruments by incorporating the most recent developments in radiative transfer modeling and ancillary input datasets (e.g., realistic representation of land surface radiation and climatology of surface temperatures and aerosols) during the past 20 years. The resulting global mean of instantaneous SW and LW fluxes is changed by less than 0.5 W m−2 with regional differences as large as 2.0 W m−2.
Maximilian Graf, Andreas Wagner, Julius Polz, Llorenç Lliso, José Alberto Lahuerta, Harald Kunstmann, and Christian Chwala
Atmos. Meas. Tech., 17, 2165–2182, https://doi.org/10.5194/amt-17-2165-2024, https://doi.org/10.5194/amt-17-2165-2024, 2024
Short summary
Short summary
Commercial microwave links (CMLs) can be used for rainfall retrieval. The detection of rainy periods in their attenuation time series is a crucial processing step. We investigate the usage of rainfall data from MSG SEVIRI for this task, compare this approach with existing methods, and introduce a novel combined approach. The results show certain advantages for SEVIRI-based methods, particularly for CMLs where existing methods perform poorly. Our novel combination yields the best performance.
Lieuwe G. Tilstra, Martin de Graaf, Victor J. H. Trees, Pavel Litvinov, Oleg Dubovik, and Piet Stammes
Atmos. Meas. Tech., 17, 2235–2256, https://doi.org/10.5194/amt-17-2235-2024, https://doi.org/10.5194/amt-17-2235-2024, 2024
Short summary
Short summary
This paper introduces a new surface albedo climatology of directionally dependent Lambertian-equivalent reflectivity (DLER) observed by TROPOMI on the Sentinel-5 Precursor satellite. The database contains monthly fields of DLER for 21 wavelength bands at a relatively high spatial resolution of 0.125 by 0.125 degrees. The anisotropy of the surface reflection is handled by parameterisation of the viewing angle dependence.
Bing Cao and Alan Z. Liu
Atmos. Meas. Tech., 17, 2123–2146, https://doi.org/10.5194/amt-17-2123-2024, https://doi.org/10.5194/amt-17-2123-2024, 2024
Short summary
Short summary
A narrow-band sodium lidar measures atmospheric waves but is limited to vertical variations. We propose to utilize phase shifts among observations from different laser beams to derive horizontal wave information. Two gravity wave packets were identified by this method. Both waves were found to interact with thin evanescent layers, partially reflected, but transmitted energy to higher altitudes. The method can detect more medium-frequency gravity waves for similar lidar systems worldwide.
Suyoung Sim, Sungwon Choi, Daeseong Jung, Jongho Woo, Nayeon Kim, Sungwoo Park, Honghee Kim, Ukkyo Jeong, Hyunkee Hong, and Kyung-Soo Han
EGUsphere, https://doi.org/10.5194/egusphere-2024-601, https://doi.org/10.5194/egusphere-2024-601, 2024
Short summary
Short summary
Our study presents a novel method for satellite-based surface reflectance estimation, using the bi-directional Reflectance Distribution Function (BRDF) model to derive Background Surface Reflectance (BSR) in UV-VIS hyperspectral satellite imagery. Through comprehensive analysis, we show that BSR offers higher accuracy and greater stability compared to Lambertian Equivalent Reflectance (LER) methods. This data can offer a promising tool for accurate climate analysis and air quality monitoring.
Xiaozhen Xiong, Xu Liu, Robert Spurr, Ming Zhao, Qiguang Yang, Wan Wu, and Liqiao Lei
Atmos. Meas. Tech., 17, 1965–1978, https://doi.org/10.5194/amt-17-1965-2024, https://doi.org/10.5194/amt-17-1965-2024, 2024
Short summary
Short summary
The term “hotspot” refers to the sharp increase in reflectance occurring when incident (solar) and reflected (viewing) directions coincide in the backscatter direction. The accurate simulation of hotspot directional signatures is important for many remote sensing applications, but current models typically require large values of computations to represent the hotspot accurately. This paper provides a numerically improved hotspot BRDF model that converges much faster and is used in VLIDORT.
Daniel Zawada, Kimberlee Dubé, Taran Warnock, Adam Bourassa, Susann Tegtmeier, and Douglas Degenstein
Atmos. Meas. Tech., 17, 1995–2010, https://doi.org/10.5194/amt-17-1995-2024, https://doi.org/10.5194/amt-17-1995-2024, 2024
Short summary
Short summary
There remain large uncertainties in long-term changes of stratospheric–atmospheric temperatures. We have produced a time series of more than 20 years of satellite-based temperature measurements from the OSIRIS instrument in the upper–middle stratosphere. The dataset is publicly available and intended to be used for a better understanding of changes in stratospheric temperatures.
Witali Krochin, Axel Murk, and Gunter Stober
Atmos. Meas. Tech. Discuss., https://doi.org/10.5194/amt-2024-42, https://doi.org/10.5194/amt-2024-42, 2024
Revised manuscript accepted for AMT
Short summary
Short summary
Atmospheric tides are global-scale oscillations with periods of a fraction of a day. Their observation in the middle atmosphere is challenging and rare, as it requires continuous measurements with a high temporal resolution. In this manuscript, temperature time series of a ground-based microwave radiometer were analyzed with a spectral filter to derive thermal tide amplitudes and phases in an altitude range of 20–50 km at the geographical location of Payerne (Switzerland).
Alban Philibert, Marie Lothon, Julien Amestoy, Pierre-Yves Meslin, Solène Derrien, Yannick Bezombes, Bernard Campistron, Fabienne Lohou, Antoine Vial, Guylaine Canut-Rocafort, Joachim Reuder, and Jennifer K. Brooke
Atmos. Meas. Tech., 17, 1679–1701, https://doi.org/10.5194/amt-17-1679-2024, https://doi.org/10.5194/amt-17-1679-2024, 2024
Short summary
Short summary
We present a new algorithm, CALOTRITON, for the retrieval of the convective boundary layer depth with ultra-high-frequency radar measurements. CALOTRITON is partly based on the principle that the top of the convective boundary layer is associated with an inversion and a decrease in turbulence. It is evaluated using ceilometer and radiosonde data. It is able to qualify the complexity of the vertical structure of the low troposphere and detect internal or residual layers.
Kamil Mroz, Alessandro Battaglia, and Ann M. Fridlind
Atmos. Meas. Tech., 17, 1577–1597, https://doi.org/10.5194/amt-17-1577-2024, https://doi.org/10.5194/amt-17-1577-2024, 2024
Short summary
Short summary
In this study, we examine the extent to which radar measurements from space can inform us about the properties of clouds and precipitation. Surprisingly, our analysis showed that the amount of ice turning into rain was lower than expected in the current product. To improve on this, we came up with a new way to extract information about the size and concentration of particles from radar data. As long as we use this method in the right conditions, we can even estimate how dense the ice is.
Giulia Roccetti, Luca Bugliaro, Felix Gödde, Claudia Emde, Ulrich Hamann, Mihail Manev, Michael Fritz Sterzik, and Cedric Wehrum
EGUsphere, https://doi.org/10.5194/egusphere-2024-167, https://doi.org/10.5194/egusphere-2024-167, 2024
Short summary
Short summary
The amount of sunlight reflected by Earth’s surface (albedo) is crucial for its radiative system. Satellite instruments offer detailed spatial and temporal albedo maps, but only in seven specific wavelength bands. We generate albedo maps that fully cover the visible and near-infrared range with a machine learning algorithm. These provide information about how the reflectivity of different land surfaces vary through the year. Our dataset enhances the understanding of Earth's energy balance.
Volker Wulfmeyer, Christoph Senff, Florian Späth, Andreas Behrendt, Diego Lange, Robert M. Banta, W. Alan Brewer, Andreas Wieser, and David D. Turner
Atmos. Meas. Tech., 17, 1175–1196, https://doi.org/10.5194/amt-17-1175-2024, https://doi.org/10.5194/amt-17-1175-2024, 2024
Short summary
Short summary
A simultaneous deployment of Doppler, temperature, and water-vapor lidar systems is used to provide profiles of molecular destruction rates and turbulent kinetic energy (TKE) dissipation in the convective boundary layer (CBL). The results can be used for the parameterization of turbulent variables, TKE budget analyses, and the verification of weather forecast and climate models.
Daisuke Hotta, Katrin Lonitz, and Sean Healy
Atmos. Meas. Tech., 17, 1075–1089, https://doi.org/10.5194/amt-17-1075-2024, https://doi.org/10.5194/amt-17-1075-2024, 2024
Short summary
Short summary
Global Navigation Satellite System (GNSS) polarimetric radio occultation (PRO) is a new type of GNSS observations that can detect heavy precipitation along the ray path between the emitter and receiver satellites. As a first step towards using these observations in numerical weather prediction (NWP), we developed a computer code that simulates GNSS-PRO observations from forecast fields produced by an NWP model. The quality of the developed simulator is evaluated with a number of case studies.
Mohamed Mossad, Irina Strelnikova, Robin Wing, and Gerd Baumgarten
Atmos. Meas. Tech., 17, 783–799, https://doi.org/10.5194/amt-17-783-2024, https://doi.org/10.5194/amt-17-783-2024, 2024
Short summary
Short summary
This numerical study addresses observational gaps' impact on atmospheric gravity wave spectra. Three methods, fast Fourier transform (FFT), generalized Lomb–Scargle periodogram (GLS), and Haar structure function (HSF), were tested on synthetic data. HSF is best for spectra with negative slopes. GLS excels for flat and positive slopes and identifying dominant frequencies. Accurately estimating these aspects is crucial for understanding gravity wave dynamics and energy transfer in the atmosphere.
Kuo-Nung Wang, Chi O. Ao, Mary G. Morris, George A. Hajj, Marcin J. Kurowski, Francis J. Turk, and Angelyn W. Moore
Atmos. Meas. Tech., 17, 583–599, https://doi.org/10.5194/amt-17-583-2024, https://doi.org/10.5194/amt-17-583-2024, 2024
Short summary
Short summary
In this article, we described a joint retrieval approach combining two techniques, RO and MWR, to obtain high vertical resolution and solve for temperature and moisture independently. The results show that the complicated structure in the lower troposphere can be better resolved with much smaller biases, and the RO+MWR combination is the most stable scenario in our sensitivity analysis. This approach is also applied to real data (COSMIC-2/Suomi-NPP) to show the promise of joint RO+MWR retrieval.
Filippo Emilio Scarsi, Alessandro Battaglia, Frederic Tridon, Paolo Martire, Ranvir Dhillon, and Anthony Illingworth
Atmos. Meas. Tech., 17, 499–514, https://doi.org/10.5194/amt-17-499-2024, https://doi.org/10.5194/amt-17-499-2024, 2024
Short summary
Short summary
The WIVERN mission, one of the two candidates to be the ESA's Earth Explorer 11 mission, aims at providing measurements of horizontal winds in cloud and precipitation systems through a conically scanning W-band Doppler radar. This work discusses four methods that can be used to characterize and correct the Doppler velocity error induced by the antenna mispointing. The proposed methodologies can be extended to other Doppler concepts featuring conically scanning or slant viewing Doppler systems.
Luis Ackermann, Joshua Soderholm, Alain Protat, Rhys Whitley, Lisa Ye, and Nina Ridder
Atmos. Meas. Tech., 17, 407–422, https://doi.org/10.5194/amt-17-407-2024, https://doi.org/10.5194/amt-17-407-2024, 2024
Short summary
Short summary
The paper addresses the crucial topic of hail damage quantification using radar observations. We propose a new radar-derived hail product that utilizes a large dataset of insurance hail damage claims and radar observations. A deep neural network was employed, trained with local meteorological variables and the radar observations, to better quantify hail damage. Key meteorological variables were identified to have the most predictive capability in this regard.
Christos Gatidis, Marc Schleiss, and Christine Unal
Atmos. Meas. Tech., 17, 235–245, https://doi.org/10.5194/amt-17-235-2024, https://doi.org/10.5194/amt-17-235-2024, 2024
Short summary
Short summary
A common method to retrieve important information about the microphysical structure of rain (DSD retrievals) requires a constrained relationship between the drop size distribution parameters. The most widely accepted empirical relationship is between μ and Λ. The relationship shows variability across the different types of rainfall (convective or stratiform). The new proposed power-law model to represent the μ–Λ relation provides a better physical interpretation of the relationship coefficients.
Liqin Jin, Jakob Mann, Nikolas Angelou, and Mikael Sjöholm
Atmos. Meas. Tech., 16, 6007–6023, https://doi.org/10.5194/amt-16-6007-2023, https://doi.org/10.5194/amt-16-6007-2023, 2023
Short summary
Short summary
By sampling the spectra from continuous-wave Doppler lidars very fast, the rain-induced Doppler signal can be suppressed and the bias in the wind velocity estimation can be reduced. The method normalizes 3 kHz spectra by their peak values before averaging them down to 50 Hz. Over 3 h, we observe a significant reduction in the bias of the lidar data relative to the reference sonic data when the largest lidar focus distance is used. The more it rains, the more the bias is reduced.
Florian Günzkofer, Gunter Stober, Dimitry Pokhotelov, Yasunobu Miyoshi, and Claudia Borries
Atmos. Meas. Tech., 16, 5897–5907, https://doi.org/10.5194/amt-16-5897-2023, https://doi.org/10.5194/amt-16-5897-2023, 2023
Short summary
Short summary
Electric currents in the ionosphere can impact both satellite and ground-based infrastructure. These currents depend strongly on the collisions of ions and neutral particles. Measuring ion–neutral collisions is often only possible via certain assumptions. The direct measurement of ion–neutral collision frequencies is possible with multifrequency incoherent scatter radar measurements. This paper presents one analysis method of such measurements and discusses its advantages and disadvantages.
Neranga K. Hannadige, Peng-Wang Zhai, Meng Gao, Yongxiang Hu, P. Jeremy Werdell, Kirk Knobelspiesse, and Brian Cairns
Atmos. Meas. Tech., 16, 5749–5770, https://doi.org/10.5194/amt-16-5749-2023, https://doi.org/10.5194/amt-16-5749-2023, 2023
Short summary
Short summary
We evaluated the impact of three ocean optical models with different numbers of free parameters on the performance of an aerosol and ocean color remote sensing algorithm using the multi-angle polarimeter (MAP) measurements. It was demonstrated that the three- and seven-parameter bio-optical models can be used to accurately represent both open and coastal waters, whereas the one-parameter model has smaller retrieval uncertainty over open water.
Konstantin Ntokas, Jörn Ungermann, Martin Kaufmann, Tom Neubert, and Martin Riese
Atmos. Meas. Tech., 16, 5681–5696, https://doi.org/10.5194/amt-16-5681-2023, https://doi.org/10.5194/amt-16-5681-2023, 2023
Short summary
Short summary
A nanosatellite was developed to obtain 1-D vertical temperature profiles in the mesosphere and lower thermosphere, which can be used to derive wave parameters needed for atmospheric models. A new processing method is shown, which allows one to extract two 1-D temperature profiles. The location of the two profiles is analyzed, as it is needed for deriving wave parameters. We show that this method is feasible, which however will increase the requirements of an accurate calibration and processing.
Daniel Durbin, Yadong Wang, and Pao-Liang Chang
EGUsphere, https://doi.org/10.5194/egusphere-2023-2220, https://doi.org/10.5194/egusphere-2023-2220, 2023
Short summary
Short summary
A method for determining Drop Size Distributions (DSDs) for rain using radar measurements from two frequencies at two polarizations is presented. Following some preprocessing and quality control, radar measurements are incorporated into a model which uses swarm intelligence to seek the most suitable DSD which would produce the input measures.
Maya García-Comas, Bernd Funke, Manuel López-Puertas, Norbert Glatthor, Udo Grabowski, Sylvia Kellmann, Michael Kiefer, Andrea Linden, Belén Martínez-Mondéjar, Gabriele P. Stiller, and Thomas von Clarmann
Atmos. Meas. Tech., 16, 5357–5386, https://doi.org/10.5194/amt-16-5357-2023, https://doi.org/10.5194/amt-16-5357-2023, 2023
Short summary
Short summary
We have released version 8 of MIPAS IMK–IAA temperatures and pointing information retrieved from MIPAS Middle and Upper Atmosphere mode version 8.03 calibrated spectra, covering 20–115 km altitude. We considered non-local thermodynamic equilibrium emission explicitly for each limb scan, essential to retrieve accurate temperatures above the mid-mesosphere. Comparisons of this temperature dataset with SABER measurements show excellent agreement, improving those of previous MIPAS versions.
Josef Innerkofler, Gottfried Kirchengast, Marc Schwärz, Christian Marquardt, and Yago Andres
Atmos. Meas. Tech., 16, 5217–5247, https://doi.org/10.5194/amt-16-5217-2023, https://doi.org/10.5194/amt-16-5217-2023, 2023
Short summary
Short summary
Atmosphere remote sensing using GNSS radio occultation provides a highly valuable basis for atmospheric and climate science. For the highest-quality demands, the Wegener Center set up a rigorous system for processing low-level measurement data. This excess-phase processing setup includes integrated quality control and uncertainty estimation. It was successfully evaluated and inter-compared, ensuring the capability of producing reliable long-term data records for climate applications.
Jingna Bai, Yidong Lou, Weixing Zhang, Yaozong Zhou, Zhenyi Zhang, Chuang Shi, and Jingnan Liu
Atmos. Meas. Tech., 16, 5249–5259, https://doi.org/10.5194/amt-16-5249-2023, https://doi.org/10.5194/amt-16-5249-2023, 2023
Short summary
Short summary
Homogenized atmospheric water vapor data are an important prerequisite for climate analysis. Compared to other techniques, GPS has an inherent homogeneity advantage but requires reprocessing and homogenization to eliminate impacts of applied strategy and observation environmental changes. The low-elevation cut-off angles are suggested for the best estimates of zenith tropospheric delay (ZTD) reprocessing time series when compared to homogenized radiosonde data or ERA5 reference time series.
James Barry, Stefanie Meilinger, Klaus Pfeilsticker, Anna Herman-Czezuch, Nicola Kimiaie, Christopher Schirrmeister, Rone Yousif, Tina Buchmann, Johannes Grabenstein, Hartwig Deneke, Jonas Witthuhn, Claudia Emde, Felix Gödde, Bernhard Mayer, Leonhard Scheck, Marion Schroedter-Homscheidt, Philipp Hofbauer, and Matthias Struck
Atmos. Meas. Tech., 16, 4975–5007, https://doi.org/10.5194/amt-16-4975-2023, https://doi.org/10.5194/amt-16-4975-2023, 2023
Short summary
Short summary
Measured power data from solar photovoltaic (PV) systems contain information about the state of the atmosphere. In this work, power data from PV systems in the Allgäu region in Germany were used to determine the solar irradiance at each location, using state-of-the-art simulation and modelling. The results were validated using concurrent measurements of the incoming solar radiation in each case. If applied on a wider scale, this algorithm could help improve weather and climate models.
Wan Wu, Xu Liu, Liqiao Lei, Xiaozhen Xiong, Qiguang Yang, Qing Yue, Daniel K. Zhou, and Allen M. Larar
Atmos. Meas. Tech., 16, 4807–4832, https://doi.org/10.5194/amt-16-4807-2023, https://doi.org/10.5194/amt-16-4807-2023, 2023
Short summary
Short summary
We present a new operational physical retrieval algorithm that is used to retrieve atmospheric properties for each single field-of-view measurement of hyper-spectral IR sounders. The physical scheme includes a cloud-scattering calculation in its forward-simulation part. The data product generated using this algorithm has an advantage over traditional IR sounder data production algorithms in terms of improved spatial resolution and minimized error due to cloud contamination.
Zhen Li, Ad Stoffelen, Anton Verhoef, Zhixiong Wang, Jian Shang, and Honggang Yin
Atmos. Meas. Tech., 16, 4769–4783, https://doi.org/10.5194/amt-16-4769-2023, https://doi.org/10.5194/amt-16-4769-2023, 2023
Short summary
Short summary
WindRAD (Wind Radar) is the first dual-frequency rotating fan-beam scatterometer in orbit. We observe non-linearity in the backscatter distribution. Therefore, higher-order calibration (HOC) is proposed, which removes the non-linearities per incidence angle. The combination of HOC and NOCant is discussed. It can remove not only the non-linearity but also the anomalous harmonic azimuth dependencies caused by the antenna rotation; hence the optimal winds can be achieved with this combination.
Marco Gabella, Martin Lainer, Daniel Wolfensberger, and Jacopo Grazioli
Atmos. Meas. Tech., 16, 4409–4422, https://doi.org/10.5194/amt-16-4409-2023, https://doi.org/10.5194/amt-16-4409-2023, 2023
Short summary
Short summary
A still wind turbine observed with a fixed-pointing radar antenna has shown distinctive polarimetric signatures: the correlation coefficient between the two orthogonal polarization states was persistently equal to 1. The differential reflectivity and the radar reflectivity factors were also stable in time. Over 2 min (2000 Hz, 128 pulses were used; consequently, the sampling time was 64 ms), the standard deviation of the differential backscattering phase shift was only a few degrees.
Cited articles
Aggarwal, Ch. (Ed.): Data Streams – Models and Algorithms, Springer, 2007.
Allison, P.: Missing Data, Sage Publications, 2001.
Babcock, B., Babu, S., Datar, M., Motwani, R., and Widom, J.: Models and Issues in Data Stream Systems, in: Proc. of the 21st ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS, 1–16, 2002.
Bacher, P., Madsen, H., and Nielsen, H. A.: Online short-term solar power forecasting, Sol. Energy, 83, 1772–1783, 2009.
Bhardwaj, S., Sharma, V., Srivastava, S., Sastry, O., Bandyopadhyay, B., Chandel, S., and Gupta, J.: Estimation of solar radiation using a combination of Hidden Markov model and generalized Fuzzy model, Sol. Energy, 93, 43–54, 2013.
Black, C., Broadstock, D., Colin, A., and Hunt, L. C.: Filling in the gaps in transport studies: a practical guide to developments in data imputation methods, Traffic Eng. Control, 48, 358–363, 2007.
Enders, C. K.: Applied Missing Data Analysis, Guilford Press, 2010.
Hammer, A., Heinemann, D., Lorenz, E., and Lückehe, B.: Short-term forecasting of solar radiation: a statistical approach using satellite data, Sol. Energy, 67, 139–150, 1999.
Hari, P. and Kulmala, M.: Station for Measuring Ecosystem-Atmosphere Relations (SMEAR II), Boreal Environ. Res., 10, 315–322, 2005.
Hastie, T., Tibshirani, R., and Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer-Verlag, 2001.
Hoerl, A. E. and Kennard, R. W.: Ridge regression: biased estimation for nonorthogonal problems, Technometrics, 12, 55–67, 1970.
Hrust, L., Klaic, Z. B., Krizana, J., Antonic, O., and Hercog, P.: Neural network forecasting of air pollutants hourly concentrations using optimised temporal averages of meteorological variables and pollutant concentrations, Atmos. Environ., 43, 5588–5596, 2009.
Jolliffe, I. T.: Principal Component Analysis, 2nd Edn., Springer, 2002.
Junninen, H., Niska, H., Tuppurainen, K., Ruuskanen, J., and Kolehmainen, M.: Methods for imputation of missing values in air quality data sets, Atmos. Environ., 38, 2895–2907, 2004.
Junninen, H., Lauri, A., Keronen, P., Aalto, P., Hiltunen, V., Hari, P., and Kulmala, M.: Smart-SMEAR: on-line data exploration and visualization tool for SMEAR stations, Boreal Environ. Res., 14, 447–457, 2009.
Kadlec, P., Gabrys, B., and Strandt, S.: Data-driven soft sensors in the process industry, Comput. Chem. Eng., 33, 795–814, 2009.
Kopp, G. and Lean, J. L.: A new, lower value of total solar irradiance: Evidence and climate significance, Geophys. Res. Lett., 38, L01706, https://doi.org/10.1029/2010GL045777, 2011.
Lerner, U., Moses, B., Maricia, S., McIlraith, Sh. A., and Koller, D.: Monitoring a Complez Physical System using a Hybrid Dynamic Bayes Net., in: Proc. of the the 18th Conference in Uncertainty in Artificial Intelligence, UAI, 301–310, 2002.
Lu, H., Hsieh, J., and Chang, T.: Prediction of daily maximum ozone concentrations from meteorological conditions using a two-stage neural network, Atmos. Res., 81, 124–139, 2006.
Marquez, R. and Coimbra, C. F.: Forecasting of global and direct solar irradiance using stochastic learning methods, ground experiments and the NWS database, Sol. Energy, 85, 746–756, 2011.
Menut, L. and Bessagnet, B.: Atmospheric composition forecasting in Europe, Ann. Geophys., 28, 61–74, https://doi.org/10.5194/angeo-28-61-2010, 2010.
Michalsky, J.: The Astronomical Almanac's algorithm for approximate solar position (1950–2050), Sol. Energy, 40, 227–235, 1988.
Ramoni, M. and Sebastiani, P.: Robust Learning with Missing Data, Machine Learning, 45, 147–170, 2001.
Valle, S., Li, W., and Qin, S. J.: Selection of the number of principal components: the variance of the reconstruction error criterion with a comparison to other methods, Ind. Eng. Chem. Res., 38, 4389–4401, 1999.
Vuilleumier, L., Calpini, B., Cattin, R., Roulet, Y.-A., Stauch, V., Stöckli, R., and Giunta, I.: Solar radiation now-casting: the need for multiple data source integration, in: Proc. of the COST Action ES1002 "WIRE" State of the Art Workshop, available at: http://www.wire1002.ch/fileadmin/user_upload/Major_events/WS_Nice_2011/Spec._presentations/Vuillemier.pdf (last access: 14 July 2014), 2011.
Wold, S., Sjostroma, M., and Eriksson, L.: PLS-regression: a basic tool of chemometrics, Chemometr. Intell. Lab., 58, 109–130, 2001.
Žliobait\.e, I. and Hollmén, J.: Fault tolerant regression for sensor data, in: Proc. of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECMLPKDD, 449–464, 2013.
Žliobait\.e, I. and Hollmén, J.: Optimizing regression models for data streams with missing values, Mach. Learn., 1–27, https://doi.org/10.1007/s10994-014-5450-3, 2014.
Short summary
We present a case study in solar/radiation nowcasting using environmental sensor measurements as inputs. While some sensor readings may oftentimes be missing, predictions need to be output continuously in near real time. We are after linear regression models that would be robust to missing data, i.e., that would perform well with or without data gaps. We recommend using regularized a PCA regression with our established guidelines for building robust regression models.
We present a case study in solar/radiation nowcasting using environmental sensor measurements as...
