Articles | Volume 7, issue 12
https://doi.org/10.5194/amt-7-4387-2014
© Author(s) 2014. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
https://doi.org/10.5194/amt-7-4387-2014
© Author(s) 2014. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
Research article
| 
11 Dec 2014
Research article |
| 11 Dec 2014
Regression models tolerant to massively missing data: a case study in solar-radiation nowcasting
I. Žliobaitė
CORRESPONDING AUTHOR
Aalto University, Department of Information and Computer Science, Espoo, Finland
Helsinki Institute for Information Technology (HIIT), Helsinki, Finland
J. Hollmén
Aalto University, Department of Information and Computer Science, Espoo, Finland
Helsinki Institute for Information Technology (HIIT), Helsinki, Finland
H. Junninen
Department of Physics, University of Helsinki, Helsinki, Finland
Related authors
No articles found.
M. Sipilä, N. Sarnela, T. Jokinen, H. Junninen, J. Hakala, M. P. Rissanen, A. Praplan, M. Simon, A. Kürten, F. Bianchi, J. Dommen, J. Curtius, T. Petäjä, and D. R. Worsnop
Atmos. Meas. Tech., 8, 4001–4011, https://doi.org/10.5194/amt-8-4001-2015, https://doi.org/10.5194/amt-8-4001-2015, 2015
Short summary
Short summary
Atmospheric concentrations of amines are poorly known mainly due to challenges related to their reliable high-sensitivity detection. We have created a method and instrument that is capable for detecting amines with lowest limit of detection of around 0.01 parts per trillion. Application of the instrument in the field study indicates that concentrations of dimethyl amine in a boreal forest site are below 0.03ppt, not enough to account for the observed new particle formation rates.
A. Kürten, S. Münch, L. Rondo, F. Bianchi, J. Duplissy, T. Jokinen, H. Junninen, N. Sarnela, S. Schobesberger, M. Simon, M. Sipilä, J. Almeida, A. Amorim, J. Dommen, N. M. Donahue, E. M. Dunne, R. C. Flagan, A. Franchin, J. Kirkby, A. Kupc, V. Makhmutov, T. Petäjä, A. P. Praplan, F. Riccobono, G. Steiner, A. Tomé, G. Tsagkogeorgas, P. E. Wagner, D. Wimmer, U. Baltensperger, M. Kulmala, D. R. Worsnop, and J. Curtius
Atmos. Chem. Phys., 15, 10701–10721, https://doi.org/10.5194/acp-15-10701-2015, https://doi.org/10.5194/acp-15-10701-2015, 2015
Short summary
Short summary
New particle formation (NPF) is an important atmospheric process. At cold temperatures in the upper troposphere the binary (H2SO4-H2O) and ternary (H2SO4-H2O-NH3) system are thought to be important for NPF. Sulfuric acid monomer (H2SO4) and sulfuric acid dimer ((H2SO4)2) concentrations were measured between 208 and 248K for these systems and dimer evaporation rates were derived. These data will help to better understand and predict binary and ternary nucleation at low temperatures.
K. Ruusuvuori, P. Hietala, O. Kupiainen-Määttä, T. Jokinen, H. Junninen, M. Sipilä, T. Kurtén, and H. Vehkamäki
Atmos. Meas. Tech., 8, 2577–2588, https://doi.org/10.5194/amt-8-2577-2015, https://doi.org/10.5194/amt-8-2577-2015, 2015
Short summary
Short summary
Ionization reagents suitable for accurate measurements of the atmospheric gas-phase amine vapour concentrations are needed. Based on computational results, acetone is a viable option for use as an ionization reagent in CI-APi-TOF measurements on atmospheric dimethylamine. However, comparison between the computational and experimental results revealed notable discrepancies. Further study is still required before the acetone CI-APi-TOF can be considered a viable option in practice.
A. P. Praplan, S. Schobesberger, F. Bianchi, M. P. Rissanen, M. Ehn, T. Jokinen, H. Junninen, A. Adamov, A. Amorim, J. Dommen, J. Duplissy, J. Hakala, A. Hansel, M. Heinritzi, J. Kangasluoma, J. Kirkby, M. Krapf, A. Kürten, K. Lehtipalo, F. Riccobono, L. Rondo, N. Sarnela, M. Simon, A. Tomé, J. Tröstl, P. M. Winkler, C. Williamson, P. Ye, J. Curtius, U. Baltensperger, N. M. Donahue, M. Kulmala, and D. R. Worsnop
Atmos. Chem. Phys., 15, 4145–4159, https://doi.org/10.5194/acp-15-4145-2015, https://doi.org/10.5194/acp-15-4145-2015, 2015
Short summary
Short summary
Our study shows, based on data from three atmospheric pressure interface time-of-flight mass spectrometers measuring in parallel charged and neutral molecules and molecular clusters, how oxidised organic compounds bind to inorganic ions (e.g. bisulfate, nitrate, ammonium). This ionisation is selective for compounds with lower molar mass due to their limited amount and variety of functional groups. We also found that extremely low volatile organic compounds (ELVOCs) can be formed immediately.
C. Rose, K. Sellegri, E. Asmi, M. Hervo, E. Freney, A. Colomb, H. Junninen, J. Duplissy, M. Sipilä, J. Kontkanen, K. Lehtipalo, and M. Kulmala
Atmos. Chem. Phys., 15, 3413–3428, https://doi.org/10.5194/acp-15-3413-2015, https://doi.org/10.5194/acp-15-3413-2015, 2015
S. Schobesberger, A. Franchin, F. Bianchi, L. Rondo, J. Duplissy, A. Kürten, I. K. Ortega, A. Metzger, R. Schnitzhofer, J. Almeida, A. Amorim, J. Dommen, E. M. Dunne, M. Ehn, S. Gagné, L. Ickes, H. Junninen, A. Hansel, V.-M. Kerminen, J. Kirkby, A. Kupc, A. Laaksonen, K. Lehtipalo, S. Mathot, A. Onnela, T. Petäjä, F. Riccobono, F. D. Santos, M. Sipilä, A. Tomé, G. Tsagkogeorgas, Y. Viisanen, P. E. Wagner, D. Wimmer, J. Curtius, N. M. Donahue, U. Baltensperger, M. Kulmala, and D. R. Worsnop
Atmos. Chem. Phys., 15, 55–78, https://doi.org/10.5194/acp-15-55-2015, https://doi.org/10.5194/acp-15-55-2015, 2015
Short summary
Short summary
We used an ion mass spectrometer at CERN's CLOUD chamber to investigate the detailed composition of ammonia--sulfuric acid ion clusters (of both polarities) as they initially form and then grow into aerosol particles, at atmospherically relevant conditions. We found that these clusters’ composition is mainly determined by the ratio of the precursor vapors and ranges from ammonia-free clusters to clusters containing > 1 ammonia per sulfuric acid. Acid--base bindings are a key formation mechanism.
M. Sipilä, T. Jokinen, T. Berndt, S. Richters, R. Makkonen, N. M. Donahue, R. L. Mauldin III, T. Kurtén, P. Paasonen, N. Sarnela, M. Ehn, H. Junninen, M. P. Rissanen, J. Thornton, F. Stratmann, H. Herrmann, D. R. Worsnop, M. Kulmala, V.-M. Kerminen, and T. Petäjä
Atmos. Chem. Phys., 14, 12143–12153, https://doi.org/10.5194/acp-14-12143-2014, https://doi.org/10.5194/acp-14-12143-2014, 2014
L. Rondo, A. Kürten, S. Ehrhart, S. Schobesberger, A. Franchin, H. Junninen, T. Petäjä, M. Sipilä, D. R. Worsnop, and J. Curtius
Atmos. Meas. Tech., 7, 3849–3859, https://doi.org/10.5194/amt-7-3849-2014, https://doi.org/10.5194/amt-7-3849-2014, 2014
M. Crippa, F. Canonaco, V. A. Lanz, M. Äijälä, J. D. Allan, S. Carbone, G. Capes, D. Ceburnis, M. Dall'Osto, D. A. Day, P. F. DeCarlo, M. Ehn, A. Eriksson, E. Freney, L. Hildebrandt Ruiz, R. Hillamo, J. L. Jimenez, H. Junninen, A. Kiendler-Scharr, A.-M. Kortelainen, M. Kulmala, A. Laaksonen, A. A. Mensah, C. Mohr, E. Nemitz, C. O'Dowd, J. Ovadnevaite, S. N. Pandis, T. Petäjä, L. Poulain, S. Saarikoski, K. Sellegri, E. Swietlicki, P. Tiitta, D. R. Worsnop, U. Baltensperger, and A. S. H. Prévôt
Atmos. Chem. Phys., 14, 6159–6176, https://doi.org/10.5194/acp-14-6159-2014, https://doi.org/10.5194/acp-14-6159-2014, 2014
Related subject area
Subject: Others (Wind, Precipitation, Temperature, etc.) | Technique: Remote Sensing | Topic: Data Processing and Information Retrieval
Analysis of 2D airglow imager data with respect to dynamics using machine learning
Estimation of extreme precipitation events in Estonia and Italy using dual-polarization weather radar quantitative precipitation estimations
Detection and localization of F-layer ionospheric irregularities with the back-propagation method along the radio occultation ray path
Observations of anomalous propagation over waters near Sweden
Validation of Aeolus wind profiles using ground-based lidar and radiosonde observations at Réunion island and the Observatoire de Haute-Provence
Dual-frequency spectral radar retrieval of snowfall microphysics: a physics-driven deep-learning approach
High-resolution 3D winds derived from a modified WISSDOM synthesis scheme using multiple Doppler lidars and observations
Atmospheric boundary layer height from ground-based remote sensing: a review of capabilities and limitations
Assessing and mitigating the radar–radar interference in the German C-band weather radar network
Spectral replacement using machine learning methods for continuous mapping of the Geostationary Environment Monitoring Spectrometer (GEMS)
Using Optimal Estimation to Retrieve Winds from VAD scans by a Doppler Lidar
Doppler spectra from DWD's operational C-band radar birdbath scan: sampling strategy, spectral postprocessing, and multimodal analysis for the retrieval of precipitation processes
High-fidelity retrieval from instantaneous line-of-sight returns of nacelle-mounted lidar including supervised machine learning
Horizontal small-scale variability of water vapor in the atmosphere: implications for intercomparison of data from different measuring systems
Satellite observations of gravity wave momentum flux in the mesosphere and lower thermosphere (MLT): feasibility and requirements
An improved near-real-time precipitation retrieval for Brazil
Efficient Collocation of GNSS Radio Occultation Soundings with Passive Nadir Microwave Soundings
Radio frequency interference detection and mitigation in the DWD C-band weather radar network
Quality control and error assessment of the Aeolus L2B wind results from the Joint Aeolus Tropical Atlantic Campaign
Long-distance propagation of 162 MHz shipping information links associated with sporadic E
Estimation of refractivity uncertainties and vertical error correlations in collocated radio occultations, radiosondes, and model forecasts
DeepPrecip: a deep neural network for precipitation retrievals
Machine learning-based prediction of Alpine foehn events using GNSS troposphere products: first results for Altdorf, Switzerland
Meteor radar vertical wind observation biases and mathematical debiasing strategies including the 3DVAR+DIV algorithm
Adaptive thermal image velocimetry of spatial wind movement on landscapes using near-target infrared cameras
Image muting of mixed precipitation to improve identification of regions of heavy snow in radar data
Extending water vapor measurement capability of photon-limited differential absorption lidars through simultaneous denoising and inversion
GPROF-NN: a neural-network-based implementation of the Goddard Profiling Algorithm
Sensitivity analysis of DSD retrievals from polarimetric radar in stratiform rain based on the μ–Λ relationship
On the use of high-frequency surface wave oceanographic research radars as bistatic single-frequency oblique ionospheric sounders
A statistically optimal analysis of systematic differences between Aeolus horizontal line-of-sight winds and NOAA's Global Forecast System
Hierarchical deconvolution for incoherent scatter radar data
An alternative cloud index for estimating downwelling surface solar irradiance from various satellite imagers in the framework of a Heliosat-V method
ERUO: a spectral processing routine for the Micro Rain Radar PRO (MRR-PRO)
On the derivation of zonal and meridional wind components from Aeolus horizontal line-of-sight wind
Quantification of lightning-produced NOx over the Pyrenees and the Ebro Valley by using different TROPOMI-NO2 and cloud research products
Sensitivity analysis of attenuation in convective rainfall at X-band frequency using the mountain reference technique
A new scanning scheme and flexible retrieval for mean winds and gusts from Doppler lidar measurements
Airborne measurements of directional reflectivity over the Arctic marginal sea ice zone
High-resolution typhoon precipitation integrations using satellite infrared observations and multisource data
Continuous temperature soundings at the stratosphere and lower mesosphere with a ground-based radiometer considering the Zeeman effect
Retrieval of solar-induced chlorophyll fluorescence (SIF) from satellite measurements: comparison of SIF between TanSat and OCO-2
Identification of tropical cyclones via deep convolutional neural network based on satellite cloud images
Time evolution of temperature profiles retrieved from 13 years of infrared atmospheric sounding interferometer (IASI) data using an artificial neural network
Emissivity retrievals with FORUM's end-to-end simulator: challenges and recommendations
Detecting wave features in Doppler radial velocity radar observations
Remote sensing of solar surface radiation – a reflection of concepts, applications and input data based on experience with the effective cloud albedo
Snow microphysical retrieval from the NASA D3R radar during ICE-POP 2018
Retrieval improvements for the ALADIN Airborne Demonstrator in support of the Aeolus wind product validation
Cloud-probability-based estimation of black-sky surface albedo from AVHRR data
René Sedlak, Andreas Welscher, Patrick Hannawald, Sabine Wüst, Rainer Lienhart, and Michael Bittner
Atmos. Meas. Tech., 16, 3141–3153, https://doi.org/10.5194/amt-16-3141-2023, https://doi.org/10.5194/amt-16-3141-2023, 2023
Short summary
Short summary
We show that machine learning can help in classifying images of the OH* airglow, a thin layer in the middle atmosphere (ca. 86 km height) emitting infrared radiation, in an efficient way. By doing this,
dynamicepisodes of strong movement in the OH* airglow caused predominantly by waves can be extracted automatically from large data sets. Within these dynamic episodes, turbulent wave breaking can also be found. We use these observations of turbulence to derive the energy released by waves.
Roberto Cremonini, Tanel Voormansik, Piia Post, and Dmitri Moisseev
Atmos. Meas. Tech., 16, 2943–2956, https://doi.org/10.5194/amt-16-2943-2023, https://doi.org/10.5194/amt-16-2943-2023, 2023
Short summary
Short summary
Extreme rainfall for a specific location is commonly evaluated when designing stormwater management systems. This study investigates the use of quantitative precipitation estimations (QPEs) based on polarimetric weather radar data, without rain gauge corrections, to estimate 1 h rainfall total maxima in Italy and Estonia. We show that dual-polarization weather radar provides reliable QPEs and effective estimations of return periods for extreme rainfall in climatologically homogeneous regions.
Vinícius Ludwig-Barbosa, Joel Rasch, Thomas Sievert, Anders Carlström, Mats I. Pettersson, Viet Thuy Vu, and Jacob Christensen
Atmos. Meas. Tech., 16, 1849–1864, https://doi.org/10.5194/amt-16-1849-2023, https://doi.org/10.5194/amt-16-1849-2023, 2023
Short summary
Short summary
In this paper, the back-propagation method's capabilities and limitations regarding the location of irregularity regions in the ionosphere, e.g. equatorial plasma bubbles, are evaluated. The assessment was performed with simulations in which different scenarios were assumed. The results showed that the location estimate is possible if the amplitude of the ionospheric disturbance is stronger than the instrument noise level. Further, multiple patches can be located if regions are well separated.
Lars Norin
Atmos. Meas. Tech., 16, 1789–1801, https://doi.org/10.5194/amt-16-1789-2023, https://doi.org/10.5194/amt-16-1789-2023, 2023
Short summary
Short summary
The atmosphere can cause radar beams to bend more or less towards the ground. When the atmosphere differs from standard atmospheric conditions, the propagation is considered anomalous. Radars affected by anomalous propagation can observe ground clutter far beyond the radar horizon. Here, 4.5 years' worth of data from five operational Swedish weather radars are presented. Analyses of the data reveal a strong seasonal cycle and weaker diurnal cycle in ground clutter from across nearby waters.
Mathieu Ratynski, Sergey Khaykin, Alain Hauchecorne, Robin Wing, Jean-Pierre Cammas, Yann Hello, and Philippe Keckhut
Atmos. Meas. Tech., 16, 997–1016, https://doi.org/10.5194/amt-16-997-2023, https://doi.org/10.5194/amt-16-997-2023, 2023
Short summary
Short summary
Aeolus is the first spaceborne wind lidar providing global wind measurements since 2018. This study offers a comprehensive analysis of Aeolus instrument performance, using ground-based wind lidars and meteorological radiosondes, at tropical and mid-latitudes sites. The analysis allows assessing the long-term evolution of the satellite's performance for more than 3 years. The results will help further elaborate the understanding of the error sources and the behavior of the Doppler wind lidar.
Anne-Claire Billault-Roux, Gionata Ghiggi, Louis Jaffeux, Audrey Martini, Nicolas Viltard, and Alexis Berne
Atmos. Meas. Tech., 16, 911–940, https://doi.org/10.5194/amt-16-911-2023, https://doi.org/10.5194/amt-16-911-2023, 2023
Short summary
Short summary
Better understanding and modeling snowfall properties and processes is relevant to many fields, ranging from weather forecasting to aircraft safety. Meteorological radars can be used to gain insights into the microphysics of snowfall. In this work, we propose a new method to retrieve snowfall properties from measurements of radars with different frequencies. It relies on an original deep-learning framework, which incorporates knowledge of the underlying physics, i.e., electromagnetic scattering.
Chia-Lun Tsai, Kwonil Kim, Yu-Chieng Liou, and GyuWon Lee
Atmos. Meas. Tech., 16, 845–869, https://doi.org/10.5194/amt-16-845-2023, https://doi.org/10.5194/amt-16-845-2023, 2023
Short summary
Short summary
Since the winds in clear-air conditions usually play an important role in the initiation of various weather systems and phenomena, the modified Wind Synthesis System using Doppler Measurements (WISSDOM) synthesis scheme was developed to derive high-quality and high-spatial-resolution 3D winds under clear-air conditions. The performance and accuracy of derived 3D winds from this modified scheme were evaluated with an extreme strong wind event over complex terrain in Pyeongchang, South Korea.
Simone Kotthaus, Juan Antonio Bravo-Aranda, Martine Collaud Coen, Juan Luis Guerrero-Rascado, Maria João Costa, Domenico Cimini, Ewan J. O'Connor, Maxime Hervo, Lucas Alados-Arboledas, María Jiménez-Portaz, Lucia Mona, Dominique Ruffieux, Anthony Illingworth, and Martial Haeffelin
Atmos. Meas. Tech., 16, 433–479, https://doi.org/10.5194/amt-16-433-2023, https://doi.org/10.5194/amt-16-433-2023, 2023
Short summary
Short summary
Profile observations of the atmospheric boundary layer now allow for layer heights and characteristics to be derived at high temporal and vertical resolution. With novel high-density ground-based remote-sensing measurement networks emerging, horizontal information content is also increasing. This review summarises the capabilities and limitations of various sensors and retrieval algorithms which need to be considered during the harmonisation of data products for high-impact applications.
Michael Frech, Cornelius Hald, Maximilian Schaper, Bertram Lange, and Benjamin Rohrdantz
Atmos. Meas. Tech., 16, 295–309, https://doi.org/10.5194/amt-16-295-2023, https://doi.org/10.5194/amt-16-295-2023, 2023
Short summary
Short summary
Weather radar data are the backbone of a lot of meteorological products. In order to obtain a better low-level coverage with radar data, additional systems have to be included. The frequency range in which radars are allowed to operate is limited. A potential radar-to-radar interference has to be avoided. The paper derives guidelines on how additional radars can be included into a C-band weather radar network and how interferences can be avoided.
Yeeun Lee, Myoung-Hwan Ahn, Mina Kang, and Mijin Eo
Atmos. Meas. Tech., 16, 153–168, https://doi.org/10.5194/amt-16-153-2023, https://doi.org/10.5194/amt-16-153-2023, 2023
Short summary
Short summary
This study aims to verify that a partly defective hyperspectral measurement can be successfully reproduced with concise machine learning models coupled with principal component analysis. Evaluation of the approach is performed with radiances and retrieval results of ozone and cloud properties. Considering that GEMS is the first geostationary UV–VIS hyperspectral spectrometer, we expect our findings can be introduced further to similar geostationary environmental instruments to be launched soon.
Sunil Baidar, Timothy J. Wagner, David D. Turner, and W. Alan Brewer
Atmos. Meas. Tech. Discuss., https://doi.org/10.5194/amt-2022-337, https://doi.org/10.5194/amt-2022-337, 2023
Revised manuscript accepted for AMT
Short summary
Short summary
This paper provides a new method for retrieving wind profile from Coherent Doppler Lidar (CDL) measurements. It takes advantage of layer-to-layer correlation in wind profiles to fill in the gaps, where CDL signal is too small, to provide continuous wind profiles up to 3 km. Comparison with current method and collocated radiosonde wind measurements showed excellent agreement with no degradation in results where the current method provides a valid results.
Mathias Gergely, Maximilian Schaper, Matthias Toussaint, and Michael Frech
Atmos. Meas. Tech., 15, 7315–7335, https://doi.org/10.5194/amt-15-7315-2022, https://doi.org/10.5194/amt-15-7315-2022, 2022
Short summary
Short summary
This study presents the new vertically pointing birdbath scan of the German C-band radar network, which provides high-resolution profiles of precipitating clouds above all DWD weather radars since the spring of 2021. Our AI-based postprocessing method for filtering and analyzing the recorded radar data offers a unique quantitative view into a wide range of precipitation events from snowfall over stratiform rain to intense frontal showers and will be used to complement DWD's operational services.
Kenneth A. Brown and Thomas G. Herges
Atmos. Meas. Tech., 15, 7211–7234, https://doi.org/10.5194/amt-15-7211-2022, https://doi.org/10.5194/amt-15-7211-2022, 2022
Short summary
Short summary
The character of the airflow around and within wind farms has a significant impact on the energy output and longevity of the wind turbines in the farm. For both research and control purposes, accurate measurements of the wind speed are required, and these are often accomplished with remote sensing devices. This article pertains to a field experiment of a lidar mounted to a wind turbine and demonstrates three data post-processing techniques with efficacy at extracting useful airflow information.
Xavier Calbet, Cintia Carbajal Henken, Sergio DeSouza-Machado, Bomin Sun, and Tony Reale
Atmos. Meas. Tech., 15, 7105–7118, https://doi.org/10.5194/amt-15-7105-2022, https://doi.org/10.5194/amt-15-7105-2022, 2022
Short summary
Short summary
Water vapor concentration in the atmosphere at small scales (< 6 km) is considered. The measurements show Gaussian random field behavior following Kolmogorov's theory of turbulence two-thirds law. These properties can be useful when estimating the water vapor variability within a given observed satellite scene or when different water vapor measurements have to be merged consistently.
Qiuyu Chen, Konstantin Ntokas, Björn Linder, Lukas Krasauskas, Manfred Ern, Peter Preusse, Jörn Ungermann, Erich Becker, Martin Kaufmann, and Martin Riese
Atmos. Meas. Tech., 15, 7071–7103, https://doi.org/10.5194/amt-15-7071-2022, https://doi.org/10.5194/amt-15-7071-2022, 2022
Short summary
Short summary
Observations of phase speed and direction spectra as well as zonal mean net gravity wave momentum flux are required to understand how gravity waves reach the mesosphere–lower thermosphere and how they there interact with background flow. To this end we propose flying two CubeSats, each deploying a spatial heterodyne spectrometer for limb observation of the airglow. End-to-end simulations demonstrate that individual gravity waves are retrieved faithfully for the expected instrument performance.
Simon Pfreundschuh, Ingrid Ingemarsson, Patrick Eriksson, Daniel A. Vila, and Alan J. P. Calheiros
Atmos. Meas. Tech., 15, 6907–6933, https://doi.org/10.5194/amt-15-6907-2022, https://doi.org/10.5194/amt-15-6907-2022, 2022
Short summary
Short summary
We used methods from the field of artificial intelligence to train an algorithm to estimate rain from satellite observations. In contrast to other methods, our algorithm not only estimates rain, but also the uncertainty of the estimate. Using independent measurements from rain gauges, we show that our method performs better than currently available methods and that the provided uncertainty estimates are reliable. Our method makes satellite-based measurements of rain more accurate and reliable.
Alex Meredith, Stephen S. Leroy, and Kerri Cahoy
EGUsphere, https://doi.org/10.5194/egusphere-2022-1266, https://doi.org/10.5194/egusphere-2022-1266, 2022
Short summary
Short summary
We developed a new efficient algorithm leveraging orbital dynamics to collocate radio occultation soundings with microwave radiance soundings. This new algorithm is 99 % accurate and is much faster than traditional collocation-finding approaches. Speeding up collocation-finding is useful for calibrating and validating microwave radiometers and for data assimilation into numerical weather prediction models. Our algorithm can also be used to predict collocation yield for new satellite missions.
Maximilian Schaper, Michael Frech, David Michaelis, Cornelius Hald, and Benjamin Rohrdantz
Atmos. Meas. Tech., 15, 6625–6642, https://doi.org/10.5194/amt-15-6625-2022, https://doi.org/10.5194/amt-15-6625-2022, 2022
Short summary
Short summary
C-band weather radar data are commonly compromised by radio frequency interference (RFI) from external sources. It is not possible to separate a superimposed interference signal from the radar data. Therefore, the best course of action is to shut down RFI sources as quickly as possible. An automated RFI detection algorithm has been developed. Since its implementation, persistent RFI sources are eliminated much more quickly, while the number of short-lived RFI sources keeps steadily increasing.
Oliver Lux, Benjamin Witschas, Alexander Geiß, Christian Lemmerz, Fabian Weiler, Uwe Marksteiner, Stephan Rahm, Andreas Schäfler, and Oliver Reitebuch
Atmos. Meas. Tech., 15, 6467–6488, https://doi.org/10.5194/amt-15-6467-2022, https://doi.org/10.5194/amt-15-6467-2022, 2022
Short summary
Short summary
We discuss the influence of different quality control schemes on the results of Aeolus wind product validation and present statistical tools for ensuring consistency and comparability among diverse validation studies with regard to the specific error characteristics of the Rayleigh-clear and Mie-cloudy winds. The developed methods are applied for the validation of Aeolus winds against an ECMWF model background and airborne wind lidar data from the Joint Aeolus Tropical Atlantic Campaign.
Alex T. Chartier, Thomas R. Hanley, and Daniel J. Emmons
Atmos. Meas. Tech., 15, 6387–6393, https://doi.org/10.5194/amt-15-6387-2022, https://doi.org/10.5194/amt-15-6387-2022, 2022
Short summary
Short summary
This is a study of anomalous long-distance (>1000 km) radio propagation that was identified in United States Coast Guard monitors of automatic identification system (AIS) shipping transmissions at 162 MHz. Our results indicate this long-distance propagation is caused by dense sporadic E layers in the daytime ionosphere, which were observed by nearby ionosondes at the same time. This finding is surprising because it indicates these sporadic E layers may be far more dense than previously thought.
Johannes K. Nielsen, Hans Gleisner, Stig Syndergaard, and Kent B. Lauritsen
Atmos. Meas. Tech., 15, 6243–6256, https://doi.org/10.5194/amt-15-6243-2022, https://doi.org/10.5194/amt-15-6243-2022, 2022
Short summary
Short summary
This paper provides a new way to estimate uncertainties and error correlations. The method is a generalization of a known method called the
three-cornered hat: Instead of calculating uncertainties from assumed knowledge about the observation method, uncertainties and error correlations are estimated statistically from tree independent observation series, measuring the same variable. The results are useful for future estimation of atmospheric-specific humidity from the bending of radio waves.
Fraser King, George Duffy, Lisa Milani, Christopher G. Fletcher, Claire Pettersen, and Kerstin Ebell
Atmos. Meas. Tech., 15, 6035–6050, https://doi.org/10.5194/amt-15-6035-2022, https://doi.org/10.5194/amt-15-6035-2022, 2022
Short summary
Short summary
Under warmer global temperatures, precipitation patterns are expected to shift substantially, with critical impact on the global water-energy budget. In this work, we develop a deep learning model for predicting snow and rain accumulation based on surface radar observations of the lower atmosphere. Our model demonstrates improved skill over traditional methods and provides new insights into the regions of the atmosphere that provide the most significant contributions to high model accuracy.
Matthias Aichinger-Rosenberger, Elmar Brockmann, Laura Crocetti, Benedikt Soja, and Gregor Moeller
Atmos. Meas. Tech., 15, 5821–5839, https://doi.org/10.5194/amt-15-5821-2022, https://doi.org/10.5194/amt-15-5821-2022, 2022
Short summary
Short summary
This study develops an innovative approach for the detection and prediction of foehn winds. The approach uses products generated from GNSS (Global Navigation Satellite Systems) in combination with machine learning-based classification algorithms to detect and predict foehn winds at Altdorf, Switzerland. Results are encouraging and comparable to similar studies using meteorological data, which might qualify the method as an additional tool for short-term foehn forecasting in the future.
Gunter Stober, Alan Liu, Alexander Kozlovsky, Zishun Qiao, Ales Kuchar, Christoph Jacobi, Chris Meek, Diego Janches, Guiping Liu, Masaki Tsutsumi, Njål Gulbrandsen, Satonori Nozawa, Mark Lester, Evgenia Belova, Johan Kero, and Nicholas Mitchell
Atmos. Meas. Tech., 15, 5769–5792, https://doi.org/10.5194/amt-15-5769-2022, https://doi.org/10.5194/amt-15-5769-2022, 2022
Short summary
Short summary
Precise and accurate measurements of vertical winds at the mesosphere and lower thermosphere are rare. Although meteor radars have been used for decades to observe horizontal winds, their ability to derive reliable vertical wind measurements was always questioned. In this article, we provide mathematical concepts to retrieve mathematically and physically consistent solutions, which are compared to the state-of-the-art non-hydrostatic model UA-ICON.
Benjamin Schumacher, Marwan Katurji, Jiawei Zhang, Peyman Zawar-Reza, Benjamin Adams, and Matthias Zeeman
Atmos. Meas. Tech., 15, 5681–5700, https://doi.org/10.5194/amt-15-5681-2022, https://doi.org/10.5194/amt-15-5681-2022, 2022
Short summary
Short summary
This investigation presents adaptive thermal image velocimetry (A-TIV), a newly developed algorithm to spatially measure near-surface atmospheric velocities using an infrared camera mounted on uncrewed aerial vehicles. A validation and accuracy assessment of the retrieved velocity fields shows the successful application of the algorithm over short-cut grass and turf surfaces in dry conditions. This provides new opportunities for atmospheric scientists to study surface–atmosphere interactions.
Laura M. Tomkins, Sandra E. Yuter, Matthew A. Miller, and Luke R. Allen
Atmos. Meas. Tech., 15, 5515–5525, https://doi.org/10.5194/amt-15-5515-2022, https://doi.org/10.5194/amt-15-5515-2022, 2022
Short summary
Short summary
Locally higher radar reflectivity values in winter storms can mean more snowfall or a transition from snow to mixtures of snow, partially melted snow, and/or rain. We use the correlation coefficient to de-emphasize regions of mixed precipitation. Visual muting is valuable for analyzing and monitoring evolving weather conditions during winter storm events.
Willem J. Marais and Matthew Hayman
Atmos. Meas. Tech., 15, 5159–5180, https://doi.org/10.5194/amt-15-5159-2022, https://doi.org/10.5194/amt-15-5159-2022, 2022
Short summary
Short summary
For atmospheric science and weather prediction, it is important to make water vapor measurements in real time. A low-cost lidar instrument has been developed by Montana State University and the National Center for Atmospheric Research. We developed an advanced signal-processing method to extend the scientific capability of the lidar instrument. With the new method we show that the maximum altitude at which the MPD can make water vapor measurements can be extended up to 8 km.
Simon Pfreundschuh, Paula J. Brown, Christian D. Kummerow, Patrick Eriksson, and Teodor Norrestad
Atmos. Meas. Tech., 15, 5033–5060, https://doi.org/10.5194/amt-15-5033-2022, https://doi.org/10.5194/amt-15-5033-2022, 2022
Short summary
Short summary
The Global Precipitation Measurement mission is an international satellite mission providing regular global rain measurements. We present two newly developed machine-learning-based implementations of one of the algorithms responsible for turning the satellite observations into rain measurements. We show that replacing the current algorithm with a neural network improves the accuracy of the measurements. A neural network that also makes use of spatial information unlocks further improvements.
Christos Gatidis, Marc Schleiss, and Christine Unal
Atmos. Meas. Tech., 15, 4951–4969, https://doi.org/10.5194/amt-15-4951-2022, https://doi.org/10.5194/amt-15-4951-2022, 2022
Short summary
Short summary
Knowledge of the raindrop size distribution (DSD) is crucial for understanding rainfall microphysics and quantifying uncertainty in quantitative precipitation estimates. In this study a general overview of the DSD retrieval approach from a polarimetric radar is discussed, highlighting sensitivity to potential sources of errors, either directly linked to the radar measurements or indirectly through the critical modeling assumptions behind the method such as the shape–size (μ–Λ) relationship.
Stephen R. Kaeppler, Ethan S. Miller, Daniel Cole, and Teresa Updyke
Atmos. Meas. Tech., 15, 4531–4545, https://doi.org/10.5194/amt-15-4531-2022, https://doi.org/10.5194/amt-15-4531-2022, 2022
Short summary
Short summary
This investigation demonstrates how useful ionospheric parameters can be extracted from existing high-frequency radars that are used for oceanographic research. The methodology presented can be used by scientists and radio amateurs to understand ionospheric dynamics.
Hui Liu, Kevin Garrett, Kayo Ide, Ross N. Hoffman, and Katherine E. Lukens
Atmos. Meas. Tech., 15, 3925–3940, https://doi.org/10.5194/amt-15-3925-2022, https://doi.org/10.5194/amt-15-3925-2022, 2022
Short summary
Short summary
A total least squares (TLS) regression is used to optimally estimate linear speed-dependent biases between Aeolus Level-2B winds and short-term (6 h) forecasts of NOAA’s FV3GFS. The winds for 1–7 September 2019 are examined. Clear speed-dependent biases for both Mie and Rayleigh winds are found, particularly in the tropics and Southern Hemisphere. Use of the TLS correction improves the forecast of the 26–28 November 2019 winter storm over the USA.
Snizhana Ross, Arttu Arjas, Ilkka I. Virtanen, Mikko J. Sillanpää, Lassi Roininen, and Andreas Hauptmann
Atmos. Meas. Tech., 15, 3843–3857, https://doi.org/10.5194/amt-15-3843-2022, https://doi.org/10.5194/amt-15-3843-2022, 2022
Short summary
Short summary
Radar measurements of thermal fluctuations in the Earth's ionosphere produce weak signals, and tuning to specific altitudes results in suboptimal resolution for other regions, making an accurate analysis of these changes difficult. A novel approach to improve the resolution and remove measurement noise is considered. The method can capture variable characteristics, making it ideal for the study of a large range of data. Synthetically generated examples and two measured datasets were considered.
Benoît Tournadre, Benoît Gschwind, Yves-Marie Saint-Drenan, Xuemei Chen, Rodrigo Amaro E Silva, and Philippe Blanc
Atmos. Meas. Tech., 15, 3683–3704, https://doi.org/10.5194/amt-15-3683-2022, https://doi.org/10.5194/amt-15-3683-2022, 2022
Short summary
Short summary
Solar radiation received by the Earth's surface is valuable information for various fields like the photovoltaic industry or climate research. Pictures taken from satellites can be used to estimate the solar radiation from cloud reflectivity. Two issues for a good estimation are different instrumentations and orbits. We modify a widely used method that is today only used on geostationary satellites, so it can be applied on instruments on different orbits and with different sensitivities.
Alfonso Ferrone, Anne-Claire Billault-Roux, and Alexis Berne
Atmos. Meas. Tech., 15, 3569–3592, https://doi.org/10.5194/amt-15-3569-2022, https://doi.org/10.5194/amt-15-3569-2022, 2022
Short summary
Short summary
The Micro Rain Radar PRO (MRR-PRO) is a meteorological radar, with a relevant set of features for deployment in remote locations. We developed an algorithm, named ERUO, for the processing of its measurements of snowfall. The algorithm addresses typical issues of the raw spectral data, such as interference lines, but also improves the quality and sensitivity of the radar variables. ERUO has been evaluated over four different datasets collected in Antarctica and in the Swiss Jura.
Isabell Krisch, Neil P. Hindley, Oliver Reitebuch, and Corwin J. Wright
Atmos. Meas. Tech., 15, 3465–3479, https://doi.org/10.5194/amt-15-3465-2022, https://doi.org/10.5194/amt-15-3465-2022, 2022
Short summary
Short summary
The Aeolus satellite measures global height resolved profiles of wind along a certain line-of-sight. However, for atmospheric dynamics research, wind measurements along the three cardinal axes are most useful. This paper presents methods to convert the measurements into zonal and meridional wind components. By combining the measurements during ascending and descending orbits, we achieve good derivation of zonal wind (equatorward of 80° latitude) and meridional wind (poleward of 70° latitude).
Francisco J. Pérez-Invernón, Heidi Huntrieser, Thilo Erbertseder, Diego Loyola, Pieter Valks, Song Liu, Dale J. Allen, Kenneth E. Pickering, Eric J. Bucsela, Patrick Jöckel, Jos van Geffen, Henk Eskes, Sergio Soler, Francisco J. Gordillo-Vázquez, and Jeff Lapierre
Atmos. Meas. Tech., 15, 3329–3351, https://doi.org/10.5194/amt-15-3329-2022, https://doi.org/10.5194/amt-15-3329-2022, 2022
Short summary
Short summary
Lightning, one of the major sources of nitrogen oxides in the atmosphere, contributes to the tropospheric concentration of ozone and to the oxidizing capacity of the atmosphere. In this work, we contribute to improving the estimation of lightning-produced nitrogen oxides in the Ebro Valley and the Pyrenees by using two different TROPOMI products and comparing the results.
Guy Delrieu, Anil Kumar Khanal, Frédéric Cazenave, and Brice Boudevillain
Atmos. Meas. Tech., 15, 3297–3314, https://doi.org/10.5194/amt-15-3297-2022, https://doi.org/10.5194/amt-15-3297-2022, 2022
Short summary
Short summary
The RadAlp experiment aims at improving quantitative precipitation estimation in the Alps thanks to X-band polarimetric radars and in situ measurements deployed in Grenoble, France. We revisit the physics of propagation and attenuation of microwaves in rain. We perform a generalized sensitivity analysis in order to establish useful parameterization for attenuation corrections. Originality lies in the use of otherwise undesired mountain returns for constraining the considered physical model.
Julian Steinheuer, Carola Detring, Frank Beyrich, Ulrich Löhnert, Petra Friederichs, and Stephanie Fiedler
Atmos. Meas. Tech., 15, 3243–3260, https://doi.org/10.5194/amt-15-3243-2022, https://doi.org/10.5194/amt-15-3243-2022, 2022
Short summary
Short summary
Doppler wind lidars (DWLs) allow the determination of wind profiles with high vertical resolution and thus provide an alternative to meteorological towers. We address the question of whether wind gusts can be derived since they are short-lived phenomena. Therefore, we compare different DWL configurations and develop a new method applicable to all of them. A fast continuous scanning mode that completes a full observation cycle within 3.4 s is found to be the best-performing configuration.
Sebastian Becker, André Ehrlich, Evelyn Jäkel, Tim Carlsen, Michael Schäfer, and Manfred Wendisch
Atmos. Meas. Tech., 15, 2939–2953, https://doi.org/10.5194/amt-15-2939-2022, https://doi.org/10.5194/amt-15-2939-2022, 2022
Short summary
Short summary
Airborne radiation measurements are used to characterize the solar directional reflection of a mixture of Arctic sea ice and open-ocean surfaces in the transition zone between both surface types. The mixture reveals reflection properties of both surface types. It is shown that the directional reflection of the mixture can be reconstructed from the directional reflection of the individual surfaces, accounting for the special conditions present in the transition zone.
You Zhao, Chao Liu, Di Di, Ziqiang Ma, and Shihao Tang
Atmos. Meas. Tech., 15, 2791–2805, https://doi.org/10.5194/amt-15-2791-2022, https://doi.org/10.5194/amt-15-2791-2022, 2022
Short summary
Short summary
A typhoon is a high-impact atmospheric phenomenon that causes most significant socioeconomic damage, and its precipitation observation is always needed for typhoon characteristics and disaster prevention. This study developed a typhoon precipitation fusion method to combine observations from satellite radiometers, rain gauges and reanalysis to provide much improved typhoon precipitation datasets.
Witali Krochin, Francisco Navas-Guzmán, David Kuhl, Axel Murk, and Gunter Stober
Atmos. Meas. Tech., 15, 2231–2249, https://doi.org/10.5194/amt-15-2231-2022, https://doi.org/10.5194/amt-15-2231-2022, 2022
Short summary
Short summary
This study leverages atmospheric temperature measurements performed with a ground-based radiometer making use of data that was collected during a 4-year observational campaign applying a new retrieval algorithm that improves the maximal altitude range from 45 to 55 km. The measurements are validated against two independent data sets, MERRA2 reanalysis data and the meteorological analysis of NAVGEM-HA.
Lu Yao, Yi Liu, Dongxu Yang, Zhaonan Cai, Jing Wang, Chao Lin, Naimeng Lu, Daren Lyu, Longfei Tian, Maohua Wang, Zengshan Yin, Yuquan Zheng, and Sisi Wang
Atmos. Meas. Tech., 15, 2125–2137, https://doi.org/10.5194/amt-15-2125-2022, https://doi.org/10.5194/amt-15-2125-2022, 2022
Short summary
Short summary
A physics-based SIF retrieval algorithm, IAPCAS/SIF, is introduced and applied to OCO-2 and TanSat measurements. The strong linear relationship between OCO-2 SIF retrieved by IAPCAS/SIF and the official product indicates the algorithm's reliability. The good consistency in the spatiotemporal patterns and magnitude of the OCO-2 and TanSat SIF products suggests that the combinative usage of multi-satellite products has potential and that such work would contribute to further research.
Biao Tong, Xiangfei Sun, Jiyang Fu, Yuncheng He, and Pakwai Chan
Atmos. Meas. Tech., 15, 1829–1848, https://doi.org/10.5194/amt-15-1829-2022, https://doi.org/10.5194/amt-15-1829-2022, 2022
Short summary
Short summary
In recent years, there has been numerous research on tropical cyclone (TC) observation based on satellite cloud images (SCIs), but most methods are limited by low efficiency and subjectivity. To overcome subjectivity and improve efficiency of traditional methods, this paper uses deep learning technology to do further research on fingerprint identification of TCs. Results provide an automatic and objective method to distinguish TCs from SCIs and are convenient for subsequent research.
Marie Bouillon, Sarah Safieddine, Simon Whitburn, Lieven Clarisse, Filipe Aires, Victor Pellet, Olivier Lezeaux, Noëlle A. Scott, Marie Doutriaux-Boucher, and Cathy Clerbaux
Atmos. Meas. Tech., 15, 1779–1793, https://doi.org/10.5194/amt-15-1779-2022, https://doi.org/10.5194/amt-15-1779-2022, 2022
Short summary
Short summary
The IASI instruments have been observing Earth since 2007. We use a neural network to retrieve atmospheric temperatures. This new temperature data record is validated against other datasets and shows good agreement. We use this new dataset to compute trends over the 2008–2020 period. We found a warming of the troposphere, more important at the poles. In the stratosphere, we found that temperatures decrease everywhere except at the South Pole. The cooling is more pronounced at the South pole.
Maya Ben-Yami, Hilke Oetjen, Helen Brindley, William Cossich, Dulce Lajas, Tiziano Maestri, Davide Magurno, Piera Raspollini, Luca Sgheri, and Laura Warwick
Atmos. Meas. Tech., 15, 1755–1777, https://doi.org/10.5194/amt-15-1755-2022, https://doi.org/10.5194/amt-15-1755-2022, 2022
Short summary
Short summary
Spectral emissivity is a key property of the Earth's surface. Few measurements exist in the far-infrared, despite recent work showing that its contribution is important for accurate modelling of global climate. In preparation for ESA’s EE9 FORUM mission (launch in 2026), this study takes the first steps towards the development of an operational emissivity retrieval for FORUM by investigating the sensitivity of the emissivity product to different physical and operational parameters.
Matthew A. Miller, Sandra E. Yuter, Nicole P. Hoban, Laura M. Tomkins, and Brian A. Colle
Atmos. Meas. Tech., 15, 1689–1702, https://doi.org/10.5194/amt-15-1689-2022, https://doi.org/10.5194/amt-15-1689-2022, 2022
Short summary
Short summary
Apparent waves in the atmosphere and similar features in storm winds can be detected by taking the difference between successive Doppler weather radar scans measuring radar-relative storm air motions. Applying image filtering to the difference data better isolates the detected signal. This technique is a useful tool in weather research and forecasting since such waves can trigger or enhance precipitation.
Richard Müller and Uwe Pfeifroth
Atmos. Meas. Tech., 15, 1537–1561, https://doi.org/10.5194/amt-15-1537-2022, https://doi.org/10.5194/amt-15-1537-2022, 2022
Short summary
Short summary
The great works of physics teach us that a central paradigm of science should be to make methods and theories as easy as possible and as complex as needed. This paper provides a brief review of remote sensing of solar surface irradiance based on this paradigm.
S. Joseph Munchak, Robert S. Schrom, Charles N. Helms, and Ali Tokay
Atmos. Meas. Tech., 15, 1439–1464, https://doi.org/10.5194/amt-15-1439-2022, https://doi.org/10.5194/amt-15-1439-2022, 2022
Short summary
Short summary
The ability to measure snowfall with weather radar has greatly advanced with the development of techniques that utilize dual-polarization measurements, which provide information about the snow particle shape and orientation, and multi-frequency measurements, which provide information about size and density. This study combines these techniques with the NASA D3R radar, which provides dual-frequency polarimetric measurements, with data that were observed during the 2018 Winter Olympics.
Oliver Lux, Christian Lemmerz, Fabian Weiler, Uwe Marksteiner, Benjamin Witschas, Stephan Rahm, Alexander Geiß, Andreas Schäfler, and Oliver Reitebuch
Atmos. Meas. Tech., 15, 1303–1331, https://doi.org/10.5194/amt-15-1303-2022, https://doi.org/10.5194/amt-15-1303-2022, 2022
Short summary
Short summary
The article discusses modifications in the wind retrieval of the ALADIN Airborne Demonstrator (A2D) – one of the key instruments for the validation of Aeolus. Thanks to the retrieval refinements, which are demonstrated in the context of two airborne campaigns in 2019, the systematic and random wind errors of the A2D were significantly reduced, thereby enhancing its validation capabilities. Finally, wind comparisons between A2D and Aeolus for the validation of the satellite data are presented.
Terhikki Manninen, Emmihenna Jääskeläinen, Niilo Siljamo, Aku Riihelä, and Karl-Göran Karlsson
Atmos. Meas. Tech., 15, 879–893, https://doi.org/10.5194/amt-15-879-2022, https://doi.org/10.5194/amt-15-879-2022, 2022
Short summary
Short summary
A new method for cloud-correcting observations of surface albedo is presented for AVHRR data. Instead of a binary cloud mask, it applies cloud probability values smaller than 20% of the A3 edition of the CLARA (CM SAF cLoud, Albedo and surface Radiation dataset from AVHRR data) record provided by the Satellite Application Facility on Climate Monitoring (CM SAF) project of EUMETSAT. According to simulations, the 90% quantile was 1.1% for the absolute albedo error and 2.2% for the relative error.
Cited articles
Aggarwal, Ch. (Ed.): Data Streams – Models and Algorithms, Springer, 2007.
Allison, P.: Missing Data, Sage Publications, 2001.
Babcock, B., Babu, S., Datar, M., Motwani, R., and Widom, J.: Models and Issues in Data Stream Systems, in: Proc. of the 21st ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS, 1–16, 2002.
Bacher, P., Madsen, H., and Nielsen, H. A.: Online short-term solar power forecasting, Sol. Energy, 83, 1772–1783, 2009.
Bhardwaj, S., Sharma, V., Srivastava, S., Sastry, O., Bandyopadhyay, B., Chandel, S., and Gupta, J.: Estimation of solar radiation using a combination of Hidden Markov model and generalized Fuzzy model, Sol. Energy, 93, 43–54, 2013.
Black, C., Broadstock, D., Colin, A., and Hunt, L. C.: Filling in the gaps in transport studies: a practical guide to developments in data imputation methods, Traffic Eng. Control, 48, 358–363, 2007.
Enders, C. K.: Applied Missing Data Analysis, Guilford Press, 2010.
Hammer, A., Heinemann, D., Lorenz, E., and Lückehe, B.: Short-term forecasting of solar radiation: a statistical approach using satellite data, Sol. Energy, 67, 139–150, 1999.
Hari, P. and Kulmala, M.: Station for Measuring Ecosystem-Atmosphere Relations (SMEAR II), Boreal Environ. Res., 10, 315–322, 2005.
Hastie, T., Tibshirani, R., and Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer-Verlag, 2001.
Hoerl, A. E. and Kennard, R. W.: Ridge regression: biased estimation for nonorthogonal problems, Technometrics, 12, 55–67, 1970.
Hrust, L., Klaic, Z. B., Krizana, J., Antonic, O., and Hercog, P.: Neural network forecasting of air pollutants hourly concentrations using optimised temporal averages of meteorological variables and pollutant concentrations, Atmos. Environ., 43, 5588–5596, 2009.
Jolliffe, I. T.: Principal Component Analysis, 2nd Edn., Springer, 2002.
Junninen, H., Niska, H., Tuppurainen, K., Ruuskanen, J., and Kolehmainen, M.: Methods for imputation of missing values in air quality data sets, Atmos. Environ., 38, 2895–2907, 2004.
Junninen, H., Lauri, A., Keronen, P., Aalto, P., Hiltunen, V., Hari, P., and Kulmala, M.: Smart-SMEAR: on-line data exploration and visualization tool for SMEAR stations, Boreal Environ. Res., 14, 447–457, 2009.
Kadlec, P., Gabrys, B., and Strandt, S.: Data-driven soft sensors in the process industry, Comput. Chem. Eng., 33, 795–814, 2009.
Kopp, G. and Lean, J. L.: A new, lower value of total solar irradiance: Evidence and climate significance, Geophys. Res. Lett., 38, L01706, https://doi.org/10.1029/2010GL045777, 2011.
Lerner, U., Moses, B., Maricia, S., McIlraith, Sh. A., and Koller, D.: Monitoring a Complez Physical System using a Hybrid Dynamic Bayes Net., in: Proc. of the the 18th Conference in Uncertainty in Artificial Intelligence, UAI, 301–310, 2002.
Lu, H., Hsieh, J., and Chang, T.: Prediction of daily maximum ozone concentrations from meteorological conditions using a two-stage neural network, Atmos. Res., 81, 124–139, 2006.
Marquez, R. and Coimbra, C. F.: Forecasting of global and direct solar irradiance using stochastic learning methods, ground experiments and the NWS database, Sol. Energy, 85, 746–756, 2011.
Menut, L. and Bessagnet, B.: Atmospheric composition forecasting in Europe, Ann. Geophys., 28, 61–74, https://doi.org/10.5194/angeo-28-61-2010, 2010.
Michalsky, J.: The Astronomical Almanac's algorithm for approximate solar position (1950–2050), Sol. Energy, 40, 227–235, 1988.
Ramoni, M. and Sebastiani, P.: Robust Learning with Missing Data, Machine Learning, 45, 147–170, 2001.
Valle, S., Li, W., and Qin, S. J.: Selection of the number of principal components: the variance of the reconstruction error criterion with a comparison to other methods, Ind. Eng. Chem. Res., 38, 4389–4401, 1999.
Vuilleumier, L., Calpini, B., Cattin, R., Roulet, Y.-A., Stauch, V., Stöckli, R., and Giunta, I.: Solar radiation now-casting: the need for multiple data source integration, in: Proc. of the COST Action ES1002 "WIRE" State of the Art Workshop, available at: http://www.wire1002.ch/fileadmin/user_upload/Major_events/WS_Nice_2011/Spec._presentations/Vuillemier.pdf (last access: 14 July 2014), 2011.
Wold, S., Sjostroma, M., and Eriksson, L.: PLS-regression: a basic tool of chemometrics, Chemometr. Intell. Lab., 58, 109–130, 2001.
Žliobait\.e, I. and Hollmén, J.: Fault tolerant regression for sensor data, in: Proc. of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECMLPKDD, 449–464, 2013.
Žliobait\.e, I. and Hollmén, J.: Optimizing regression models for data streams with missing values, Mach. Learn., 1–27, https://doi.org/10.1007/s10994-014-5450-3, 2014.
Short summary
We present a case study in solar/radiation nowcasting using environmental sensor measurements as inputs. While some sensor readings may oftentimes be missing, predictions need to be output continuously in near real time. We are after linear regression models that would be robust to missing data, i.e., that would perform well with or without data gaps. We recommend using regularized a PCA regression with our established guidelines for building robust regression models.
We present a case study in solar/radiation nowcasting using environmental sensor measurements as...
