Articles | Volume 18, issue 9
https://doi.org/10.5194/amt-18-2083-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/amt-18-2083-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Research article
| 
12 May 2025
Research article |
| 12 May 2025
| 12 May 2025
Infrared radiometric image classification and segmentation of cloud structures using a deep-learning framework from ground-based infrared thermal camera observations
Kélian Sommer
CORRESPONDING AUTHOR
Laboratoire Univers et Particules de Montpellier, Université de Montpellier, CNRS, Montpellier, France
Wassim Kabalan
CORRESPONDING AUTHOR
Astroparticule et Cosmologie, Université Paris Cité, CNRS, 75013 Paris, France
Romain Brunet
CORRESPONDING AUTHOR
Aix-Marseille Université, CNRS, CNES, LAM, Marseille, France
Cited articles
Aebi, C., Gröbner, J., and Kämpfer, N.: Cloud fraction determined by thermal infrared and visible all-sky cameras, Atmos. Meas. Tech., 11, 5549–5563, https://doi.org/10.5194/amt-11-5549-2018, 2018. a
Agarap, A. F.: Deep learning using rectified linear units (relu), arXiv [preprint], https://doi.org/10.48550/arXiv.1803.08375, 22 March 2018. a
Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M.: Optuna: A Next-generation Hyperparameter Optimization Framework, arXiv [preprint], https://doi.org/10.48550/arXiv.1907.10902, 25 July 2019. a
Akula, A., Ghosh, R., and Sardana, H.: Thermal imaging and its application in defence systems, AIP Conf. Proc., 1391, 333–335, 2011. a
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A. Q., Duan, Y., Al-Shamma, O., Santamaría, J. I., Fadhel, M. A., Al-Amidie, M., and Farhan, L.: Review of deep learning: concepts, CNN architectures, challenges, applications, future directions, Journal of Big Data, 8, 53, https://doi.org/10.1186/s40537-021-00444-8, 2021. a
Astropy Collaboration, T.: Astropy: A community Python package for astronomy, Astron. Astrophys., 558, A33, https://doi.org/10.1051/0004-6361/201322068, 2013. a, b
Badrinarayanan, V., Kendall, A., and Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE T. Pattern Anal., 39, 2481–2495, 2017. a
Bates, D.: Rayleigh scattering by air, Planet. Space Sci., 32, 785–790, 1984. a
Betoule, M., Antier, S., Bertin, E., Éric Blanc, P., Bongard, S., Cohen Tanugi, J., Dagoret-Campagne, S., Feinstein, F., Hardin, D., Juramy, C., Le Guillou, L., Le Van Suu, A., Moniez, M., Neveu, J., Nuss, É., Plez, B., Regnault, N., Sepulveda, E., Sommer, K., Souverin, T., and Wang, X. F.: StarDICE I: sensor calibration bench and absolute photometric calibration of a Sony IMX411 sensor, arXiv [preprint], https://doi.org/10.48550/arXiv.2211.04913, 9 November 2022. a, b
Boers, R., Acarreta, J. R., and Gras, J. L.: Satellite monitoring of the first indirect aerosol effect: Retrieval of the droplet concentration of water clouds, J. Geophys. Res.-Atmos., 111, D22208, https://doi.org/10.1029/2005JD006838, 2006. a
Bohlin, R. C.: Hubble Space Telescopecalspec Flux Standards: Sirius (And Vega), Astron. J., 147, 127, https://doi.org/10.1088/0004-6256/147/6/127, 2014. a
Bohlin, R. C., Gordon, K. D., Rieke, G. H., Ardila, D., Carey, S., Deustua, S., Engelbracht, C., Ferguson, H. C., Flanagan, K., Kalirai, J., Meixner, M., Noriega-Crespo, A., Su, K. Y. L., and Tremblay, P.-E.: Absolute Flux Calibration Of The Irac Instrument On The Spitzer Space Telescope Using Hubble Space Telescope Flux Standards, Astron. J., 141, 173, https://doi.org/10.1088/0004-6256/141/5/173, 2011. a
Bohlin, R. C., Hubeny, I., and Rauch, T.: New Grids of Pure-hydrogen White Dwarf NLTE Model Atmospheres and the HST/STIS Flux Calibration, Astron. J., 160, 21, https://doi.org/10.3847/1538-3881/ab94b4, 2020. a
Bower, K. N., Choularton, T. W., Gallagher, M. W., Beswick, K. M., Flynn, M. J., Allen, A. G., Davison, B. M., James, J. D., Robertson, L., Harrison, R. M., Hewitt, C. N., Cape, J. N., McFadyen, G. G., Milford, C., Sutton, M. A., Martinsson, B. G., Frank, G., Swietlicki, E., Zhou, J., Berg, O. H., Mentes, B., Papaspiropoulos, G., Hansson, H.-C., Leck, C., Kulmala, M., Aalto, P., Väkevä, M., Berner, A., Bizjak, M., Fuzzi, S., Laj, P., Facchini, M.-C., Orsi, G., Ricci, L., Nielsen, M., Allan, B. J., Coe, H., McFiggans, G., Plane, J. M. C., Collett Jr., J. L., Moore, K. F., and Sherman, D. E.: ACE-2 HILLCLOUD. An overview of the ACE-2 ground-based cloud experiment, Tellus B, 52, 750–778, 2000. a
Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., and Zhang, Q.: JAX: composable transformations of Python+NumPy programs, GitHub [code], http://github.com/google/jax (last access: 1 January 2024), 2018. a, b
Bradski, G.: The OpenCV Library, Dr. Dobb's Journal: Software Tools for the Professional Programmer, Miller Freeman Inc., 25, 120–123, 2000. a
Brout, D., Taylor, G., Scolnic, D., Wood, C. M., Rose, B. M., Vincenzi, M., Dwomoh, A., Lidman, C., Riess, A., Ali, N., Qu, H., and Dai, M.: The Pantheon+ Analysis: SuperCal-fragilistic Cross Calibration, Retrained SALT2 Light-curve Model, and Calibration Systematic Uncertainty, Astrophys. J., 938, 111, https://doi.org/10.3847/1538-4357/ac8bcc, 2022. a
Burke, D. L., Axelrod, T., Blondin, S., Claver, C., Željko Ivezić, Jones, L., Saha, A., Smith, A., Smith, R. C., and Stubbs, C. W.: Precision Determination Of Atmospheric Extinction At Optical And Near-Infrared Wavelengths, Astrophys. J., 720, 811, https://doi.org/10.1088/0004-637X/720/1/811, 2010. a, b
Burke, D. L., Saha, A., Claver, J., Axelrod, T., Claver, C., DePoy, D., Ivezić, Ž ., Jones, L., Smith, R. C., and Stubbs, C. W.: All-Weather Calibration Of Wide-Field Optical And Nir Surveys, Astron. J., 147, 19, https://doi.org/10.1088/0004-6256/147/1/19, 2013. a
Burke, D. L., Rykoff, E. S., Allam, S., Annis, J., Bechtol, K., Bernstein, G. M., Drlica-Wagner, A., Finley, D. A., Gruendl, R. A., James, D. J., Kent, S., Kessler, R., Kuhlmann, S., Lasker, J., Li, T. S., Scolnic, D., Smith, J., Tucker, D. L., Wester, W., Yanny, B., Abbott, T. M. C., Abdalla, F. B., Benoit-Lé vy, A., Bertin, E., Rosell, A. C., Kind, M. C., Carretero, J., Cunha, C. E., D'Andrea, C. B., da Costa, L. N., Desai, S., Diehl, H. T., Doel, P., Estrada, J., García-Bellido, J., Gruen, D., Gutierrez, G., Honscheid, K., Kuehn, K., Kuropatkin, N., Maia, M. A. G., March, M., Marshall, J. L., Melchior, P., Menanteau, F., Miquel, R., Plazas, A. A., Sako, M., Sanchez, E., Scarpine, V., Schindler, R., Sevilla-Noarbe, I., Smith, M., Smith, R. C., Soares-Santos, M., Sobreira, F., Suchyta, E., Tarle, G., and Walker, A. R.: Forward Global Photometric Calibration of the Dark Energy Survey, Astron. J., 155, 41, https://doi.org/10.3847/1538-3881/aa9f22, 2017. a
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. L.: DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs, IEEE T. Pattern Anal., 40, 834–848, 2017. a
Conley, A., Guy, J., Sullivan, M., Regnault, N., Astier, P., Balland, C., Basa, S., Carlberg, R. G., Fouchez, D., Hardin, D., Hook, I. M., Howell, D. A., Pain, R., Palanque-Delabrouille, N., Perrett, K. M., Pritchet, C. J., Rich, J., Ruhlmann-Kleider, V., Balam, D., Baumont, S., Ellis, R. S., Fabbro, S., Fakhouri, H. K., Fourmanoit, N., González-Gaitán, S., Graham, M. L., Hudson, M. J., Hsiao, E., Kronborg, T., Lidman, C., Mourao, A. M., Neill, J. D., Perlmutter, S., Ripoche, P., Suzuki, N., and Walker, E. S.: Supernova Constraints and Systematic Uncertainties from the First Three Years of the Supernova Legacy Survey, American Astronomical Society, 192, 1, https://doi.org/10.1088/0067-0049/192/1/1, 2011. a
Cortes, C. and Vapnik, V.: Support-vector networks, Mach. Learn., 20, 273–297, 1995. a
Currie, M., Rubin, D., Aldering, G., Deustua, S., Fruchter, A., and Perlmutter, S.: Evaluating the Calibration of SN Ia Anchor Datasets with a Bayesian Hierarchical Model, arXiv [preprint], https://doi.org/10.48550/arXiv.2007.02458, 5 July 2020. a
Dark Energy Survey Collaboration, Abbott, T., Abdalla, F. B., Aleksić, J., Allam, S., Amara, A., Bacon, D., Balbinot, E., Banerji, M., Bechtol, K., Benoit-Lévy, A., Bernstein, G. M., Bertin, E., Blazek, J., Bonnett, C., Bridle, S., Brooks, D., Brunner, R. J., Buckley-Geer, E., Burke, D. L., Caminha, G. B., Capozzi, D., Carlsen, J., Carnero-Rosell, A., Carollo, M., Carrasco-Kind, M., Carretero, J., Castander, F. J., Clerkin, L., Collett, T., Conselice, C., Crocce, M., Cunha, C. E., D'Andrea, C. B., da Costa, L. N., Davis, T. M., Desai, S., Diehl, H. T., Dietrich, J. P., Dodelson, S., Doel, P., Drlica-Wagner, A., Estrada, J., Etherington, J., Evrard, A. E., Fabbri, J., Finley, D. A., Flaugher, B., Foley, R. J., Fosalba, P., Frieman, J., García-Bellido, J., Gaztanaga, E., Gerdes, D. W., Giannantonio, T., Goldstein, D. A., Gruen, D., Gruendl, R. A., Guarnieri, P., Gutierrez, G., Hartley, W., Honscheid, K., Jain, B., James, D. J., Jeltema, T., Jouvel, S., Kessler, R., King, A., Kirk, D., Kron, R., Kuehn, K., Kuropatkin, N., Lahav, O., Li, T. S., Lima, M., Lin, H., Maia, M. A. G., Makler, M., Manera, M., Maraston, C., Marshall, J. L., Martini, P., McMahon, R. G., Melchior, P., Merson, A., Miller, C. J., Miquel, R., Mohr, J. J., Morice-Atkinson, X., Naidoo, K., Neilsen, E., Nichol, R. C., Nord, B., Ogando, R., Ostrovski, F., Palmese, A., Papadopoulos, A., Peiris, H. V., Peoples, J., Percival, W. J., Plazas, A. A., Reed, S. L., Refregier, A., Romer, A. K., Roodman, A., Ross, A., Rozo, E., Rykoff, E. S., Sadeh, I., Sako, M., Sánchez, C., Sanchez, E., Santiago, B., Scarpine, V., Schubnell, M., Sevilla-Noarbe, I., Sheldon, E., Smith, M., Smith, R. C., Soares-Santos, M., Sobreira, F., Soumagnac, M., Suchyta, E., Sullivan, M., Swanson, M., Tarle, G., Thaler, J., Thomas, D., Thomas, R. C., Tucker, D., Vieira, J. D., Vikram, V., Walker, A. R., Wechsler, R. H., Weller, J., Wester, W., Whiteway, L., Wilcox, H., Yanny, B., Zhang, Y., and Zuntz, J.: The Dark Energy Survey: more than dark energy – an overview, Mon. Not. R. Astron. Soc., 460, 1270–1299, https://doi.org/10.1093/mnras/stw641, 2016. a
Dev, S., Savoy, F. M., Lee, Y. H., and Winkler, S.: Nighttime sky/cloud image segmentation, IEEE Press, 345–349, https://doi.org/10.1109/ICIP.2017.8296300, 2017. a, b
Dev, S., Savoy, F., Lee, Y. H., and Winkler, S.: Singapore Whole sky Nighttime Image SEGmentation Database, IEEE Dataport [data set], https://doi.org/10.21227/jsf0-ga67, 2019b. a, b
Fa, T., Xie, W., Yiren, W., and Xia, Y.: Development of an all-sky imaging system for cloud cover assessment, Appl. Optics, 58, 5516, https://doi.org/10.1364/AO.58.005516, 2019. a
Funke, I., Rivoir, D., Krell, S., and Speidel, S.: TUNeS: A Temporal U-Net with Self-Attention for Video-based Surgical Phase Recognition, arXiv [preprint], https://doi.org/10.48550/arXiv.2307.09997, 19 July 2023. a
Gallo, M. A., Willits, D. S., Lubke, R. A., and Thiede, E. C.: Low-cost uncooled IR sensor for battlefield surveillance, in: Infrared Technology XIX, SPIE, vol. 2020, 351–362, 1993. a
Geer, A., Baordo, F., Bormann, N., Chambon, P., English, S., Kazumori, M., Lawrence, H., Lean, P., Lonitz, K., and Lupu, C.: The growing impact of satellite observations sensitive to humidity, cloud and precipitation, Q. J. Roy. Meteor. Soc., 143, 3189–3206, 2017. a
Goobar, A. and Leibundgut, B.: Supernova Cosmology: Legacy and Future, Annu. Rev. Nucl. Part. S., 61, 251–279, https://doi.org/10.1146/annurev-nucl-102010-130434, 2011. a
Hack, E. D., Pauliquevis, T., Barbosa, H. M. J., Yamasoe, M. A., Klebe, D., and Correia, A. L.: Precipitable water vapor retrievals using a ground-based infrared sky camera in subtropical South America, Atmos. Meas. Tech., 16, 1263–1278, https://doi.org/10.5194/amt-16-1263-2023, 2023. a
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M., Haldane, A., del Rio, J. F., Wiebe, M., Peterson, P., Gerard-Marchant, P., Sheppard, K., Reddy, T., Weckesser, W., Abbasi, H., Gohlke, C., and Oliphant, T. E.: Array programming with NumPy, Nature, 585, 357–362, https://doi.org/10.1038/s41586-020-2649-2, 2020. a
Hazenberg, F.: Calibration photométrique des supernovae de type Ia pour la caractérisation de l'énergie noire avec l'expérience StarDICE, PhD thesis, Sorbonne université, http://www.theses.fr/2019SORUS142 (last access: 1 January 2024), 2019. a
He, K., Zhang, X., Ren, S., and Sun, J.: Deep Residual Learning for Image Recognition, arXiv [preprint], https://doi.org/10.48550/arXiv.1512.03385, 10 December 2015. a
He, Y., Liu, P., Wang, Z., Hu, Z., and Yang, Y.: Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration, arXiv [preprint], https://doi.org/10.48550/arXiv.1811.00250, 1 November 2018. a
Hoerl, A. E. and Kennard, R. W.: Ridge regression: Biased estimation for nonorthogonal problems, Technometrics, 12, 55–67, 1970. a
Houghton, J. T. and Lee, A. C. L.: Atmospheric Transmission in the 10-12 µm Window, Nature Physical Science, 238, 117–118, https://doi.org/10.1038/physci238117a0, 1972. a
Hu, Y., Wielicki, B. A., Yang, P., Stackhouse, P. W., Lin, B., and Young, D. F.: Application of deep convective cloud albedo observation to satellite-based study of the terrestrial atmosphere: Monitoring the stability of spaceborne measurements and assessing absorption anomaly, IEEE T. Geosci. Remote, 42, 2594–2599, 2004. a
Hunter, J. D.: Matplotlib: A 2D Graphics Environment, Comput. Sci. Eng., 9, 90–95, https://doi.org/10.1109/MCSE.2007.55, 2007. a
Ishimwe, R., Abutaleb, K., Ahmed, F., Ishimwe, R., Abutaleb, K., and Ahmed, F.: Applications of thermal imaging in agriculture – A review, Adv. Remote Sens., 3, 128–140, https://doi.org/10.4236/ars.2014.33011, 2014. a
Itseez: Open Source Computer Vision Library, GitHub [code], https://github.com/itseez/opencv (last access: 1 January 2024), 2015. a
Kelsey, V., Riley, S., and Minschwaner, K.: Atmospheric precipitable water vapor and its correlation with clear-sky infrared temperature observations, Atmos. Meas. Tech., 15, 1563–1576, https://doi.org/10.5194/amt-15-1563-2022, 2022. a
Kimata, M.: Uncooled infrared focal plane arrays, IEEJ T. Electr. Electr., 13, 4–12, https://doi.org/10.1002/tee.22563, 2018. a
Kingma, D. P. and Ba, J.: Adam: A method for stochastic optimization, arXiv [preprint], https://doi.org/10.48550/arXiv.1412.6980, 22 December 2014. a, b
Klebe, D., Sebag, J., Blatherwick, R. D., and Zimmer, P. C.: All-Sky Mid-Infrared Imagery to Characterize Sky Conditions and Improve Astronomical Observational Performance, Publ. Astron. Soc. Pac., 124, 1309, https://doi.org/10.1086/668866, 2012. a
Klebe, D. I., Blatherwick, R. D., and Morris, V. R.: Ground-based all-sky mid-infrared and visible imagery for purposes of characterizing cloud properties, Atmos. Meas. Tech., 7, 637–645, https://doi.org/10.5194/amt-7-637-2014, 2014. a, b, c
Krauz, L., Janout, P., Blažek, M., and Páta, P.: Assessing Cloud Segmentation in the Chromacity Diagram of All-Sky Images, Remote Sens., 12, 1902, https://doi.org/10.3390/rs12111902, 2020. a
Larason, T. and Houston, J.: Spectroradiometric Detector Measurements: Ultraviolet, Visible, and Near Infrared Detectors for Spectral Power, NIST Special Publication 250-41, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=841061 (last access: 1 January 2024), 2008. a
Li, M., Liu, S., and Zhang, Z.: Deep tensor fusion network for multimodal ground-based cloud classification in weather station networks, Ad Hoc Netw., 96, 101991, https://doi.org/10.1016/j.adhoc.2019.101991, 2020. a
Li, Q., Lu, W., and Yang, J.: A Hybrid Thresholding Algorithm for Cloud Detection on Ground-Based Color Images, J. Atmos. Ocean. Tech., 28, 1286–1296, https://doi.org/10.1175/JTECH-D-11-00009.1, 2011. a, b
Li, S., Wang, M., Wu, J., Sun, S., and Zhuang, Z.: CloudDeepLabV3+: a lightweight ground-based cloud segmentation method based on multi-scale feature aggregation and multi-level attention feature enhancement, Int. J. Remote Sens., 44, 4836–4856, https://doi.org/10.1080/01431161.2023.2240034, 2023. a, b
Li, T. S., DePoy, D. L., Marshall, J. L., Tucker, D., Kessler, R., Annis, J., Bernstein, G. M., Boada, S., Burke, D. L., Finley, D. A., James, D. J., Kent, S., Lin, H., Marriner, J., Mondrik, N., Nagasawa, D., Rykoff, E. S., Scolnic, D., Walker, A. R., Wester, W., Abbott, T. M. C., Allam, S., Benoit-Lé vy, A., Bertin, E., Brooks, D., Capozzi, D., Rosell, A. C., Kind, M. C., Carretero, J., Crocce, M., Cunha, C. E., D'Andrea, C. B., da Costa, L. N., Desai, S., Diehl, H. T., Doel, P., Flaugher, B., Fosalba, P., Frieman, J., Gaztanaga, E., Goldstein, D. A., Gruen, D., Gruendl, R. A., Gutierrez, G., Honscheid, K., Kuehn, K., Kuropatkin, N., Maia, M. A. G., Melchior, P., Miller, C. J., Miquel, R., Mohr, J. J., Neilsen, E., Nichol, R. C., Nord, B., Ogando, R., Plazas, A. A., Romer, A. K., Roodman, A., Sako, M., Sanchez, E., Scarpine, V., Schubnell, M., Sevilla-Noarbe, I., Smith, R. C., Soares-Santos, M., Sobreira, F., Suchyta, E., Tarle, G., Thomas, D., and Vikram, V.: Assessment Of Systematic Chromatic Errors That Impact Sub-1 % Photometric Precision In Large-Area Sky Surveys, Astron. J., 151, 157, https://doi.org/10.3847/0004-6256/151/6/157, 2016. a
Li, X., Wang, B., Qiu, B., and Wu, C.: An all-sky camera image classification method using cloud cover features, Atmos. Meas. Tech., 15, 3629–3639, https://doi.org/10.5194/amt-15-3629-2022, 2022. a
Liandrat, O., Cros, S., Braun, A., Saint-Antonin, L., Decroix, J., and Schmutz, N.: Cloud cover forecast from a ground-based all sky infrared thermal camera, in: Remote Sensing of Clouds and the Atmosphere XXII, Proc. SPIE 10424, 10 pp., https://doi.org/10.1117/12.2278636, 2017a. a
Liou, K.-N.: Radiation and cloud processes in the atmosphere. Theory, observation, and modeling, Oxford University Press, New York, ISBN 978-0-19-504910-7, 1992. a
Liu, L., Sun, X., Chen, F., Zhao, S., and Gao, T.: Cloud Classification Based on Structure Features of Infrared Images, J. Atmos. Ocean. Tech., 28, 410–417, https://doi.org/10.1175/2010JTECHA1385.1, 2011. a, b
Liu, L., Jiang, H., He, P., Chen, W., Liu, X., Gao, J., and Han, J.: On the variance of the adaptive learning rate and beyond, arXiv [preprint], https://doi.org/10.48550/arXiv.1908.03265, 8 August 2019. a
Liu, S., Li, M., Zhang, Z., Xiao, B., and Cao, X.: Multimodal ground-based cloud classification using joint fusion convolutional neural network, Remote Sens., 10, 822, https://doi.org/10.3390/rs10060822, 2018. a
Liu, S., Zhang, J., Zhang, Z., Cao, X., and Durrani, T. S.: TransCloudSeg: Ground-Based Cloud Image Segmentation With Transformer, IEEE J. Sel. Top. Appl., 15, 6121–6132, https://doi.org/10.1109/JSTARS.2022.3194316, 2022. a, b
Liu, Y., Shao, Z., and Hoffmann, N.: Global Attention Module: Better Exploiting Non-local Similarity in Deep CNNs, arXiv [preprint], https://doi.org/10.48550/arXiv.2112.05561, 10 December 2021. a
Long, C. N., Sabburg, J. M., Calbó, J., and Pagès, D.: Retrieving Cloud Characteristics from Ground-Based Daytime Color All-Sky Images, J. Atmos. Ocean. Tech., 23, 633–652, https://doi.org/10.1175/JTECH1875.1, 2006. a
Lopez, T., Antoine, R., Baratoux, D., and Rabinowicz, M.: Contribution of thermal infrared images on the understanding of the subsurface/atmosphere exchanges on Earth., in: EGU General Assembly 2018, 8–13 April 2018, Vienna, Austria, EGU General Assembly Conference Abstracts, 19, EGU2017-11811, 2017. a
Loshchilov, I. and Hutter, F.: SGDR: Stochastic Gradient Descent with Warm Restarts, arXiv [preprint], https://doi.org/10.48550/arXiv.1608.03983, 23 February 2017. a
Luo, Q., Meng, Y., Liu, L., Zhao, X., and Zhou, Z.: Cloud classification of ground-based infrared images combining manifold and texture features, Atmos. Meas. Tech., 11, 5351–5361, https://doi.org/10.5194/amt-11-5351-2018, 2018. a, b
Mandat, D., Pech, M., Hrabovsky, M., Schovanek, P., Palatka, M., Travnicek, P., Prouza, M., and Ebr, J.: All Sky Camera instrument for night sky monitoring, arXiv [preprint], https://doi.org/10.48550/arXiv.1402.4762, 19 February 2014. a
Manzo, M. and Pellino, S.: Voting in transfer learning system for ground-based cloud classification, Machine Learning and Knowledge Extraction, 3, 542–553, 2021. a
Martin, R. V.: Satellite remote sensing of surface air quality, Atmos. Environ., 42, 7823–7843, 2008. a
McCallum, A.: Efficiently inducing features of conditional random fields, arXiv [preprint], https://doi.org/10.48550/arXiv.1212.2504, 19 October 2012. a
Mikołajczyk, A. and Grochowski, M.: Data augmentation for improving deep learning in image classification problem, in: 2018 International Interdisciplinary PhD Workshop (IIPhDW), 9–12 May 2018, Swinoujscie, Poland, 117–122, https://doi.org/10.1109/IIPHDW.2018.8388338, 2018. a
Mokhov, I. L. and Schlesinger, M. E.: Analysis of global cloudiness: 2. Comparison of ground-based and satellite-based cloud climatologies, J. Geophys. Res.-Atmos., 99, 17045–17065, 1994. a
Mommert, M.: Cloud Identification from All-sky Camera Data with Machine Learning, Astrophys. J., 159, 178, https://doi.org/10.3847/1538-3881/ab744f, 2020. a, b
Nikolenko, I. and Maslov, I.: Infrared (thermal) camera for monitoring the state of the atmosphere above the sea horizon of the Simeiz Observatory INASAN, INASAN Science Reports, 6, 85–87, 2021. a
O'Shea, K. and Nash, R.: An Introduction to Convolutional Neural Networks, arXiv [preprint], https://doi.org/10.48550/arXiv.1511.08458, 26 November 2015. a
Otsu, N.: A Threshold Selection Method from Gray-Level Histograms, IEEE T. Syst. Man Cyb., 9, 62–66, https://doi.org/10.1109/TSMC.1979.4310076, 1979. a, b
Paczyński, B.: Monitoring All Sky for Variability, Publ. Astron. Soc. Pac., 112, 1281, https://doi.org/10.1086/316623, 2000. a
Pandas Development Team: pandas-dev/pandas: Pandas, Zenodo [code], https://doi.org/10.5281/zenodo.3509134, 2020. a
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E.: Scikit-learn: Machine Learning in Python, J. Mach. Learn. Res., 12, 2825–2830, 2011. a, b
Perez, L. and Wang, J.: The Effectiveness of Data Augmentation in Image Classification using Deep Learning, arXiv [preprint], https://doi.org/10.48550/arXiv.1712.04621, 13 December 2017. a
Petzold, A., Marshall, J., Nédélec, P., Smit, H. G. J., Frieß, U., Flaud, J.-M., Wahner, A., Cammas, J.-P., Volz-Thomas, A., and the IAGOS Team: Global-scale atmosphere monitoring by in-service aircraft–current achievements and future prospects of the European Research Infrastructure IAGOS, Tellus B, 67, 28452, https://doi.org/10.3402/tellusb.v67.28452, 2015. a
Ring, E. and Ammer, K.: Infrared thermal imaging in medicine, Physiol. Meas., 33, R33–R46, 2012. a
Rogalski, A.: Recent progress in infrared detector technologies, Infrared Phys. Techn., 54, 136–154, https://doi.org/10.1016/j.infrared.2010.12.003, 2011. a
Rogalski, A. and Chrzanowski, K.: Infrared Devices And Techniques (Revision), Metrol. Meas. Syst., 21, 565–618, https://doi.org/10.2478/mms-2014-0057, 2014. a
Román, R., Antuña-Sánchez, J. C., Cachorro, V. E., Toledano, C., Torres, B., Mateos, D., Fuertes, D., López, C., González, R., Lapionok, T., Herreras-Giralda, M., Dubovik, O., and de Frutos, Á. M.: Retrieval of aerosol properties using relative radiance measurements from an all-sky camera, Atmos. Meas. Tech., 15, 407–433, https://doi.org/10.5194/amt-15-407-2022, 2022. a
Ronneberger, O., Fischer, P., and Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation, arXiv [preprint], https://doi.org/10.48550/arXiv.1505.04597, 18 May 2015. a, b, c, d
Roy, P., Behera, M., and Srivastav, S.: Satellite remote sensing: sensors, applications and techniques, P. Natl. A. Sci. India A, 87, 465–472, 2017. a
Rubin, D., Aldering, G., Amanullah, R., Barbary, K., Dawson, K. S., Deustua, S., Faccioli, L., Fadeyev, V., Fakhouri, H. K., Fruchter, A. S., Gladders, M. D., de Jong, R. S., Koekemoer, A., Krechmer, E., Lidman, C., Meyers, J., Nordin, J., Perlmutter, S., Ripoche, P., Schlegel, D. J., Spadafora, A., and Suzuki, N.: A Calibration Of Nicmos Camera 2 For Low Count Rates, Astron. J., 149, 159, https://doi.org/10.1088/0004-6256/149/5/159, 2015. a
Rubin, D., Aldering, G., Antilogus, P., Aragon, C., Bailey, S., Baltay, C., Bongard, S., Boone, K., Buton, C., Copin, Y., Dixon, S., Fouchez, D., Gangler, E., Gupta, R., Hayden, B., Hillebrandt, W., Kim, A. G., Kowalski, M., Kuesters, D., Leget, P. F., Mondon, F., Nordin, J., Pain, R., Pecontal, E., Pereira, R., Perlmutter, S., Ponder, K. A., Rabinowitz, D., Rigault, M., Runge, K., Saunders, C., Smadja, G., Suzuki, N., Tao, C., Taubenberger, S., Thomas, R. C., Vincenzi, M., and Factory, T. N. S.: The LSST DESC DC2 Simulated Sky Survey, arXiv [preprint], https://doi.org/10.48550/arXiv.2205.01116, 21 June 2022. a
Salamalikis, V., Tzoumanikas, P., Argiriou, A. A., and Kazantzidis, A.: Estimation of Precipitable Water Using Thermal Infrared Images, Environmental Sciences Proceedings, 26, 33, https://doi.org/10.3390/environsciproc2023026033, 2023. a
Schiffer, R. A. and Rossow, W. B.: The International Satellite Cloud Climatology Project (ISCCP): The first project of the world climate research programme, B. Am. Meteorol. Soc., 64, 779–784, 1983. a
Schreiner, A. J., Unger, D. A., Menzel, W. P., Ellrod, G. P., Strabala, K. I., and Pellet, J. L.: A comparison of ground and satellite observations of cloud cover, B. Am. Meteorol. Soc., 74, 1851–1862, 1993. a
Scolnic, D., Casertano, S., Riess, A., Rest, A., Schlafly, E., Foley, R. J., Finkbeiner, D., Tang, C., Burgett, W. S., Chambers, K. C., Draper, P. W., Flewelling, H., Hodapp, K. W., Huber, M. E., Kaiser, N., Kudritzki, R. P., Magnier, E. A., Metcalfe, N., and Stubbs, C. W.: Supercal: Cross-Calibration Of Multiple Photometric Systems To Improve Cosmological Measurements With Type Ia Supernovae, Astrophys. J., 815, 117, https://doi.org/10.1088/0004-637X/815/2/117, 2015. a
Sebag, J., Andrew, J., Klebe, D., and Blatherwick, R.: LSST all-sky IR camera cloud monitoring test results, Proc. SPIE, 7733, 773348, https://doi.org/10.1117/12.856337, 2010. a
Sharma, T., Parihar, P., and Kemkar, M.: All sky scanning cloud monitor for NLOT site survey, J. Phys. Conf. Ser., 595, 012032, https://doi.org/10.1088/1742-6596/595/1/012032, 2015. a
Shaw, J. A. and Nugent, P. W.: Physics principles in radiometric infrared imaging of clouds in the atmosphere, Eur. J. Phys., 34, S111–S121, https://doi.org/10.1088/0143-0807/34/6/s111, 2013. a, b
Sherstinsky, A.: Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network, Physica D, 404, 132306, https://doi.org/10.1016/j.physd.2019.132306, 2020. a
Shi, C., Zhou, Y., Qiu, B., Guo, D., and Li, M.: CloudU-Net: A Deep Convolutional Neural Network Architecture for Daytime and Nighttime Cloud Images' Segmentation, IEEE Geosci. Remote S., 18, 1688–1692, https://doi.org/10.1109/LGRS.2020.3009227, 2021b. a, b
Skidmore, W., Els, S., Travouillon, T., Riddle, R., Seguel, J., Bustos, E., Walker, D., and Blum, R.: Using All Sky Cameras to determine cloud statistics for the Thirty Meter Telescope candidate sites, in: Ground-based and Airborne Telescopes II, SPIE, vol. 7012, 862–870, 2008. a
Smith, S. and Toumi, R.: Measuring Cloud Cover and Brightness Temperature with a Ground-Based Thermal Infrared Camera, J. Appl. Meteorol. Clim., 47, 683–693, https://doi.org/10.1175/2007JAMC1615.1, 2008. a
Sommer, K.: Kelian98/tau2_thermalcapture: Release for Zenodo (main), Zenodo [code], https://doi.org/10.5281/zenodo.15311830, 2025. a, b
Sommer, K., Brunet, R., and Kabalan, W.: ASKabalan/infrared-cloud-detection: v0.1.0 (v0.1.0), Zenodo [code], https://doi.org/10.5281/zenodo.15316607, 2025. a
Stubbs, C. W. and Brown, Y. J.: Precise astronomical flux calibration and its impact on studying the nature of the dark energy, Mod. Phys. Lett. A, 30, 1530030, https://doi.org/10.1142/S021773231530030X, 2015. a
Stubbs, C. W. and Tonry, J. L.: The Calibration of Astronomical Observations Made Through the Atmosphere, arXiv [preprint], https://doi.org/10.48550/arXiv.1206.6695, 28 June 2012. a
Sun, X., Liu, L., and Zhao, S.: Whole Sky Infrared Remote Sensing of Cloud, Proced. Earth Plan. Sc., 2, 278–283, https://doi.org/10.1016/j.proeps.2011.09.044, 2011. a, b
Sun, X.-J., Gao, T.-C., Zhai, D.-L., Zhao, S.-J., and Lian, J.-G.: Whole sky infrared cloud measuring system based on the uncooled infrared focal plane array, Infrared and Laser Engineering, 37, 761–764, 2008. a
Szejwach, G.: Determination of Semi-Transparent Cirrus Cloud Temperature from Infrared Radiances: Application to METEOSAT, J. Appl. Meteorol. Clim., 21, 384–393, https://doi.org/10.1175/1520-0450(1982)021<0384:DOSTCC>2.0.CO;2, 1982. a
The Astropy Collaboration: Astropy: A community Python package for astronomy, Astron. Astrophys., 558, A33, https://doi.org/10.3847/1538-3881/aac387, 2018. a, b
Thurairajah, B. and Shaw, J.: Cloud statistics measured with the infrared cloud imager (ICI), IEEE T. Geosci. Remote, 43, 2000–2007, https://doi.org/10.1109/TGRS.2005.853716, 2005. a, b
TLCDD: shuangliutjnu/TJNU-Large-Scale-Cloud-Detection-Database: v1.0.0, Zenodo [code], https://doi.org/10.5281/zenodo.6464743, 2022. a
Tzoumanikas, P., Nikitidou, E., Bais, A., and Kazantzidis, A.: The effect of clouds on surface solar irradiance, based on data from an all-sky imaging system, Renew. Energ., 95, 314–322, 2016. a
Ugolnikov, O. S., Galkin, A. A., Pilgaev, S. V., and Roldugin, A. V.: Noctilucent cloud particle size determination based on multi-wavelength all-sky analysis, Planet. Space Sci., 146, 10–19, 2017. a
Vadera, S. and Ameen, S.: Methods for Pruning Deep Neural Networks, arXiv [preprint], https://doi.org/10.48550/arXiv.2011.00241, 31 October 2020. a
Várnai, T. and Marshak, A.: Satellite observations of cloud-related variations in aerosol properties, Atmosphere, 9, 430, https://doi.org/10.3390/atmos9110430, 2018. a
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I.: Attention Is All You Need, in: Advances in Neural Information Processing Systems, edited by: Guyon, I., Von Luxburg, U., Bengio,S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., Curran Associates, Inc., https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf (last access: 1 January 2024), 2017. a
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., Carey, C. J., Polat, İ., Feng, Y., Moore, E. W., VanderPlas, J., Laxalde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E. A., Harris, C. R., Archibald, A. M., Ribeiro, A. H., Pedregosa, F., van Mulbregt, P., and SciPy 1.0 Contributors: SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python, Nat. Methods, 17, 261–272, https://doi.org/10.1038/s41592-019-0686-2, 2020. a
Wang, M., Zhuang, Z., Wang, K., Zhou, S., and Liu, Z.: Intelligent classification of ground-based visible cloud images using a transfer convolutional neural network and fine-tuning, Opt. Express, 29, 41176–41190, https://doi.org/10.1364/OE.442455, 2021a. a
Wang, Y., Liu, D., Xie, W., Yang, M., Gao, Z., Ling, X., Huang, Y., Li, C., Liu, Y., and Xia, Y.: Day and Night Clouds Detection Using a Thermal-Infrared All-Sky-View Camera, Remote Sens., 13, 1852, https://doi.org/10.3390/rs13091852, 2021b. a, b
Wei, L., Zhu, T., Guo, Y., and Ni, C.: MMST: A Multi-Modal Ground-Based Cloud Image Classification Method, Sensors, 23, 4222, https://doi.org/10.3390/s23094222, 2023. a
Wells, D. C., Greisen, E. W., and Harten, R. H.: FITS – a Flexible Image Transport System, Astronomy and Astrophysics Supplement, 44, 363–370, 1981. a
Wong, W. K., Tan, P. N., Loo, C. K., and Lim, W. S.: An effective surveillance system using thermal camera, in: 2009 International Conference on Signal Acquisition and Processing, 3–5 April 2009, Kuala Lumpur, Malaysia, 13–17, https://doi.org/10.1109/ICSAP.2009.12, 2009. a
Xie, W., Liu, D., Yang, M., Chen, S., Wang, B., Wang, Z., Xia, Y., Liu, Y., Wang, Y., and Zhang, C.: SegCloud: a novel cloud image segmentation model using a deep convolutional neural network for ground-based all-sky-view camera observation, Atmos. Meas. Tech., 13, 1953–1961, https://doi.org/10.5194/amt-13-1953-2020, 2020. a, b, c, d, e
Yamashita, M. and Yoshimura, M.: Ground-based cloud observation for satellite-based cloud discrimination and its validation, Int. Arch. Photogramm., 39, 137–140, 2012. a
Yang, J., Lu, W., Ma, Y., and Yao, W.: An Automated Cirrus Cloud Detection Method for a Ground-Based Cloud Image, J. Atmos. Ocean. Tech., 29, 527–537, https://doi.org/10.1175/JTECH-D-11-00002.1, 2012. a
Yang, S., Xiao, W., Zhang, M., Guo, S., Zhao, J., and Shen, F.: Image Data Augmentation for Deep Learning: A Survey, arXiv [preprint], https://doi.org/10.48550/arXiv.2204.08610, 19 April 2022. a
Ye, L., Cao, Z., and Xiao, Y.: DeepCloud: Ground-Based Cloud Image Categorization Using Deep Convolutional Features, IEEE T. Geosci. Remote, 55, 5729–5740, https://doi.org/10.1109/TGRS.2017.2712809, 2017. a, b
Yoshimura, M. and Yamashita, M.: Contribution of ground-based cloud observation to satellite-based cloud discrimination, J. Environ. Sci. Eng. A, 2, 379–382, 2013. a
Zhang, Z., Yang, S., Liu, S., Cao, X., and Durrani, T. S.: Ground-Based Remote Sensing Cloud Detection Using Dual Pyramid Network and Encoder–Decoder Constraint, IEEE T. Geosci. Remote, 60, 1–10, https://doi.org/10.1109/TGRS.2022.3163917, 2022. a
Zhao, C., Zhang, X., Luo, H., Zhong, S., Tang, L., Peng, J., and Fan, J.: Detail-Aware Multiscale Context Fusion Network for Cloud Detection, IEEE Geosci. Remote S., 19, 1–5, https://doi.org/10.1109/LGRS.2022.3207426, 2022. a
Zhou, Q., Zhang, Y., Li, B., Li, L., Feng, J., Jia, S., Lv, S., Tao, F., and Guo, J.: Cloud-base and cloud-top heights determined from a ground-based cloud radar in Beijing, China, Atmos. Environ., 201, 381–390, 2019. a
Zhou, Z., Zhang, F., Xiao, H., Wang, F., Hong, X., Wu, K., and Zhang, J.: A novel ground-based cloud image segmentation method by using deep transfer learning, IEEE Geosci. Remote S., 19, 1–5, 2021. a
Short summary
Our research introduces a novel deep-learning approach for classifying and segmenting ground-based infrared thermal images, a crucial step in cloud monitoring. Tests based on self-captured data showcase its excellent accuracy in distinguishing image types and in structure segmentation. With potential applications in astronomical observations, our work pioneers a robust solution for ground-based sky quality assessment, promising advancements in the photometric observation experiments.
Our research introduces a novel deep-learning approach for classifying and segmenting...
Altmetrics
Final-revised paper
Preprint