Surveillance Camera-Based Deep Learning Framework for High-Resolution Ground Hydrometeor Phase Observation

Abstract. Urban surveillance cameras offer a valuable resource for high spatiotemporal resolution observations of ground hydrometeor phase (GHP), with significant implications for sectors such as transportation, agriculture, and meteorology. However, distinguishing between common GHPs—rain, snow, and graupel—present considerable challenges due to their visual similarities in surveillance videos. This study addresses these challenges by analyzing both daytime and nighttime videos, leveraging meteorological, optical, and imaging principles to identify distinguishing features for each GHP. Considering both computational accuracy and efficiency, a new deep learning framework is proposed. It leverages transfer learning with a pre-trained MobileNet V2 for spatial feature extraction and incorporates a Gated Recurrent Unit network to model temporal dependencies between video frames. Using the newly developed 94-hour Hydrometeor Phase Surveillance Video (HSV) dataset, the proposed model is trained and evaluated alongside 24 comparative algorithms. Results show that our proposed method achieves an accuracy of 0.9677 on the HSV dataset, outperforming all other relevant algorithms. Furthermore, in real-world experiments, the proposed model achieves an accuracy of 0.9301, as validated against manually corrected Two-Dimensional Video Disdrometer measurements. It remains robust against variations in camera parameters, maintaining consistent performance in both daytime and nighttime conditions, and demonstrates wind resistance with satisfactory results when wind speeds are below 5 m/s. These findings highlight the model's suitability for large-scale, practical deployment in urban environments. Overall, this study demonstrates the feasibility of using low-cost surveillance cameras to build an efficient GHP monitoring network, potentially enhancing urban precipitation observation capabilities in a cost-effective manner.