A data-driven persistence test for robust (probabilistic) quality control of measured environmental time series: constant value episodes

Kaffashzadeh, Najmeh

doi:https://doi.org/10.5194/amt-16-3085-2023

Articles | Volume 16, issue 12

https://doi.org/10.5194/amt-16-3085-2023

© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/amt-16-3085-2023

© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 16, issue 12

Research article

|

21 Jun 2023

Research article |

| 21 Jun 2023

A data-driven persistence test for robust (probabilistic) quality control of measured environmental time series: constant value episodes

Najmeh Kaffashzadeh

Data sets

TOAR Data Infrastructure Sabine Schröder, Martin G. Schultz, Niklas Selke, Jianing Sun, Jessica Ahring, Amirpasha Mozaffari, Mathilde Romberg, Eleonora Epp, Max Lensing, Sander Apweiler, Lukas H. Leufen, Clara Betancourt, Björn Hagemeier, and Saini Rajveer https://doi.org/10.34730/4d9a287dec0b42f1aa6d244de8f19eb3

Model code and software

A statistical data-driven test for probabilistic data quality control Najmeh Kaffashzadeh https://doi.org/10.5281/zenodo.7951896

Short summary

Although quality control is a well-known issue in data application, research initiatives and organizations apply given methods based on traditional techniques (ad hoc thresholds and manual). These approaches are not only error prone but also unsuitable for a large volume of data. The method proposed in this paper is based on a new concept (probability) as an intuitive indicator and data’s characteristics, which leads it to be applicable to a wide variety of data and eases its fit for purpose.