Design and Implementation of A Graphical User Interface for Outlier Data Analysis: A Case Study on the Yesilirmak River

Abstract

Water quality control, especially in large-scale monitoring regions or networks, requires easy and automatic processes for detecting potential outliers in a reproducible manner. This study focuses on removing outlier values from a dataset collected by an online monitoring station on the Yeşilırmak River between 2007 and 2009. Seven different parameters were evaluated: dissolved oxygen (luminescence dissolved oxygen, LDO), temperature, pH, conductivity, total organic carbon (TOC), nitrate nitrogen (NO3-N), and ammonium nitrogen (NH4-N). Five methods – median, mean, Grubbs’, generalized extreme studentized deviate (GESD), and interquartile range (IQR) – were used for outlier removal. The developed models were integrated into a graphical user interface (GUI) in the MATLAB environment, facilitating practical and easy access. This study enables users to input any dataset into the software and remove outlier values using various methods in a few steps, thus preparing the data for modeling studies. It was observed that the median algorithm removed the most data points among the outlier data-removal methods.

Keywords

• River water quality data
• outlier detection
• statistical methods
• graphical user interface (GUI)

References

1. Bae I, Ji U, (2019) Outlier detection and smoothing process for water level data measured by ultrasonic sensor in stream flows. Water 11, 1-16. doi: 10.3390/w11050951
2. Bashir A, Awawdeh M, Faisal T, Flower Queen MP, (2022) Matlab-based Graphical User Interface for IoT Sensor Measurements Subject to Outlier. In: Proceeding of the Advances in Science and Engineering Technology International Conferences (ASET), pp. 1-6, Dubai, United Arab Emirates. https://www.scribd.com/document/716118362/C-2022
3. Ben-Gal I, (2005) Outlier detection in data mining and knowledge discovery handbook: A complete guide for practitioners and researchers (Ed. by O. Maimon and L. Rockach) pp. 131–146. https://is.muni.cz/el/fi/jaro2013/PV056/um/outlier-HandbookOfKD
4. Berendrecht W, van Vliet M, Griffioen J, (2023) Combining statistical methods for detecting potential outliers in groundwater quality time series. Environ. Modell. Asses. 195, 1-14. doi: 10.1007/s10661-022-10661-0
5. Bonakdari H, Zeynoddin M, (2022) Stochastic Modeling: A Thorough Guide to Evaluate, Pre-Process, Model and compare time series with MATLAB software (Ed. by Aera Gariguez) pp. 1-353.
6. https://www.scribd.com/document/722552043/Full-download-book-Stochastic-Modeling-A-Thorough-Guide-To-Evaluate-Pre-Process-Model-And-Compare-Time-Series-With-Matlab-Software-Pdf-pdf
7. Chelishchev P, Popov A, Sorby K, (2018) An investigation of outlier detection procedures for CMM measurement data. Matec Web of Conferences 220, 1-6. doi: 10.1051/matecconf/201822004002
8. Cho H., Oh JH, Kim KO, Shim JS, (2013) Outlier detection and missing data filling methods for coastal water temperature data. Journal of Coastal Research 165,1898–1903. doi: 10.2112/SI65-321.1

9. Di Blasi JIP, JM, Torres, Nieto PJG, Fernández JRA, Muñiz CD, Taboada J., (2015) Analysis and detection of functional outliers in water quality parameters from different automated monitoring stations in the Nalón River Basin (Northern Spain). Environmental Science and Pollution Research 22, 387–396. doi: 10.1007/s11356-014-3318-5
10. Dogo EM, Nwulu NI, Twala B, Aigbavboa C, (2019) A survey of machine learning methods applied to anomaly detection on drinking-water quality data. Urban Water Journal 16, 235-248. doi: 10.1080/1573062X.2019.1637002
11. Friel M, Enderle J, DeFrancesco V, Rosow E, Greenshields I, (2004) Design, implementation and validation of a graphical user interface for analysis of patient monitor event logs. In: Proceeding of the IEEE 30th Annual Northeast Bioengineering Conference, pp. 182-183, Springfield, MA, USA. doi: 10.1109/NEBC.2004.1300055
12. Grubbs FE, (1969) Procedures for detecting outlying observations in samples, Technometrics, 11, 1–21. doi: 10.1080/00401706.1969.10490657
13. Jamshidi EJ, Yusup Y, Kayode JS, Kamaruddin MA, (2022) Detecting outliers in a univariate time series dataset using unsupervised combined statistical methods: A case study on surface water temperature. Ecological Informatics 69, 1-12. doi: 10.1016/j.ecoinf.2022.101672
14. Jingang J, Lu S, Zhongya F, Jiaguo Q, (2017) Outlier detection and sequence reconstruction in continuous timeseries of ocean observation data based on difference analysis and the Dixon criterion. Limnology and Oceanography: Methods 15, 916-927. doi: 10.1002/lom3.10212
15. Lai KH, Zha D, Wang G, Xu J, Zhao Y, Kumar D, Chen Y, Zumkhawaka P, Wan M, Martinez D, Hu X, (2021) TODS: An Automated Time Series Outlier Detection System. In: Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), pp. 16060-16062. doi: 10.1609/aaai.v35i18.18012
16. Muñiz CD, Nieto PJG, Fernández JRA, Torres JM, Taboada J, (2012) Detection of outliers in water quality monitoring samples using functional data analysis in San Esteban estuary (Northern Spain). Science of the Total Environment 439, 54-61. doi: 10.1016/j.scitotenv.2012.08.083
17. Plazas-Nossa L, Angulo MAA, Torres A, (2016) Detection of outliers and imputing of missing values for water quality UV-VIS Absorbance Time Series. INGENIERÍA Journal: Computational Intelligence Section 22, 111-124. doi: 10.14483/UDISTRITAL.JOUR.REVING.2017.1.A01
18. Saberi A, (2015) Automatic outlier detection in automated water quality measurement stations. M.Sc. Thesis, University Laval, 1-112. https://modeleau.fsg.ulaval.ca/fileadmin/modeleau/documents/ Publications/MSc_s/saberiatefeh_msc.pdf
19. Sun G, Jiang P, Xu H, Yu S,1 Guo D, Lin G, Wu H, (2019) Outlier detection and correction for monitoring data of water quality based on improved VMD and LSSVM. Complexity 2019, 9643921, 1-12. doi: 10.1155/2019/9643921
20. Vera MJ, Dubravka B, Nikola J, Vojin I, Bojana PB, (2013) Detecting and removing outlier(s) in electromyographic gait-related patterns. J. Appl. Stat. 40, 1319-1332. doi: 10.1080/02664763.2013.785495