Design and Implementation of A Graphical User Interface for Outlier Data Analysis: A Case Study on the Yesilirmak River

Year 2024, Volume: 19 Issue: 2, 57 - 68, 30.06.2024

Eda Göz , Zübeyde Zengül , Erdal Karadurmuş , Mehmet Yüceer

Abstract

Water quality control, especially in large-scale monitoring regions or networks, requires easy and automatic processes for detecting potential outliers in a reproducible manner. This study focuses on removing outlier values from a dataset collected by an online monitoring station on the Yeşilırmak River between 2007 and 2009. Seven different parameters were evaluated: dissolved oxygen (luminescence dissolved oxygen, LDO), temperature, pH, conductivity, total organic carbon (TOC), nitrate nitrogen (NO3-N), and ammonium nitrogen (NH4-N). Five methods – median, mean, Grubbs’, generalized extreme studentized deviate (GESD), and interquartile range (IQR) – were used for outlier removal. The developed models were integrated into a graphical user interface (GUI) in the MATLAB environment, facilitating practical and easy access. This study enables users to input any dataset into the software and remove outlier values using various methods in a few steps, thus preparing the data for modeling studies. It was observed that the median algorithm removed the most data points among the outlier data-removal methods.

Keywords

River water quality data, outlier detection, statistical methods, graphical user interface (GUI)

References

• Bae I, Ji U, (2019) Outlier detection and smoothing process for water level data measured by ultrasonic sensor in stream flows. Water 11, 1-16. doi: 10.3390/w11050951
• Bashir A, Awawdeh M, Faisal T, Flower Queen MP, (2022) Matlab-based Graphical User Interface for IoT Sensor Measurements Subject to Outlier. In: Proceeding of the Advances in Science and Engineering Technology International Conferences (ASET), pp. 1-6, Dubai, United Arab Emirates. https://www.scribd.com/document/716118362/C-2022
• Ben-Gal I, (2005) Outlier detection in data mining and knowledge discovery handbook: A complete guide for practitioners and researchers (Ed. by O. Maimon and L. Rockach) pp. 131–146. https://is.muni.cz/el/fi/jaro2013/PV056/um/outlier-HandbookOfKD
• Berendrecht W, van Vliet M, Griffioen J, (2023) Combining statistical methods for detecting potential outliers in groundwater quality time series. Environ. Modell. Asses. 195, 1-14. doi: 10.1007/s10661-022-10661-0
• Bonakdari H, Zeynoddin M, (2022) Stochastic Modeling: A Thorough Guide to Evaluate, Pre-Process, Model and compare time series with MATLAB software (Ed. by Aera Gariguez) pp. 1-353.
• https://www.scribd.com/document/722552043/Full-download-book-Stochastic-Modeling-A-Thorough-Guide-To-Evaluate-Pre-Process-Model-And-Compare-Time-Series-With-Matlab-Software-Pdf-pdf
• Chelishchev P, Popov A, Sorby K, (2018) An investigation of outlier detection procedures for CMM measurement data. Matec Web of Conferences 220, 1-6. doi: 10.1051/matecconf/201822004002
• Cho H., Oh JH, Kim KO, Shim JS, (2013) Outlier detection and missing data filling methods for coastal water temperature data. Journal of Coastal Research 165,1898–1903. doi: 10.2112/SI65-321.1
• Di Blasi JIP, JM, Torres, Nieto PJG, Fernández JRA, Muñiz CD, Taboada J., (2015) Analysis and detection of functional outliers in water quality parameters from different automated monitoring stations in the Nalón River Basin (Northern Spain). Environmental Science and Pollution Research 22, 387–396. doi: 10.1007/s11356-014-3318-5
• Dogo EM, Nwulu NI, Twala B, Aigbavboa C, (2019) A survey of machine learning methods applied to anomaly detection on drinking-water quality data. Urban Water Journal 16, 235-248. doi: 10.1080/1573062X.2019.1637002
• Friel M, Enderle J, DeFrancesco V, Rosow E, Greenshields I, (2004) Design, implementation and validation of a graphical user interface for analysis of patient monitor event logs. In: Proceeding of the IEEE 30th Annual Northeast Bioengineering Conference, pp. 182-183, Springfield, MA, USA. doi: 10.1109/NEBC.2004.1300055
• Grubbs FE, (1969) Procedures for detecting outlying observations in samples, Technometrics, 11, 1–21. doi: 10.1080/00401706.1969.10490657
• Jamshidi EJ, Yusup Y, Kayode JS, Kamaruddin MA, (2022) Detecting outliers in a univariate time series dataset using unsupervised combined statistical methods: A case study on surface water temperature. Ecological Informatics 69, 1-12. doi: 10.1016/j.ecoinf.2022.101672
• Jingang J, Lu S, Zhongya F, Jiaguo Q, (2017) Outlier detection and sequence reconstruction in continuous timeseries of ocean observation data based on difference analysis and the Dixon criterion. Limnology and Oceanography: Methods 15, 916-927. doi: 10.1002/lom3.10212
• Lai KH, Zha D, Wang G, Xu J, Zhao Y, Kumar D, Chen Y, Zumkhawaka P, Wan M, Martinez D, Hu X, (2021) TODS: An Automated Time Series Outlier Detection System. In: Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), pp. 16060-16062. doi: 10.1609/aaai.v35i18.18012
• Muñiz CD, Nieto PJG, Fernández JRA, Torres JM, Taboada J, (2012) Detection of outliers in water quality monitoring samples using functional data analysis in San Esteban estuary (Northern Spain). Science of the Total Environment 439, 54-61. doi: 10.1016/j.scitotenv.2012.08.083
• Plazas-Nossa L, Angulo MAA, Torres A, (2016) Detection of outliers and imputing of missing values for water quality UV-VIS Absorbance Time Series. INGENIERÍA Journal: Computational Intelligence Section 22, 111-124. doi: 10.14483/UDISTRITAL.JOUR.REVING.2017.1.A01
• Saberi A, (2015) Automatic outlier detection in automated water quality measurement stations. M.Sc. Thesis, University Laval, 1-112. https://modeleau.fsg.ulaval.ca/fileadmin/modeleau/documents/ Publications/MSc_s/saberiatefeh_msc.pdf
• Sun G, Jiang P, Xu H, Yu S,1 Guo D, Lin G, Wu H, (2019) Outlier detection and correction for monitoring data of water quality based on improved VMD and LSSVM. Complexity 2019, 9643921, 1-12. doi: 10.1155/2019/9643921
• Vera MJ, Dubravka B, Nikola J, Vojin I, Bojana PB, (2013) Detecting and removing outlier(s) in electromyographic gait-related patterns. J. Appl. Stat. 40, 1319-1332. doi: 10.1080/02664763.2013.785495