Audit Opinion Prediction with Data Mining Methods: Evidence From Borsa Istanbul (BIST)

Abstract

This study compares the performance of 12 different data mining methods in predicting audit opinions for companies’ financial statements. The study dataset consists of 2,093 firm-year observations from 161 companies listed on Borsa Istanbul for 2010-2022. The independent audit opinion types were classified using a set of 28 independent financial and non-financial variables. The following prediction models were used in the study: Bayesian Networks, Naive Bayes, Logistic Regression, Artificial Neural Networks, Radial Basis Function, Support Vector Machines, K-Nearest Neighbor, AdaBoost.M1 Algorithm, Decision Trees (J48), Random Forest, Decision Stump, and Classification and Regression Tree (CART). According to the results of the analysis, the Random Forest model demonstrated the best performance with a prediction accuracy rate of 96.68% for predicting audit opinions. The statistical results of the models were compared based on prediction accuracy, confusion matrix, detailed accuracy results, Type I error rate, Type II error rate, and performance measures. This study pioneers developing models capable of accurately predicting audit opinions using 2,093 firm-year observations based on financial and non-financial variables. This contributes likely to the audit literature by predicting audit opinion types through data mining classification methods. The framework design used in the study is anticipated to serve as a decision support tool for internal and external auditors, accountants, shareholders, company executives, tax authorities, other public institutions, individual and institutional investors, stock exchanges, law firms, financial analysts, credit rating agencies, and the banking system when making decisions.

Keywords

• Audit Opinion Prediction
• Auditing
• Data Mining
• Classification
• Machine Learning

Veri Madenciliği Yöntemleriyle Denetim Görüşü Tahmini: Borsa İstanbul’dan (BIST) Kanıtlar

Abstract

Bu çalışma, şirketlerin finansal tablolarına ilişkin denetim görüşlerinin tahmin edilmesinde on iki farklı veri madenciliği yönteminin performanslarını karşılaştırmaktadır. Araştırma veri seti, Borsa İstanbul’da işlem gören 161 şirketin 2010-2022 yıllarına ait 2.093 şirket-yıl gözleminden oluşmaktadır. Finansal ve finansal olmayan 28 bağımsız değişken seti kullanılarak bağımsız denetim görüş türü sınıflandırması yapılmıştır. Bu çalışmada, tahmin modelleri için Bayes İnanç Ağı, Naive Bayes, Lojistik Regresyon, Yapay Sinir Ağları, Radyal Tabanlı Fonksiyon, Destek Vektör Makineleri, K-En Yakın Komşu, AdaBoost.M1 Algoritması, Karar Ağaçları (J48), Rastgele Ormanlar, Karar Kütüğü ile Sınıflandırma ve Regresyon Ağacı kullanılmıştır. Analiz sonuçlarına göre, denetim görüşünü tahmin etmede tahmin doğruluk performansı açısından %96.68 oranıyla en iyi performansı Rastgele Ormanlar modeli göstermiştir. Modellerin istatistiksel sonuçları tahmin doğruluğu, sınıflandırma matrisi, detaylı doğruluk sonuçları, Tip I hata oranı, Tip II hata oranı ve performans sonuçları ölçütleri dikkate alınarak karşılaştırılmıştır. Bu çalışma, finansal ve finansal olmayan değişkenlere dayalı 2.093 şirket-yıl gözlemini kullanarak, doğru denetim görüşünü tahmin edebilecek modeller geliştiren öncü bir çalışmadır. Bu çalışmanın veri madenciliği sınıflandırma yöntemleri ile denetim görüş türünü tahmin ederek denetim literatürüne katkıda bulunması beklenmektedir. Araştırmada kullanılan çerçeve tasarımı hem iç hem de bağımsız denetçiler, muhasebeciler, hissedarlar, şirket yöneticileri, vergi otoriteleri ve diğer kamu kurumları, bireysel ve kurumsal yatırımcılar, borsalar, hukuk şirketleri, finansal analistler, kredi derecelendirme kuruluşları ve bankacılık sistemi için alacakları kararlarda bir karar destek aracı olarak hizmet edebileceği düşünülmektedir.

Keywords

• Denetim Görüşü Tahmini
• Denetim
• Veri Madenciliği
• Sınıflandırma
• Makine Öğrenimi

References

1. [1] “Independent auditing standard 315.” Accessed: Dec. 18, 2024. [Online]. Available: https://www.kgk.gov.tr/Portalv2Uploads/files/Duyurular/v2/BDS/bdsyeni25.12.2017/BDS%20315-Site.pdf
2. [2] H. Valipour, F. Salehi, and M. Bahrami, “Predicting audit reports using meta-heuristic algorithms,” J. Distrib. Sci., vol. 11, no. 6, pp. 13–19, 2013, doi: 10.13106/jds.2013.vol11.no6.13.
3. [3] A. K. Nawaiseh, M. F. Abbod, and T. Itagaki, “Financial statement audit using support vector machines, artificial neural networks and k-nearest neighbor: empirical study of UK and Ireland,” Int. J. Simul. Syst. Sci. Technol., Mar. 2020, doi: 10.5013/IJSSST.a.21.02.07.
4. [4] F. A. Amani and A. M. Fadlalla, “Data mining applications in accounting: A review of the literature and organizing framework,” Int. J. Account. Inf. Syst., vol. 24, pp. 32–58, 2017, doi: https://doi.org/10.1016/j.accinf.2016.12.004.
5. [5] O. Pourheydari, H. Nezamabadi-pour, and Z. Aazami, “Identifying qualified audit opinions by artificial neural networks,” Afr. J. Bus. Manag., vol. 6, no. 44, pp. 11077–11087, Nov. 2012, doi: 10.5897/AJBM12.855.
6. [6] S. M. Saif, M. Sarikhani, and F. Ebrahimi, “Finding rules for audit opinions prediction through data mining methods,” SSRN Electron. J., 2012, doi: 10.2139/ssrn.2185919.
7. [7] M. A. Fernandez-Gamez, F. Garcia-Lagos, and J. R. Sanchez-Serrano, “Integrating corporate governance and financial variables for the identification of qualified audit opinions with neural networks,” Neural Comput. Appl., vol. 27, no. 5, pp. 1427–1444, Jul. 2016, doi: 10.1007/s00521-015-1944-6.
8. [8] E. Kirkos, C. Spathis, A. Nanopoulos, and Y. Manolopoulos, “Identifying qualified auditors’ opinions: a data mining approach,” J. Emerg. Technol. Account., vol. 4, no. 1, pp. 183–197, Jan. 2007, doi: 10.2308/jeta.2007.4.1.183.

9. [9] N. Dopuch, R. W. Holthausen, and R. W. Leftwich, “Predicting audit qualifications with financial and market variables,” Account. Rev., pp. 431–454, 1987.
10. [10] K. Keasey, R. Watson, and P. Wynarczyk, “The small company audit qualification: a preliminary investigation,” Account. Bus. Res., vol. 18, no. 72, pp. 323–334, Sep. 1988, doi: 10.1080/00014788.1988.9729379.
11. [11] D. Zdolšek, T. Jagrič, and M. Odar, “Identification of auditor’s report qualifications: an empirical analysis for Slovenia,” Econ. Res.-Ekon. Istraživanja, vol. 28, no. 1, pp. 994–1005, Jan. 2015, doi: 10.1080/1331677X.2015.1101960.
12. [12] E. Kirkos, C. Spathis, and Y. Manolopoulos, “Data mining techniques for the detection of fraudulent financial statements,” Expert Syst. Appl., vol. 32, no. 4, pp. 995–1003, 2007, doi: 10.1016/j.eswa.2006.02.016.
13. [13] P. Ravisankar, V. Ravi, G. Raghava Rao, and I. Bose, “Detection of financial statement fraud and feature selection using data mining techniques,” Decis. Support Syst., vol. 50, no. 2, pp. 491–500, Jan. 2011, doi: 10.1016/j.dss.2010.11.006.
14. [14] G. L. Gray and R. S. Debreceny, “A taxonomy to guide research on the application of data mining to fraud detection in financial statement audits,” Int. J. Account. Inf. Syst., vol. 15, no. 4, pp. 357–380, Dec. 2014, doi: 10.1016/j.accinf.2014.05.006.
15. [15] M. Cecchini, H. Aytug, G. J. Koehler, and P. Pathak, “Detecting management fraud in public companies,” Manag. Sci., vol. 56, no. 7, pp. 1146–1160, Jul. 2010, doi: 10.1287/mnsc.1100.1174.
16. [16] K. Takahashi, Y. Kurokawa, and K. Watase, “Corporate bankruptcy prediction in Japan,” J. Bank. Finance, vol. 8, no. 2, pp. 229–247, Jun. 1984, doi: 10.1016/0378-4266(84)90005-0.
17. [17] W. Kwak, Y. Shi, and C. F. Lee, “Data mining applications in accounting and finance context,” in Handbook of Financial Econometrics, Mathematics, Statistics, and Machine Learning, World Scientific, 2020, pp. 823–857. doi: 10.1142/9789811202391_0021.
18. [18] I. Dutta, S. Dutta, and B. Raahemi, “Detecting financial restatements using data mining techniques,” Expert Syst. Appl., vol. 90, pp. 374–393, Dec. 2017, doi: 10.1016/j.eswa.2017.08.030.
19. [19] E. Kirkos, C. Spathis, and Y. Manolopoulos, “Applying data mining methodologies for auditor selection,” in Proceedings 11th Pan-Hellenic conference in informatics (PCI), 2007, pp. 165–178. [Online]. Available: https://datalabold. csd.auth.gr/wp-content/uploads/publications/PCI07ksm.pdf
20. [20] T. G. Calderon and J. J. Cheh, “A roadmap for future neural networks research in auditing and risk assessment,” Int. J. Account. Inf. Syst., vol. 3, no. 4, pp. 203–236, Dec. 2002, doi: 10.1016/S1467-0895(02)00068-4.
21. [21] E. Koskivaara, “Artificial neural networks in analytical review procedures,” Manag. Audit. J., vol. 19, no. 2, pp. 191– 223, Feb. 2004, doi: 10.1108/02686900410517821.
22. [22] E. Koskivaara and B. Back, “Artificial neural network assistant (anna) for continuous auditing and monitoring of financial data,” J. Emerg. Technol. Account., vol. 4, no. 1, pp. 29–45, Jan. 2007, doi: 10.2308/jeta.2007.4.1.29.
23. [23] A. Yaşar, E. Yakut, and M. M. Gutnu, “Predicting qualified audit opinions using financial ratios: Evidence from the Istanbul Stock Exchange,” Int. J. Bus. Soc. Sci., vol. 6, no. 8, pp. 57–67, 2015.
24. [24] A. Yaşar, “Olumlu goruş dışındaki denetim goruşlerinin veri madenciliği yontemleriyle tahminine ilişkin karar ve birliktelik kuralları,” Financ. Anal. Cozum Derg., vol. 26, no. 133, 2016, Accessed: Jan. 07, 2025. [Online]. Available: https://search.ebscohost.com
25. [25] T. Buyuktanir and T. Toraman, “Auditor’s opinions prediction with machine learning algorithms,” in 2020 28th Signal Processing and Communications Applications Conference (SIU), IEEE, 2020, pp. 1–4. Accessed: Dec. 18, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9302441/
26. [26] A. Ozcan, “Determining factors affecting audit opinion: evidence from Turkey,” Int. J. Account. Financ. Report., vol. 6, no. 2, Art. no. 2, Aug. 2016, doi: 10.5296/ijafr.v6i2.9775.
27. [27] B. Adiloglu and B. Vuran, “Identification of key performance ındicators of auditor’s reports: evidence from Borsa Istanbul (BIST),” Press. Procedia, vol. 3, no. 1, pp. 854–859, 2017, doi: https://doi.org/10.17261/Pressacademia.2017.665.
28. [28] M. Mat and S. Onal, “Bağımsız denetim raporlarında denetim goruşunu etkileyen faktorlerin belirlenmesi: Borsa İstanbul’da bir uygulama,” Muhasebe Bilim Dunya. Derg., vol. 21, no. 3, pp. 733–760, 2019, doi: 10.31460/mbdd.533127.
29. [29] İ. Sakin, “Bağımsız denetim goruşunu etkileyen faktorlerin tespiti: Borsa İstanbula kayıtlı işletmeler uzerine bir uygulama,” Master’s Thesis, Sosyal Bilimler Enstitusu, 2019. Accessed: Jan. 06, 2025. [Online]. Available: https://acikbilim.yok.gov.tr/handle/20.500.12812/124427
30. [30] H. A. Ata and I. H. Seyrek, “The use of data mining techniques in detecting fraudulent financial statements: an application on manufacturing firms,” Suleyman Demirel Univ. J. Fac. Econ. Adm. Sci., vol. 14, no. 2, 2009, Accessed: Dec. 18, 2024. [Online]. Available: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=27ff227919838c7c0991b061751f0e246557788d
31. [31] S. Terzi, “Hile ve usulsuzluklerin tespitinde veri madenciliğinin kullanımı,” Muhasebe Ve Finans. Derg., no. 54, Art. no. 54, Apr. 2012.
32. [32] S. Terzi and İ. Şen, “Finansal tablo hilelerinin veri madenciliği yardımıyla tespit edilmesi: uretim sektorunde bir araştırma,” J. Account. Tax. Stud., vol. 5, no. 2, Art. no. 2, Jul. 2012.
33. [33] M. C. Sorkun and T. Toraman, “Fraud detection on financial statements using data mining techniques,” Int. J. Intell. Syst. Appl. Eng., vol. 3, no. 5, pp. 132–134, Sep. 2017, doi: 10.18201/ijisae.2017531428.
34. [34] K. Kirda and M. Katkat Ozcelik, “Finansal tablo hilesi riski taşıyan şirketlerin veri madenciliği ile belirlenmesi,” Muhasebe Ve Vergi Uygulamaları Derg., vol. 14, no. 2, pp. 609–639, Jul. 2021, doi: 10.29067/muvu.802703.
35. [35] B. Tatar and H. Kiymik, “Finansal tablolarda hile riskinin tespit edilmesinde veri madenciliği yontemlerinin kullanılmasına yonelik bir araştırma,” J. Yaşar Univ., vol. 16, no. 64, pp. 1700–1719, Oct. 2021, doi: 10.19168/jyasar.956623.
36. [36] I. Kilic and S. Onal, “Finansal hilelerin tespit edilmesinde kullanılan veri madenciliği yontemleri ve Borsa Istanbul’da bir uygulama,” Muhasebe Ve Denetime Bakış, vol. 22, no. 67, pp. 181–208, Sep. 2022, doi: 10.55322/mdbakis.1068503.
37. [37] G. Ozdagoglu, A. Ozdagoglu, Y. Gumus, and G. Kurt Gumus, “The application of data mining techniques in manipulated financial statement classification: the case of Turkey,” J. AI Data Min., no. Online First, 2017, doi: 10.22044/jadm.2016.664.
38. [38] I. F. Ceyhan, “Bağımsız denetim kalitesini artırıcı bir yontem olarak veri madenciliği: Borsa Istanbul uygulaması,” PhD Thesis, Sakarya Universitesi (Turkey), 2014. Accessed: Dec. 18, 2024. [Online]. Available: https://search.proquest.com/openview/692e0761f5f5ea36a5a8559ac464da35/1?pqorigsite= gscholar&cbl=2026366&diss=y&casa_token=zh2KnS5lw3sAAAAA:PdmSn3AqhdFbuV9omDXFsQlyOuy1_ dnp9Q0HpkNiebBNVEMyW_TafJoFqO4F1QbkgcFl3Zozw
39. [39] N. Comert and M. Kaymaz, “Arac sigortası hilelerinde veri madenciliğinin kullanımı,” Marmara Universitesi İktisadi Ve İdari Bilim. Derg., vol. 41, no. 2, pp. 364–390, Jan. 2020, doi: 10.14780/muiibd.665058.
40. [40] E. K. Laitinen and T. Laitinen, “Qualified audit reports in Finland: evidence from large companies,” Eur. Account. Rev., vol. 7, no. 4, pp. 639–653, Dec. 1998, doi: 10.1080/096381898336231.
41. [41] C. T. Spathis, “Audit qualification, firm litigation, and financial ınformation: an empirical analysis in greece,” Int. J. Audit., vol. 7, no. 1, pp. 71–85, Mar. 2003, doi: 10.1111/1099-1123.00006.
42. [42] C. Spathis, M. Doumpos, and C. Zopounidis, “Using client performance measures to identify pre-engagement factors associated with qualified audit reports in Greece,” Int. J. Account., vol. 38, no. 3, pp. 267–284, Sep. 2003, doi: 10.1016/S0020-7063(03)00047-5.
43. [43] M. Doumpos, C. Gaganis, and F. Pasiouras, “Explaining qualifications in audit reports using a support vector machine methodology,” Intell. Syst. Account. Finance Manag., vol. 13, no. 4, pp. 197–215, Dec. 2005, doi: 10.1002/isaf.268.
44. [44] C. Caramanis and C. Spathis, “Auditee and audit firm characteristics as determinants of audit qualifications: Evidence from the Athens stock exchange,” Manag. Audit. J., vol. 21, no. 9, pp. 905–920, Dec. 2006, doi: 10.1108/02686900610705000.
45. [45] C. Gaganis, F. Pasiouras, and M. Doumpos, “Probabilistic neural networks for the identification of qualified audit opinions,” Expert Syst. Appl., vol. 32, no. 1, pp. 114–124, Jan. 2007, doi: 10.1016/j.eswa.2005.11.003.
46. [46] C. Gaganis, F. Pasiouras, C. Spathis, and C. Zopounidis, “A comparison of nearest neighbours, discriminant and logit models for auditing decisions,” Intell. Syst. Account. Finance Manag., vol. 15, no. 1–2, pp. 23–40, Jan. 2007, doi: 10.1002/isaf.283.
47. [47] N. Stanišić, T. Radojević, and N. Stanić, “Predicting the type of auditor opinion: Statistics, machine learning, or a combination of the two?,” Eur. J. Appl. Econ., vol. 16, no. 2, pp. 1–58, 2019, doi: 10.5937/EJAE16-21832.
48. [48] A. K. Nawaiseh and M. F. Abbod, “Financial statement audit utilising naive bayes networks, decision trees, linear discriminant analysis and logistic regression,” in The ımportance of new technologies and entrepreneurship in business development: in the context of economic diversity in developing countries, B. Alareeni, A. Hamdan, and I. Elgedawy, Eds., Cham: Springer International Publishing, 2021, pp. 1305–1320. doi: 10.1007/978-3-030-69221-6_97.
49. [49] H. Zarei, H. Yazdifar, M. Dahmarde Ghaleno, and R. Azhmaneh, “Predicting auditors’ opinions using financial ratios and non-financial metrics: evidence from Iran,” J. Account. Emerg. Econ., vol. 10, no. 3, pp. 425–446, Jun. 2020, doi: 10.1108/JAEE-03-2018-0027.
50. [50] M. I. H.-P. Wu and L. Li, “The BP neural network with adam optimizer for predicting audit opinions of listed companies,” 2021. Accessed: Jan. 07, 2025. [Online]. Available: https://www.semanticscholar.org/paper/The-BPNeural- Network-with-Adam-Optimizer-for-Audit-Wu-Li/a08f165c1ad8b530c0a1a356d9233639e23bb52f
51. [51] A. Saeedi, “Audit opinion prediction: a comparison of data mining techniques,” J. Emerg. Technol. Account., vol. 18, no. 2, pp. 125–147, Sep. 2021, doi: 10.2308/JETA-19-10-02-40.
52. [52] N. Japkowicz and M. Shah, Evaluating learning algorithms: a classification perspective. Cambridge University Press, 2011.
53. [53] J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data.,” Biometrics, vol. 33, no. 1, p. 159, Mar. 1977, doi: 10.2307/2529310.
54. [54] G. S. Monroe and S. T. Teh, “Predicting uncertainty audit qualifications in Australia using publicly available information,” Account. Finance, vol. 33, no. 2, pp. 79–106, Nov. 1993, doi: 10.1111/j.1467-629X.1993.tb00200.x.