A Supervised Evaluation Framework for Privacy Risk Scoring Models

Year 2025, Volume: 14 Issue: 2 , 1 - 17 , 23.06.2025

Yasir Kılıç

https://doi.org/10.55859/ijiss.1599063

https://izlik.org/JA25HS46UJ

Abstract

The rise of online social networks (OSNs) has heightened concerns regarding user privacy, as sensitive attributes disclosed on profiles are increasingly susceptible to misuse, including identity theft and targeted manipulation. Each user's privacy risk varies based on the nature of the shared data and its intended audience. To quantify these risks, researchers have introduced privacy risk scores, inspired by credit scoring systems, to measure vulnerability to privacy violations. However, despite the proliferation of scoring models, their evaluation frameworks often rely on unsupervised methods, such as goodness-of-fit tests, which limit their practical reliability. To address the limitation, we propose SPR-EVAL, a supervised evaluation framework that systematically assesses the performance of privacy scoring models using various real-world attack scenarios, offering a more robust and actionable approach to privacy risk assessment. SPR-EVAL integrates simulations of various real-world privacy attacks as a core evaluation mechanism. The framework is adaptable to any OSN dataset and supports the incorporation of diverse privacy risk scoring models and privacy attacks. To validate the proposed framework, we conducted experiments on a real-world Facebook OSN dataset. The results highlight the effectiveness of SPR-EVAL in evaluating and comparing popular privacy scoring models under supervised conditions. By offering a rigorous supervised evaluation metric, SPR-EVAL overcomes the limitations of traditional unsupervised methods, representing a notable advancement in the domain of privacy risk scoring for OSNs.

Keywords

privacy risk scoring , online social networks (OSNs) , privacy , attribute disclosure attacks , point multiserial correlation(PMS)

References

• Anonymous, “Digital 2024: 5 billion social media users,” , 2024. Accessed on September, 1, 2024
• Wang, Cheng and Yang, Bo and Cui, Jipeng and Wang, Chaodong, “Fusing behavioral projection models for identity theft detection in online social networks,” IEEE Transactions on Computational Social Systems, vol. 6, no. 4, pp. 637–648, 2019.
• Wang, Cheng and Zhu, Hangyu and Yang, Bo, “Composite behavioral modeling for identity theft detection in online social networks,” IEEE Transactions on Computational Social Sys- tems, vol. 9, no. 2, pp. 428–439, 2021.
• Tucker, Catherine E, “Social networks, personalized advertising, and privacy controls,” Journal of marketing research, vol. 51, no. 5, pp. 546–562, 2014.
• Zhang, Kunpeng and Bhattacharyya, Siddhartha and Ram, Sudha, “Large-scale network analysis for online social brand advertising,” Mis Quarterly, vol. 40, no. 4, pp. 849–868, 2016.
• Ali, Ahmad and Malik, Ahmad Kamran and Ahmed, Mansoor and Raza, Basit and Ilyas, Muhammad, “Privacy concerns in online social networks: A users’ perspective,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 7, pp. 601–613, 2019.
• Jain, Ankit Kumar and Sahoo, Somya Ranjan and Kaubiyal, Jyoti, “Online social networks security and privacy: compre- hensive review and analysis,” Complex & Intelligent Systems, vol. 7, no. 5, pp. 2157–2177, 2021.
• Confessore, Nicholas, “Cambridge Analytica and Facebook: The scandal and the fallout so far,” The New York Times, vol. 4, pp. 2018, 2018.
• Paulina Okunyt˙ e, “Almost 500 million Instagram users had their data scraped, hackers claim,” Access: https://cybernews.com/news/instagram-user-data-scraping/, 2024.
• Liu, Kun and Terzi, Evimaria, “A Framework for Computing the Privacy Scores of Users in Online Social Networks,” Pro- ceedings of the 2009 Ninth IEEE International Conference on Data Mining, pp. 288–297, 2009.
• Liu, Kun and Terzi, Evimaria, “A framework for computing the privacy scores of users in online social networks,” ACM Transactions on Knowledge Discovery from Data (TKDD), vol. 5, no. 1, pp. 6, 2010.
• Srivastava, Agrima and Geethakumari, G, “Measuring privacy leaks in online social networks,” 2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 2095–2100, 2013.
• Domingo-Ferrer, Josep, “Rational privacy disclosure in social networks,” International Conference on Modeling Decisions for Artificial Intelligence, pp. 255–265, 2010.
• Nepali, Raj Kumar and Wang, Yong, “Sonet: A social network model for privacy monitoring and ranking,” 2013 IEEE 33rd International Conference on Distributed Computing Systems Workshops, pp. 162–166, 2013.
• Sramka, Michal, “Evaluating Privacy Risks in Social Networks from the User’s Perspective,” Advanced Research in Data Privacy, pp. 251–267, 2015.
• Petkos, Georgios and Papadopoulos, Symeon and Kompatsiaris, Yiannis, “PScore: a framework for enhancing privacy awareness in online social networks,” 2015 10th International Conference on Availability, Reliability and Security, pp. 592–600, 2015.
• Kilic, Yasir and Inan, Ali, “Privacy Scoring over OSNs: Shared Data Granularity as a Latent Dimension,” ACM Transactions on the Web, vol. 17, no. 4, pp. 1–28, 2023.
• Bioglio, Livio and Pensa, Ruggero G, “Impact of neighbors on the privacy of individuals in online social networks,” Procedia Computer Science, vol. 108, pp. 28–37, 2017.
• Pensa, Ruggero G and Di Blasi, Gianpiero and Bioglio, Livio, “Network-aware privacy risk estimation in online social net- works,” Social Network Analysis and Mining, vol. 9, no. 1, pp. 15, 2019.
• Alemany, J and del Val, E and Alberola, J and Garc´ ıa-Fornes, Ana, “Estimation of privacy risk through centrality metrics,” Future Generation Computer Systems, vol. 82, pp. 63–76, 2018.
• De, Sourya Joyee and Imine, Abdessamad, “Privacy Risk Analysis of Online Social Networks,” Synthesis Lectures on Information Security, Privacy, and Trust, vol. 10, no. 1, pp. 1–109, 2020.
• Braunstein, Alex and Granka, Laura and Staddon, Jessica, “Indirect content privacy surveys: measuring privacy without 16 INTERNATIONAL JOURNAL OF INFORMATION SECURITY SCIENCE Y. Kilic. ,Vol.14, No.2, pp.1-17. doi asking about it,” Proceedings of the Seventh Symposium on Usable Privacy and Security, pp. 15, 2011.
• Pensa, Ruggero G and Di Blasi, Gianpiero, “A centrality-based measure of user privacy in online social networks,” Proceedings of the 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pp. 1438–1439, 2016.
• Bioglio, Livio and Capecchi, Sara and Peiretti, Federico and Sayed, Dennis and Torasso, Antonella and Pensa, Ruggero G, “A social network simulation game to raise awareness of privacy among school children,” IEEE Transactions on Learning Technologies, vol. 12, no. 4, pp. 456–469, 2018.
• Wang, Haizhou and Song, Mingzhou, “Ckmeans. 1d. dp: opti- mal k-means clustering in one dimension by dynamic program- ming,” The R journal, vol. 3, no. 2, pp. 29, 2011.
• Gupta, S Das, “Point biserial correlation coefficient and its generalization,” Psychometrika, vol. 25, no. 4, pp. 393–408, 1960.
• Bartlett, Maurice Stevenson, “Properties of sufficiency and statistical tests,” Proceedings of the Royal Society of London. Series A-Mathematical and Physical Sciences, vol. 160, no. 901, pp. 268–282, 1937.
• Tamhane, Ajit C, “Multiple comparisons in model I one-way ANOVA with unequal variances,” Communications in Statistics- Theory and Methods, vol. 6, no. 1, pp. 15–32, 1977.
• Kruskal, William H and Wallis, W Allen, “Use of ranks in one- criterion variance analysis,” Journal of the American statistical Association, vol. 47, no. 260, pp. 583–621, 1952.
• Coban, Onder and Inan, Ali and Ozel, Selma Ayse, “Towards the design and implementation of an OSN crawler: A case of Turkish Facebook users,” International Journal of Information Security Science, vol. 9, no. 2, pp. 76–93, 2020.
• Thomas, Kurt and Grier, Chris and Nicol, David M, “unfriendly: Multi-party privacy risks in social networks,” Privacy Enhanc- ing Technologies: 10th International Symposium, PETS 2010, Berlin, Germany, July 21-23, 2010. Proceedings 10, pp. 236– 252, 2010.
• Chaabane, Abdelberi and Acs, Gergely and Kaafar, Mohamed Ali and others, “You are what you like! information leakage through users’ interests,” Proceedings of the 19th annual net- work & distributed system security symposium (NDSS), 2012.
• Kosinski, Michal and Stillwell, David and Graepel, Thore, “Private traits and attributes are predictable from digital records of human behavior,” Proceedings of the national academy of sciences, vol. 110, no. 15, pp. 5802–5805, 2013.
• De Ayala, Ralph J, “An introduction to polytomous item re- sponse theory models.,” Measurement and evaluation in Coun- seling and Development, 1993.