Deepfake Dilemmas: Imagine Tomorrow’s Surveillance Society through Three Scenarios

Year 2024, Issue: 70, 121 - 134, 20.12.2024

Fatma Doğan Akkaya

https://doi.org/10.26650/JECS2024-1462119

Abstract

As deepfakes—hyperrealistic, manipulated media content becomes increasingly sophisticated, they raise critical questions about the future of social interaction and trust. Drawing on sociological insights enriched by the literature on virtual reality surveillance, this study explores the social landscapes that may emerge with the widespread adoption of deepfake avatars in the near future. Moving beyond technical considerations, the research employs social prediction methods to explore three potential future scenarios involving deepfake avatars: seamless integration that facilitates self-expression but complicates digital identity control; pervasive algorithmic surveillance that exacerbates social control and privacy issues; and the concept of ghost avatars, that is, digital representations that function as commodities or tools of resistance. This analysis highlights the ethical dilemmas surrounding deepfakes, particularly in relation to mass surveillance, digital self-control, and resistance movements. Contributing to scholarly discourse, the study underscores the need for proactive social and ethical frameworks to navigate the evolving deepfake landscape and highlights the importance of ongoing research for the responsible development and use of this powerful technology.

Keywords

deepfake technology, Artificial Intelligence (AI), virtual surveillance, digital society, future scenarios

References

• Adams, S. (2013). Post-panoptic surveillance through healthcare rating sites: Who’s watching whom?. Information, Communication & Society, 16(2), 215-235. google scholar
• Albrechtslund, A. (2008). Online social networking as participatory surveillance. First Monday, 13(3). https://doi.org/10.5210/fm.v13i3.2142 google scholar
• Aldridge, A. (1999). Prediction in sociology: Prospects for a devalued activity. Sociological Research Online, 4(3), 57-62. Algorithmic Justice League (n.d). https://www.ajl.org/ google scholar
• Angell, M. (2020, Feb 9). How real is the deepfake threat? American Banker. Retrieved from https://www.americanbanker.com/creditunions/news/how-real-is-the-deepfake-threat google scholar
• Attestiv (2021, May) Deepfakes: The Business threat: Inagural deepfakes in business survey results. Retrieved from https://f.hubspotusercontent30.net/hubfs/7603739/deepfake_report2021.pdf?__hstc=143725418.1879e4f87f3af9d371de4bc1807298ac. 1724673569402.1724673569402.1724673569402.1&__hssc=143725418.1.1724673569402&__hsfp=1308513962&hsCtaTracking= 2eea443f-cbd5-4229-a7c2-5237a703046d%7Cdab0da4f-7ddd-4586-b257-5fce836345bd google scholar
• Bardzell, J. (2014). Social media and the making of global imaginaries. Information, Communication & Society, 17(3), 329-347. google scholar
• Barfield, W. (2015). Cyber-Humans: Our Future with Machines. Copernicus Cham Springer. google scholar
• Bassett, D. J. (2018). Digital afterlifes: From social media platforms to thanabots and beyond. In C. Tandy (Ed.), Death and anti-death, Volume 16: Two hundred years after Frankenstein. (pp. 27-38). Ria University Press. google scholar
• Bentham, J., (1995). The Panopticon Writings. Bozovic, Miran (Ed.),. London, UK: Verso. google scholar
• Benzinga (2022, May 14). Apple Vs. Google: Which one is winning the augmented reality race? Business Insider. Retrieved from https://markets.businessinsider.com/news/stocks/apple-vs-google-which-one-is-winning-the-augmented-reality-race-1031460242 google scholar
• Bigo, D. (2006). Security, exception, ban and surveillance. In D. Lyon (Ed.), Theorising surveillance: The panopticon and beyond (pp. 46-68). Portland: Willan Publishing. google scholar
• Björklund, F. (2021). Trust and surveillance: An odd couple or a perfect pair?. (Eds)cViola, L. A., & Laidler, P. (2022). In Trust and Transparency in an Age of Surveillance (pp. 183-200). Routledge. google scholar
• Blodgett, B., & Tapia, A. (2011). Do avatars dream of electronic picket lines?: The blurring of work and play in virtual environments, Information Technology & People, 24 (1), 26-45. google scholar
• Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford University Press. google scholar
• Boyd, D., (2014). It’s complicated: The social lives of networked teens. Yale University Press. google scholar
• Brynjolfsson, E., & McAfee, A. (2011). Race against the machine: How the Digital Revolution is Accelerating Innovation, Driving Productivity, and Irreversibly Transforming Employment and the Economy. Lexington, VA: Beard Books. google scholar
• Busch, E., Ware, J., & Brief, I. P. (2023, December). The weaponisation of deepfakes. International center for counter-terrorism. Retrieved from https://www.icct.nl/sites/default/files/2023-12/The%20Weaponisation%20of%20Deepfakes.pdf google scholar
• Chen, Y., Wu, X., Hu, A. He, G., & Ju, G. (2021). Social prediction: a new research paradigm based on machine learning. Journal of Chinese Sociology. 8 (15). google scholar
• Cheney-Lippold, J. (2011). A new algorithmic identity: Soft biopolitics and the modulation of control. Theory, Culture and Society, 28(6), pp. 164-181. google scholar
• Clarke, R. (1988). Information technology and dataveillance. Communications of the ACM, 31(5), 498-512. google scholar
• Crawford, K. (2021). Atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press. google scholar
• Dastin, J. (2018, Oct 11). Amazon scraps a secret AI recruiting tool that showed bias against women. Retrieved from https://www.reuters.com/article/us-amazon-comjobs-automation-insight-idUSKCN1MK08G/ google scholar
• De Rancourt-Raymond, A. and Smaili, N. (2023). The unethical use of deepfakes. Journal of Financial Crime, 30(4), 1066-1077. google scholar
• Defleur, L. B. (1982). Technology, social change, and the future of sociology. The Pacific Sociological Review, 25 (4), 403-417. google scholar
• Deleuze, G., and Guattari, F. (1987). A thousand plateaus: capitalism and schizophrenia. Minneapolis: University of Minnesota Press. google scholar
• Deleuze, G. (1992). Postscript on the societies of control. October 59, 3-7. http://www.jstor.org/stable/778828 google scholar
• Dufva, T., & Dufva, M. (2019). Grasping the future of the digital society. Futures, 107, 17-28. google scholar
• Fagni, T., Falchi, F., Gambini, M., Martella, A., & Tesconi, M. (2021). TweepFake: About detecting deepfake tweets. Plos One, 16(5). Retrieved from https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0251415 google scholar
• Fourcade, M., & Healy, K. (2017). Seeing like a market. Socio-economic Review, 15(1), 9-29. google scholar
• Foucault, M., (1977). Discipline and punish: The birth of the prison. Vintage, New York. google scholar
• Galic, M., Timan, T., & Koops, B.-J. (2017). Bentham, Deleuze and Beyond: An overview of surveillance theories from the panopticon to participation. Philosophy & Technology, 30(1), 9-37. google scholar
• Gamage, D., Chen, J., & Sasahara, K. (2021, Oct 7). The emergence of deepfakes and their societal implications: A systematic review. Paper presented at Truth and Trust International Conference. Retrieved from https://truthandtrustonline.com/wp-content/uploads/2021/10/TTO2021_paper_20.pdf google scholar
• Gibson, A. (2021, May 17). Keeping It Real: Historians in the Deepfake Era. Perspectives on History. Retrieved from https://www.historians.org/perspectives-article/keeping-it-real-historians-in-the-deepfake-era-may-2021/ google scholar
• Gissen, L. (2022, Nov 21). Talented impressionist morphs into twenty different celebrities using jaw-dropping deepfake technology... So, can you work out who they all are of them? Mail Online. Retrieved from https://www.dailymail.co.uk/femail/article-11445427/Impressionist-transforms-celebrities-using-wild-deepfake-technology.html google scholar
• Greengard, S. (2019). Will deepfakes do deep damage?. Communications oftheACM, 63(1), 17-19. google scholar
• Giusti, S., & Piras, E. (Eds.). (2020). Democracy and fake news: information manipulation and post-truth politics. Routledge. google scholar
• Gonzalez, S. M., & Deckard, F. M. (2022). We Got Witnesses: Black women’s counter-surveillance for navigating police violence and legal estrangement. Social Problems. 71 (3), 894-911. google scholar
• Hadan, H., Choong, L., Zhang-Kennedy, L., & Nacke, L. E. (2024). Deceived by immersion: A systematic analysis of deceptive design in extended reality. ACM Computing Surveys, 56(10), 1-25. google scholar
• Haggerty, K. D., & Ericson, R. V. (1999). Governing the young. In R. Smandych (ed.) Governable Places: Readings on Governmentality and Crime Control, Dartmouth, UK: Ashgate. google scholar
• Haggerty, K.D, & Ericson, R.V. (2000) The surveillant assemblage. British Journal of Sociology. 51(4), 605-622. google scholar
• Haggerty, K. (2006). Tear down the walls: on demolishing the panopticon. In D. Lyon (Ed.), Theorising surveillance: The panopticon and beyond (pp. 23-45). Portland: Willan Publishing. google scholar
• Hancock, J. T, Naaman, M., & Levy, K. (2020) AI-mediated communication: Definition, research agenda, and ethical considerations. Journal of Computer-Mediated Communication. 25(1). 89-10. https://doi.org/10.1093/jcmc/zmz022 google scholar
• Hancock, J. T., & Bailenson, J. N. (2021). The social impact of deepfakes. Cyberpsychology, Behavior, and Social Networking. 24(3).149-152. http://doi.org/10.1089/cyber.2021.29208.jth google scholar Henshel, R. L. (1982). Sociology and social forecasting. Annual Review of Sociology, 8(1), 57-79. google scholar
• Hermida, A., & Hernandez-Santaolalla, V. (2018). Twitter and video activism as tools for counter-surveillance: The case of social protests in Spain. Information, Communication & Society, 21(3), 416-433 google scholar
• Hofman, J.M., A. Sharma, &D.J. Watts. (2017). Prediction and explanation in social systems. Science. 355(6324). 486-488. google scholar
• Horowitz, M. C., Allen, G. C., Kania, E. B., & Scharre, P. (2022, Jul 25). Strategic competition in the era of artificial intelligence. Center for a New American Security. Retrieved from https://www.cnas.org/publications/reports/strategic-competition-in-an-era-of-artificial-intelligence google scholar
• Hwang, S. (2023, Sep 26). In South Korea, AI-driven virtual humans are becoming mainstream. The Japan Times. Retrieved from https://www.japantimes.co.jp/business/2023/09/26/tech/ai-humans-south-korea/ google scholar
• iProov.com. (2022, Aug 26). Protecting against deepfakes: statistics and solutions. Retrieved from http://www.iproov.com/blog/deepfakes-statistics-solutions-biometric-protection google scholar
• Jungherr, A., and Schlarb, D. B. (2022). The extended reach of game engine companies: Epic Games and Unity Technologies provide platforms for extended reality and metaverse applications. Social Media + Society, 8(2), doi:10.1177/20563051221107641 google scholar
• Kang, J., Shilton, K., Estrin, D., & Burke, J. (2011). Self-Surveillance and privacy. Iowa Law Review, 97, 809. google scholar
• Kaplan, O. (1940). Prediction in the social sciences. Philosophy of Science. 7(4): 492-498. google scholar
• Kelly, J. (2022, Jan 21). The metaverse set off a battle between Google, Apple, Microsoft, and Meta to build virtual and augmented reality (VR) headsets. Forbes. Retrieved from https://www.forbes.com/sites/jackkelly/2022/01/21/the-metaverse-set-off-a-battle-between-tech-giants-google-apple-microsoft-and-meta-to-build-virtual-and-augmented-reality-headsets/?sh=6b4516c239c6. google scholar
• Kietzmann, J., Lee, L. W., McCarthy, I. P., & Kietzmann, T. C. (2020). Deepfakes: Trick or treat?. Business Horizons, 63(2), 135-146. google scholar
• Koskela, H. (2004). Webcams, TV shows, and mobile phones: empowering exhibitionism. Surveillance and Society, 2(2/3), 199-215. google scholar
• Köbis, N. C., Dolezalova, B., & Soraperra, I. (2021). Fooled twice: People cannot detect deepfakes but think they can. Iscience, 24(11). https://doi.org/10.1016/j.isci.2021.103364. google scholar
• Lacroix, C. (2020, Sep). Preventing discrimination caused by the use of artificial intelligence [Report 1]. Committee on Equality and Non-Discrimination. Council of Europe. Retrieved from https://assembly.coe.int/LifeRay/EGA/Pdf/TextesProvisoires/2020/20200915-PreventingDiscriminationAI-EN.pdf google scholar
• Langa, J. (2021). Deepfakes, real consequences: Crafting legislation to combat threats posed by deepfakes. Boston University Law Review,101, 761-801. google scholar
• Lyon, D. (Ed.). (2003). Surveillance as social sorting: privacy, risk, and digital discrimination. London: Routledge. google scholar
• Lyon, D. (2006). The search for surveillance theories. In D. Lyon (Ed.), Theorising surveillance: The panopticon and beyond (pp. 3-20). Portland: Willan Publishing. google scholar
• Lyon, D., (2007). Surveillance studies: An overview. Polity Press, UK: Cambridge. google scholar
• Lyon, D., (2003). Surveillance after September 11. Polity Press, UK: Cambridge. google scholar
• Mager, A., & Katzenbach, C. (2021). Future imaginaries in the making and governing of digital technology: Multiple, contested, commodified. New Media and Society, 23(2), 223-236. google scholar
• Mann, S., Nolan, J., & Wellman, B. (2003) Sousveillance: inventing and using wearable computing devices for data collection in surveillance environments. Surveillance and Society, 1(3), 331-355. google scholar
• Mann, S. (2004). "Sousveillance": inverse surveillance in multimedia imaging. In Proceedings of the 12th annual ACM international conference on Multimedia. Association for Computing Machinery. New York, USA, 620-627. https://doi.org/10.1145/1027527.1027673 google scholar
• Manovich, L. (2014). The language of new media. MIT Press. google scholar
• Marcellino, W., Beauchamp-Mustafaga, N., Kerrigan, A., Navarre Chao, L., & Smith, J. (2023, Sep 7). The rise of generative AI and the coming era of social media manipulation 3.0: Next-Generation Chinese Astroturfing and Coping with Ubiquitous AI. RAND Commentary. Retrieved from https://www.rand.org/pubs/perspectives/PEA2679-1.html google scholar
• Martin, T., Hofman, J. M., Sharma, A., Anderson, A., & Watts, D. J. (2016, April). Exploring limits to prediction in complex social systems. In Proceedings of the 25th international conference on world wide web (pp. 683-694). Retrieved from https://www.cs.toronto.edu/ ashton/pubs/limpred.pdf google scholar
• Marx, G. (2003) ‘A tack in the shoe: neutralizing and resisting the new surveillance’. Journal of Social Issues, 59(2),. 369-390. google scholar
• Mearns, G. W., Richardson, R. and Robson, L. (2015). Enact the internet and social media on the public sector’s frontline. New Technology, Work and Employment, 30(3), 190-208. google scholar
• Murphy, G. & Flynn, E. (2021). Deepfake false memories. Memory, 30(4), 480-492. google scholar
• Murray, J. (2021). Deepfakes and digital avatars: the new celebrity brand ambassadors. Retrieved from https://www.campaignlive.co.uk/article/deepfakes-digital-avatars-new-celebrity-brand-ambassadors/1713070. google scholar
• MIT Open Learning (2020, Jul 20): Tackling the misinformation epidemic with “In Event of Moon Disaster”. Retrieved from https://news.mit.edu/2020/mit-tackles-misinformation-in-event-of-moon-disaster-0720. google scholar
• Murphy, G.; Ching, D.; Twomey, J., & Linehan C. (2023) Face/Off: Changing the face of movies with deepfakes. Plos One. 18(7). https://doi.org/10.1371/journal.pone.0287503 google scholar
• Negroponte, N. (1995). Being digital. Alfred A. Knopf. google scholar
• Nellis, S. & Dave, P. (2021, Apr 1). Microsoft wins $21.9 billion contract with U.S. Army to supply augmented reality headsets. Reuters. Retrieved from https://www.reuters.com/article/us-microsoft-army/microsoft-wins-21-9-billion-contract-with-u-s-army-to-supply-augmented-reality-headsets-idUSKBN2BN36B/ google scholar
• Nojeim, G. ,& Laperruque, J. (2023, Dec 8). Government surveillance: A trojan horse in a house Intel google scholar
• committee bill massively expands FISA 702 surveillance. Center for Democracy & Technology. Retrieved from https://cdt.org/insights/a-trojan-horse-in-a-house-intel-committee-bill-massively-expands-fisa-702-surveillance/ google scholar
• Nowak, K. L., & Fox, J. (2018). Avatars and computer-mediated communication: a review of the definitions, uses, and effects of digital representations. Review of Communication Research. 6. 30-53. google scholar
• Poster, M. (1995). The second machine age. Polity. google scholar
• M. S. Rana, M. N. Nobi, and B. Murali & Sung H. (2022) .Deepfake detection: A systematic literature review. IEEE Access 10, 25494-25513. doi: 10.1109/ACCESS.2022.3154404. google scholar
• Rodriguez Resendiz, H., & Rodriguez Resendiz, J. (2024). Digital resurrection: Challenging the boundary between life and death with artificial intelligence. Philosophies, 9(3), 71. google scholar
• Savin-Baden, M., & Mason-Robbie, V. (Eds.). (2020). Digital afterlife: Death matters in a digital age. CRC Press. google scholar
• Sherlock, A. (2013). Larger than life: Digital resurrection and the re-enchantment of society. The Information Society, 29(3), pp. 164-176. google scholar
• Schmidt, E., & Cohen, J. (2013). The new digital age: Reshaping the future of people, nations and business. John Murray. google scholar
• Schwab, K. (2016). The fourth industrial revolution. Crown publishing. google scholar
• Sentinel (2020, Oct). Deepfaked 2020: The tipping point: The current threat landscape, its impact on the U.S. 2020 elections, and the coming of AI-generated events at scale. [Report]. Retrieved from https://thesentinel.ai/media/Deepfakes%202020:%20Tipping%20Point,%20Sentinel.pdf google scholar
• Somers, M. (2020, Jul 21) Deepfakes, explained. MIT Management Sloan School. Retrieved from https://mitsloan.mit.edu/ideas-made-to-matter/deepfakes-explained google scholar
• Suchman, L. (2012, online publication), (2006, print publication). Human-machine reconfigurations: Plans and situated actions. Cambridge University Press. google scholar
• Talvitie-Lamberg, K. (2018). Video streaming and internalized surveillance. Surveillance and Society, 16(2). google scholar
• Tariq, S., Abuadbba, A. and Moore, K. (2023, July). Deepfake in the metaverse: Security implications for virtual gaming, meetings, and offices. Proceedings of the 2nd Workshop on Security Implications of Deepfakes and Cheapfakes (pp. 16-19). Retrieved from https://doi.org/10.1145/3595353.3595880 google scholar
• Yu, P., Xia, Z., Fei, J. & Lu, Y. (2021). A survey on deepfake video detection. IET Biometrics: The Institue of Engineering and Technology. 10(6). 607-624. google scholar
• Watts, D. J. (2014). Common Sense and Sociological Explanations. American Journal of Sociology. 120(2): 313-351. google scholar
• Westling, J. (2019). Are Deep Fakes a Shallow Concern? A critical analysis of the likely societal reaction to deep fakes. TPRC47: The 47th Research Conference on Communication, Information, and Internet Policy; 2019. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3426174. google scholar
• Whitaker, R. (1999). The end of privacy: How total surveillance is becoming a reality. New York: The New Press. google scholar
• Vaccari, C., & Chadwick, A. (2020). Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. Social media+ society, 6(1), 2056305120903408. google scholar
• Zuboff, S. (2015). Big other: surveillance capitalism and the prospects of an information civilization. Journal of Information Technology. 30. 75-89. google scholar