Perceptual Judgment in AI-Assisted VFX Compositing: A Conceptual Review of Technical Capabilities, Cognitive Foundations, and Automation Boundaries

Abstract

This conceptual review examines the integration of artificial intelligence (AI) tools into visual effects (VFX) compositing workflows across three dimensions: (i) the current technical capabilities of AI tools, including generative compositing, deep image harmonization, and neural rendering; (ii) the cognitive science and human factors foundations of human perceptual judgment; and (iii) a proposed conceptual taxonomy organizing the gap between these two domains. Drawing on situation awareness theory, trust in automation research, the ironies of automation, and studies of tacit knowledge in visual expertise, the review structures compositing processes into a five-level taxonomy: Automated Visual Processing (L1), Physical Rendering Consistency (L2), Temporal-Sequence Consistency (L3), Semantic-Narrative Alignment (L4), and Human Perceptual Decision (L5). The taxonomy is explicitly positioned as a domain-specific manifestation of established human-automation interaction frameworks, notably Sheridan's levels of automation and supervisory control, adapted to the unique aesthetic and narrative dimensions of VFX compositing. The evidence reviewed suggests that while generative AI and neural rendering tools expand automation reach at Levels 1–2, high-level perceptual decisions requiring temporal, semantic, and narrative consistency (L4–L5) remain predominantly human-dependent under current paradigms. However, this may reflect temporal contingency rather than structural inevitability. A critical implication, underexplored in the literature, is the deskilling risk: if AI absorbs L1–L3 tasks, junior compositors may not develop the tacit perceptual expertise required for L4–L5 judgments. Rather than eliminating perceptual expertise, current AI technologies appear to redistribute the compositor's role from technical executor to perceptual supervisor, with significant implications for training, trust calibration, and professional development.

Keywords

• Artificial Intelligence
• Visual Effects
• Compositing
• Supervisory Control
• Trust in Automation

Yapay Zeka Destekli Görsel Efekt Birleştirme İşlemlerinde Algısal Yargı: Teknik Yetenekler, Bilişsel Temeller ve Otomasyon Sınırlarına İlişkin Kavramsal Bir İnceleme

Abstract

Bu kavramsal derleme, yapay zekâ (YZ) araçlarının görsel efekt (VFX) compositing akışlarına entegrasyonunu üç boyutta incelemektedir: (i) üretken compositing, derin görüntü harmonizasyonu ve nöral renderlama dahil olmak üzere YZ araçlarının mevcut teknik kapasiteleri; (ii) insan algısal yargılarının bilişsel bilim ve insan faktörleri temelleri; ve (iii) bu iki alan arasındaki boşluğu organize eden kavramsal bir taksonomi önerisi. Durum farkındalığı teorisi, otomasyona güven araştırmaları, otomasyon ironileri ve görsel uzmanlıkta örtük bilgi çalışmalarından yararlanan bu inceleme, compositing süreçlerini beş düzeyli bir taksonomi altında yapılandırmaktadır. İncelenen kanıtlar, üretken YZ ve nöral renderlama araçlarının Düzey 1–2'de otomasyon kapsamını genişletmesine rağmen, temporal, semantik ve anlatı tutarlılığı gerektiren yüksek düzeyli algısal kararların (D4–D5) mevcut paradigmalar altında ağırlıklı olarak insana bağımlı kaldığını düşündürmektedir. Literatürde yeterince ele alınmamış kritik bir çıkarım, beceri kaybı (deskilling) riskidir: YZ, D1–D3 görevlerini üstlenirse, kıdemsiz compositor'lar D4–D5 yargıları için gereken örtük algısal uzmanlığı geliştiremeyebilir. Mevcut YZ teknolojileri, algısal uzmanlığı ortadan kaldırmak yerine, compositor'ın rolünü teknik uygulayıcıdan algısal denetçiye doğru yeniden dağıtıyor gibi görünmektedir.

Keywords

• Yapay Zeka
• Görsel Efektler
• Kompozitleme
• Denetleyici Kontrol
• Otomasyona Güven

References

1. Amabile, T. M. (1996). Creativity in context. Westview Press.
2. Ameur, Z., Lefebvre, F., De Lagrange, P., & Radosavljević, M. (2025). FGA-NN: Film grain analysis neural network. arXiv. https://doi.org/10.48550/arXiv.2506.14350
3. Andrew, D. (1984). Concepts in film theory. Oxford University Press.
4. Armengol-Urpi, A., Salazar-Gomez, A. F., & Sinha, P. (2026). Cognitive reinforcement: Capturing tacit knowledge and enhancing expertise with a biofeedback interface for visual attention. Journal of Neural Engineering, 23(2), Article 026002. https://doi.org/10.1088/1741-2552/ae3eb8
5. Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775–779. https://doi.org/10.1016/0005-1098(83)90046-8
6. Benner, P. (1984). From novice to expert: Excellence and power in clinical nursing practice. Addison-Wesley.
7. Boden, M. A. (1990). The creative mind: Myths and mechanisms. Basic Books.
8. Bordwell, D. (1985). Narration in the fiction film. University of Wisconsin Press.

9. Bordwell, D., & Thompson, K. (2008). Film art: An introduction (8th ed.). McGraw-Hill.
10. Brinkmann, R. (2008). The art and science of digital compositing (2nd ed.). Morgan Kaufmann.
11. Bruce, V., Green, P. R., & Georgeson, M. A. (2003). Visual perception: Physiology, psychology and ecology (4th ed.). Psychology Press.
12. Casner, S. M., Geven, R. W., Recker, M. P., & Schooler, J. W. (2014). The retention of manual flying skills in the automated cockpit. Human Factors, 56(8), 1506–1516. https://doi.org/10.1177/0018720814535628
13. Christiansen, R. (2016, August 25). The number 1 skill you can learn to become a great compositor: Color matching. Frame.io Insider. https://blog.frame.io/2016/08/25/first-essential-vfx-compositing-matching/
14. Clark, A. (1997). Being there: Putting brain, body, and world together again. MIT Press.
15. Collins, H. (2010). Tacit and explicit knowledge. University of Chicago Press.
16. Cong, W., Zhang, J., Niu, L., Liu, L., Ling, Z., Li, W., & Zhang, L. (2020). DoveNet: Deep image harmonization via domain verification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 8394–8403). https://doi.org/10.1109/cvpr42600.2020.00842
17. Csikszentmihalyi, M. (1996). Creativity: Flow and the psychology of discovery and invention. HarperCollins.
18. Dickson, B., Maini, S. S., Sanders, C., Nosofsky, R., & Tiganj, Z. (2025). Comparing perceptual judgments in large multimodal models and humans. Behavior Research Methods, 57(7), Article 203. https://doi.org/10.3758/s13428-025-02728-w
19. Dreyfus, H. L. (1972). What computers can't do: A critique of artificial reason. Harper & Row.
20. Dreyfus, H. L. (1992). What computers still can't do: A critique of artificial reason. MIT Press.
21. Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32–64. https://doi.org/10.1518/001872095779049543
22. Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363–406. https://doi.org/10.1037/0033-295X.100.3.363
23. Evans, J. St. B. T. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59, 255–278. https://doi.org/10.1146/annurev.psych.59.103006.093629
24. Failes, I. (2022, January 25). How Framestore used machine learning techniques in their comps on 'The Matrix Resurrections'. befores & afters. https://beforesandafters.com/2022/01/25/how-framestore-used-machine-learning-techniques-in-their-comps-on-the-matrix-resurrections/
25. Fairchild, M. D. (2013). Color appearance models (3rd ed.). Wiley.
26. Fast-Berglund, Å., Thorvald, P., & Billing, E. (2018). Conceptualizing embodied automation to increase transfer of tacit knowledge in the learning factory. In Proceedings of the 2018 IEEE International Conference on Intelligent Systems (IS) (pp. 358–364). https://doi.org/10.1109/is.2018.8710482
27. Fleming, R. W. (2014). Visual perception of materials and their properties. Vision Research, 94, 62–75. https://doi.org/10.1016/j.visres.2013.11.004
28. Freeman, J., & Simoncelli, E. P. (2011). Metamers of the ventral stream. Nature Neuroscience, 14(9), 1195–1201. https://doi.org/10.1038/nn.2889
29. Fu, S., Tamir, N., Sundaram, S., Chai, L., Zhang, R., Dekel, T., & Isola, P. (2023). DreamSim: Learning new dimensions of human visual similarity using synthetic data. Advances in Neural Information Processing Systems (NeurIPS 2023).
30. Gao, X., Jiang, S., & Liu, B. (2026). VEFX-Bench: A holistic benchmark for generic video editing and visual effects. arXiv. https://doi.org/10.48550/arXiv.2604.16272
31. Gibson, J. J. (1979). The ecological approach to visual perception. Houghton Mifflin.
32. Goldstein, E. B. (2010). Sensation and perception (8th ed.). Wadsworth.
33. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
34. Gu, J., Cai, H., & Dong, C. (2021). NTIRE 2021 challenge on perceptual image quality assessment. arXiv. https://doi.org/10.48550/arXiv.2105.03072
35. Hoff, K. A., & Bashir, M. (2014). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/10.1177/0018720814547570
36. Johnson, T., Fletcher, S., & Baker, W. H. (2019). How and why we need to capture tacit knowledge in manufacturing: Case studies of visual inspection. Applied Ergonomics, 74, 1–9. https://doi.org/10.1016/j.apergo.2018.07.016
37. Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.
38. Kingdom, F. A. A., & Prins, N. (2016). Psychophysics: A practical introduction (2nd ed.). Academic Press.
39. Klein, G. A. (1993). A recognition-primed decision (RPD) model of rapid decision making. In G. A. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 138–147). Ablex.
40. Koffka, K. (1935). Principles of Gestalt psychology. Harcourt, Brace.
41. Köhler, W. (1947). Gestalt psychology. Liveright.
42. Kubilius, J., Bracci, S., & Op de Beeck, H. P. (2016). Deep neural networks as a computational model for human shape sensitivity. PLOS Computational Biology, 12(4), Article e1004896. https://doi.org/10.1371/journal.pcbi.1004896
43. Lavoué, G., Larabi, M. C., & Váša, L. (2016). On the efficiency of image metrics for evaluating the visual quality of 3D models. IEEE Transactions on Visualization and Computer Graphics, 22(8), 1987–1999. https://doi.org/10.1109/TVCG.2015.2480079
44. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
45. Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392
46. Li, W., Viola, F., Starck, J., Brostow, G. J., & Campbell, N. D. F. (2016). Roto++: Accelerating professional rotoscoping using shape manifolds. ACM Transactions on Graphics, 35(4), 1–15. https://doi.org/10.1145/2897824.2925973
47. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390(6657), 279–281. https://doi.org/10.1038/36846
48. Mantiuk, R., Kim, K. J., Rempel, A. G., & Heidrich, W. (2011). HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Transactions on Graphics, 30(4), 1–14. https://doi.org/10.1145/2010324.1964935
49. Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. W. H. Freeman.
50. McCullauch, J. (2025, January 7). AI/VFX roundtable: Revolutionizing imagery – the future of AI and newer tech in VFX. VFX Voice. https://vfxvoice.com/ai-vfx-roundtable-revolutionizing-imagery-the-future-of-ai-and-newer-tech-in-vfx/
51. Nadler, E. O., Darragh-Ford, E., Desikan, B. S., Conaway, C., Chu, M., Hull, T., & Guilbeault, D. (2023). Divergences in color perception between deep neural networks and humans. Cognition, 241, Article 105621. https://doi.org/10.1016/j.cognition.2023.105621
52. Narayan, A. D., Caillard, D., Matthews, J., & Nairn, A. (2022). Artificial imagination: Industry attitudes on the impact of AI on the visual effects process. Interactions: Studies in Communication & Culture, 13(2), 113–131. https://doi.org/10.1386/iscc_00056_1
53. Niu, L., Cong, W., Liu, L., Hong, Y., Zhang, B., Liang, J., & Zhang, L. (2021). Making images real again: A comprehensive survey on deep image composition. arXiv. https://doi.org/10.48550/arXiv.2106.14490
54. Nonaka, I., & Takeuchi, H. (1995). The knowledge-creating company. Oxford University Press.
55. Norman, D. A. (1988). The design of everyday things. Doubleday.
56. Onnasch, L., Wickens, C. D., Li, H., & Manzey, D. (2014). Human performance consequences of stages and levels of automation: An integrated meta-analysis. Human Factors, 56(3), 476–488. https://doi.org/10.1177/0018720813501549
57. Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4. https://doi.org/10.1207/S15326985EP3801_1
58. Palmer, S. E. (1999). Vision science: Photons to phenomenology. MIT Press.
59. Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. https://doi.org/10.1518/001872097778543886
60. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/3468.844354
61. Patterson, R., Pierce, B. J., & Bell, H. H. (2010). Implicit learning, tacit knowledge, expertise development, and naturalistic decision making. Journal of Cognitive Engineering and Decision Making, 4(4), 289–303. https://doi.org/10.1177/155534341000400403
62. Polanyi, M. (1966). The tacit dimension. Doubleday.
63. Pylyshyn, Z. W. (1999). Vision and cognition: How do they intersect? Behavioral and Brain Sciences, 22(3), 443–444. https://doi.org/10.1017/S0140525X99242084
64. Qi, L., Wu, J., & Choi, J. M. (2025). Over++: Generative video compositing for layer interaction effects. arXiv. https://doi.org/10.48550/arXiv.2512.19661
65. Rasmussen, J. (1983). Skills, rules, and knowledge: Signals, signs, and symbols, and other distinctions in human performance models. IEEE Transactions on Systems, Man, and Cybernetics, 13(3), 257–266. https://doi.org/10.1109/TSMC.1983.6313160
66. Reinhard, E., Khan, E. A., Akyüz, A. O., & Johnson, G. (2008). Color imaging: Fundamentals and applications. A K Peters.
67. Rickitt, R. (2006). Special effects: The history and technique. Billboard Books.
68. Ring, D., Barbier, J., Gales, G., Kent, B., & Lutz, S. (2019). Jumping in at the deep end: How to experiment with machine learning in post-production software. In DigiPro '19: Proceedings of the 2019 Digital Production Symposium (pp. 1–5). https://doi.org/10.1145/3329715.3338880
69. Schön, D. A. (1983). The reflective practitioner: How professionals think in action. Basic Books.
70. Sheridan, T. B. (1992). Telerobotics, automation, and human supervisory control. MIT Press.
71. Sheridan, T. B. (2017). Comments on "Issues in Human-Automation Interaction Modeling." Journal of Cognitive Engineering and Decision Making, 12(1), 25–28. https://doi.org/10.1177/1555343417724964
72. Sheridan, T. B., & Verplank, W. L. (1978). Human and computer control of undersea teleoperators (Technical Report). MIT Man-Machine Systems Laboratory.
73. Smith, T. J. (2012). The attentional theory of cinematic continuity. Projections, 6(1), 1–27. https://doi.org/10.3167/proj.2012.060102
74. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4
75. Tao, Y., Yang, W., & Xu, J. (2024). Barriers to industry adoption of AI video generation tools: A study based on the perspectives of video production professionals in China. Applied Sciences, 14(13), Article 5770. https://doi.org/10.3390/app14135770
76. Tewari, A., Thies, J., Mildenhall, B., Srinivasan, P., Tretschk, E., Wang, Y., Lassner, C., Sitzmann, V., Martin-Brualla, R., Lombardi, S., Simon, T., Theobalt, C., Nießner, M., Barron, J. T., Wetzstein, G., Zollhöfer, M., & Golyanik, V. (2022). Advances in neural rendering. Computer Graphics Forum, 41(2), 703–735. https://doi.org/10.1111/cgf.14507
77. Thies, J., Zollhöfer, M., & Nießner, M. (2019). Deferred neural rendering: Image synthesis using neural textures. ACM Transactions on Graphics, 38(4), 1–12. https://doi.org/10.1145/3306346.3323035
78. Tous, R. (2024). Lester: Rotoscope animation through video object segmentation and tracking. Algorithms, 17(8), Article 330. https://doi.org/10.3390/a17080330
79. Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97–136. https://doi.org/10.1016/0010-0285(80)90005-5
80. Tsai, Y.-H., Shen, X., Lin, Z., Sunkavalli, K., Lu, X., & Yang, M.-H. (2017). Deep image harmonization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3789–3797). https://doi.org/10.1109/cvpr.2017.299
81. Varela, F. J., Thompson, E., & Rosch, E. (1991). The embodied mind: Cognitive science and human experience. MIT Press.
82. Wertheimer, M. (1923). Untersuchungen zur Lehre von der Gestalt, II. Psychologische Forschung, 4, 301–350.
83. Woods, D. D. (1996). Decomposing automation: Apparent simplicity, real complexity. In R. Parasuraman & M. Mouloua (Eds.), Automation and human performance: Theory and applications (pp. 3–17). Lawrence Erlbaum.
84. Wright, S. (2017). Digital compositing for film and video (4th ed.). Routledge.
85. Yuce, D. (2026). Deep learning-based rotoscoping: A systematic review of methods and applications. Journal of Mechatronics and Artificial Intelligence in Engineering, 7(1), 25–44. https://doi.org/10.21595/jmai.2026.26653
86. Zeki, S. (1999). Inner vision: An exploration of art and the brain. Oxford University Press.
87. Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2018.00068