Comparative Analysis of U-Net and ResNet for MangaColorization: Consistency, Detail, and Computational Trade-offs

Abstract

The automated colorization of manga presents unique challenges due to its distinctive artistic style and complex visuals. While deep learning has shown promise in image colorization, existing approaches often struggle with consistency and artistic integrity of manga artwork. This paper presents a comparative analysis of two deep learning architectures for manga colorization: a modified U-Net with progressive dropout and a ResNet-based autoencoder with adaptive skip connections. We introduce a novel composite loss function that specifically addresses manga-specific challenges by incorporating structural and perceptual components. Experiments on a diverse manga dataset show that the ResNet-based model achieves higher color consistency and better stability, producing fewer artifacts in uniform areas. However, U-Net preserves fine details more effectively. These results provide insights into trade-offs between architectures, guiding practical implementations of manga colorization systems.

Keywords

• Manga colorization
• Deep learning
• U-Net
• ResNet
• Automated artistic processing

Manga için U-Net ve ResNet'in Karşılaştırmalı Analizi Renklendirme: Tutarlılık, Detay ve Hesaplamalı Ödünleşimler

Abstract

Manganın otomatik olarak renklendirilmesi, kendine özgü sanatsal tarzı ve karmaşık görselliği nedeniyle benzersiz zorluklar ortaya koymaktadır. Derin öğrenme görüntü renklendirmede umut vaat etse de, mevcut yaklaşımlar genellikle manga sanat eserlerinin tutarlılığı ve sanatsal bütünlüğü ile mücadele etmektedir. Bu makale, manga renklendirme için iki derin öğrenme mimarisinin karşılaştırmalı bir analizini sunmaktadır: aşamalı bırakma özelliğine sahip değiştirilmiş bir U-Net ve uyarlanabilir atlama bağlantılarına sahip ResNet tabanlı bir otomatik kodlayıcı. Yapısal ve algısal bileşenleri bir araya getirerek mangaya özgü zorlukları özellikle ele alan yeni bir bileşik kayıp fonksiyonu sunuyoruz. Farklı bir manga veri kümesi üzerinde yapılan deneyler, ResNet tabanlı modelin daha yüksek renk tutarlılığı ve daha iyi kararlılık sağladığını ve tek tip alanlarda daha az yapaylık ürettiğini gösteriyor. Bununla birlikte, U-Net ince ayrıntıları daha etkili bir şekilde korur. Bu sonuçlar, manga renklendirme sistemlerinin pratik uygulamalarına rehberlik ederek mimariler arasındaki değiş tokuşlar hakkında fikir vermektedir.

Keywords

• Manga renklendirme
• Derin öğrenme
• U-Net
• ResNet
• Otomatik sanatsal işleme

References

1. Anime News Network, “Manga market in Japan hits record 612.6 billion yen in 2020,” Feb. 2021, archived Nov. 14, 2022, accessed Nov. 14, 2021. [Online]. Available: https://www.animenewsnetwork.com/news/2021-02-26/manga-market-in-japan-hits-record-612.6-billion-yen-in-2020/.169987.
2. S. Guadarrama, R. Dahl, D. Bieber, M. Norouzi, J. Shlens, and K. Murphy, “PixColor: Pixel recursive colorization,” arXiv:1705.07208, 2017.
3. M. Kumar, D. Weissenborn, and N. Kalchbrenner, “Colorization transformer,” arXiv:2102.04432, 2021.
4. T. Jiramahapokee, “Inkn’hue: Enhancing manga colorization from multiple priors with alignment multi-encoder VAE,” arXiv:2311.01804, 2023.
5. L. Zhang, Y. Ji, X. Lin, and C. Liu, “Style transfer for anime sketches with enhanced residual U-Net and auxiliary classifier GAN,” in Proc. 4th IAPR Asian Conf. Pattern Recognit. (ACPR), 2017, pp. 506–511.
6. Y. Ci, X. Ma, Z. Wang, H. Li, and Z. Luo, “User-guided deep anime line art colorization with conditional adversarial networks,” in Proc. 26th ACM Int. Conf. Multimedia, 2018, pp. 1536–1544.
7. Y. Wu, X. Wang, Y. Li, H. Zhang, X. Zhao, and Y. Shan, “Towards vivid and diverse image colorization with generative color prior,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2021, pp. 14377–14386.
8. N. Zabari, A. Azulay, A. Gorkor, T. Halperin, and O. Fried, “Diffusing colors: Image colorization with text guided diffusion,” in SIGGRAPH Asia Conf. Papers, 2023, pp. 1–11.

9. S. Anwar, M. Tahir, C. Li, A. Mian, F. S. Khan, and A. W. Muzaffar, “Image colorization: A survey and dataset,” arXiv:2008.10774, 2020.
10. M. B. Islam and M. I. Hosen, “Emotion, age and gender prediction through masked face inpainting,” in Int. Conf. Pattern Recognit., 2022, pp. 37–48.
11. M. I. Hosen, T. Aydin, and M. B. Islam, “WNet: A dual-encoded multi-human parsing network,” IET Image Process., vol. 18, no. 12, pp. 3316–3328, 2024.
12. M. I. Hosen and S. Kahraman, “Deep learning-based efficient drone detection and tracking in real-time,” in Proc. 33rd Signal Process. Commun. Appl. Conf. (SIU), 2025, pp. 1–4.
13. R. Zhang, P. Isola, and A. A. Efros, “Colorful image colorization,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2016, pp. 649–666.
14. S. Iizuka, E. Simo-Serra, and H. Ishikawa, “Let there be color! Joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification,” ACM Trans. Graph., vol. 35, no. 4, pp. 1–11, 2016.
15. C. Furusawa, K. Hiroshiba, K. Ogaki, and Y. Odagiri, “Comicolorization: Semi-automatic manga colorization,” in SIGGRAPH Asia Tech. Briefs, 2017, pp. 1–4.
16. L. Zhang, C. Li, T.-T. Wong, Y. Ji, and C. Liu, “Two-stage sketch colorization,” ACM Trans. Graph., vol. 37, no. 6, pp. 1–14, 2018.
17. Y. Li, M.-Y. Liu, X. Li, M.-H. Yang, and J. Kautz, “A closed-form solution to photorealistic image stylization,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2018, pp. 453–468.
18. H. Kim, H. Y. Jhoo, E. Park, and S. Yoo, “Tag2Pix: Line art colorization using text tag with SECat and changing loss,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2019, pp. 9056–9065.
19. Y. Cao, X. Meng, P. Mok, X. Liu, T.-Y. Lee, and P. Li, “AnimeDiffusion: Anime face line drawing colorization via diffusion models,” arXiv:2303.11137, 2023.
20. S. Chen, D. Li, Z. Bao, Y. Zhou, L. Tan, Y. Zhong, and Z. Zhao, “Manga generation via layout-controllable diffusion,” arXiv:2412.19303, 2024.
21. A. Maejima et al., “Continual few-shot patch-based learning for anime-style colorization,” Comput. Vis. Media, vol. 10, no. 4, pp. 705–723, 2024.
22. Y. Cao, X. Meng, P. Mok, T.-Y. Lee, X. Liu, and P. Li, “AnimeDiffusion: Anime diffusion colorization,” IEEE Trans. Vis. Comput. Graph., vol. 30, no. 10, pp. 6956–6969, 2024.
23. J. Lin, X. Liu, C. Li, M. Xie, and T.-T. Wong, “Sketch2Manga: Shaded manga screening from sketch with diffusion models,” in Proc. IEEE Int. Conf. Image Process. (ICIP), 2024, pp. 2389–2395.
24. J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2016, pp. 694–711.
25. L. A. Gatys, A. S. Ecker, and M. Bethge, “Image style transfer using convolutional neural networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 2414–2423.
26. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Interv. (MICCAI), 2015, pp. 234–241.
27. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 770–778.
28. I. Loshchilov and F. Hutter, “SGDR: Stochastic gradient descent with warm restarts,” arXiv:1608.03983, 2016.
29. M. I. Hosen and M. B. Islam, “Masked face inpainting through residual attention U-Net,” in Proc. Innov. Intell. Syst. Appl. Conf. (ASYU), 2022, pp. 1–5.
30. MichaelP84, “Manga-colorization-dataset,” Hugging Face Dataset, 2024, accessed Sep. 20, 2025. [Online]. Available: https://huggingface.co/datasets/MichaelP84/manga-colorization-dataset.