Transforming Sketches into Realistic Images: Leveraging Machine Learning and Image Processing for Enhanced Architectural Visualization

Year 2023, Volume: 27 Issue: 6, 1209 - 1216, 18.12.2023

İlker Karadağ

https://doi.org/10.16984/saufenbilder.1319166

Cited By: 1

Abstract

This article presents a novel approach for transforming architectural sketches into realistic images through the utilization of machine learning and image processing techniques. The proposed method leverages the Stable Diffusion model, a deep learning framework specifically designed for text-to-image generation. By integrating image processing algorithms into the workflow, the model gains a better understanding of the input sketches, resulting in visually coherent and meaningful output images. The study explores the application of the Stable Diffusion model in the context of architectural design, showcasing its potential to enhance the visualization process and support designers in generating accurate and compelling representations. The efficacy of the method is evaluated through qualitative assessment, demonstrating its effectiveness in bridging the gap between initial sketches and photorealistic renderings. This research contributes to the growing body of knowledge on the integration of machine learning and image processing in architecture, providing insights and practical implications for architects, design professionals and researchers in the field.

Keywords

Architectural visualization, sketch-to-image transformation, machine learning, image processing, stable diffusion model.

References

• [1] Frazer, J 1995, Evolutionary Architecture. London: Architectural Association, 1995.
• [2] Chaillou, S, “AI + Architecture | Towards a New Approach,” Master’s Thesis, Dept. Arch., Harvard University. Cambridge, MA, 2019.
• [3] I. Karadag, O. Z. Güzelci, S. Alaçam, “EDU-AI: a twofold machine learning model to support classroom layout generation,” Construction Innovation, Sep. 2022.
• [4] E. Kurucay, I. Karadag, “Computational Approaches in 21st Century Architectural Design: Defining Digital Representation Methods,” Duzce University Journal of Science and Technology, vol. 10, no. 3, pp. 1201– 1217, Jul. 2022.
• [5] N. Dehouche, K. Dehouche, “What’s in a text-to-image prompt? The potential of stable diffusion in visual arts education,” Heliyon, vol. 9, no. 6, p. e16757, Jun. 2023.
• [6] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, Oct. 2020.
• [7] Ho, J., Jain, A., Abbeel, P. “Advances in Neural Information Processing Systems”, in Denoising Diffusion Probabilistic Models, Vancouver, Canada, 2023, pp. 6840–6851.
• [8] P. A. Geroski, “Models of technology diffusion,” Research Policy, vol. 29, no. 4–5, pp. 603–625, Apr. 2000.
• [9] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2022.
• [10] C. Meng, Y. He, Y. Song, J. Song, J. Wu, J. Zhu, S. Ermon, “SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations”, arXiv [cs.CV]. 2022.
• [11] AUTOMATIC1111. (2022, Nov. 10). Feature showcase for stable-diffusionwebui, GitHub [Online]. Available at: https://github.com/AUTOMATIC1111/ stable-diffusion-webui-featureshowcase (Accessed: 12 June 2023). [12] L. Zhang, M. Agrawala, “Adding Conditional Control to Text-to-Image Diffusion Models”, arXiv [cs.CV]. 2023.