Improved YOLO for sky detection in urban environments: An innovative approach to complex scenarios

Abstract

Accurate sky identification in street images is crucial for calculating the sky view factor (SVF) and understanding heat effects in urban environments. However, variable weather conditions—such as overcast, sunny, and rainy scenarios—pose significant challenges to this task. Additional difficulties arise due to limited visible sky regions, shadows, and color similarities between the sky and surrounding elements like buildings or walls. This study proposes an Improved YOLO model featuring a Swin Transformer backbone to address these issues, effectively capturing contextual information under complex and diverse weather conditions. The model incorporates SPPF as the neck to aggregate multi-scale features and GS-ELAN as the head to enhance information flow and feature sharing. A Segmentation Head is also integrated to provide precise pixel-level sky predictions. Data augmentation techniques simulating various weather and perspective conditions were employed to improve robustness. Experimental results reveal that the Improved YOLO achieves high performance with Precision, Recall, mAP, and F1 scores of 0.87, 0.94, 0.87, and 0.95, respectively. Compared to YOLOv11, these results show notable improvements of 5.43%, 14.63%, 7.41%, and 9.20% across key metrics. Despite the widespread use of semantic segmentation models for SVF estimation, YOLO-based architectures for sky view factor detection in perspective street images remain limited. This study addresses this gap with an improved YOLOv11 model. The model’s strong performance under diverse environmental conditions demonstrates its effectiveness for real-time sky detection, offering promising applications in urban planning and environmental research. Overall, this work contributes significantly to urban heat island research by enabling accurate and efficient assessment of sky view factors in urban areas.

Keywords

• Sky view factor
• Deep learning
• Urban analytics
• Complex conditions
• Semantic segmentation

References

1. Bi, K., Hao, H., & Chouw, N. (2013). 3D FEM analysis of pounding response of bridge structures at a canyon site to spatially varying ground motions. Advances in Structural Engineering, 16(4), 619-640. https://doi.org/10.1260/1369-4332.16.4.619
2. Biljecki, F., & Ito, K. (2021). Street view imagery in urban analytics and GIS: A review. Landscape and Urban Planning, 215, 104217. https://doi.org/10.1016/j.landurbplan.2021.104217
3. Cao, R., Fukuda, T., & Yabuki, N. (2019). Quantifying Visual Environment by Semantic Segmentation Using Deep Learning: A Prototype for Sky View Factor. Proceedings of the 24th International Conference of the Association for Computer-Aided Architectural Design Research in Asia (CAADRIA), Singapore, 623–632.
4. Chatzipoulka, C., Compagnon, R., Kaempf, J., & Nikolopoulou, M. (2018). Sky view factor as predictor of solar availability on building façades. Solar Energy, 170, 1026–1038. https://doi.org/10.1016/j.solener.2018.06.028
5. Cheung, H. K. W., Coles, D., & Levermore, G. J. (2016). Urban heat island analysis of Greater Manchester, UK using sky view factor analysis. Building Services Engineering Research and Technology, 37(1), 5-17. https://doi.org/10.1177/0143624415588890
6. Du, K., Ning, J., & Yan, L. (2020). How long is the sun duration in a street canyon? Analysis of the view factors of street canyons. Building and Environment, 172, 106680. https://doi.org/10.1016/j.buildenv.2020.106680
7. Fujiwara, K., Ito, K., Ignatius, M., & Biljecki, F. (2024). A panorama-based technique to estimate sky view factor and solar irradiance considering transmittance of tree canopies. Building and Environment, 266, 112071. https://doi.org/10.1016/j.buildenv.2024.112071
8. Gong, F. Y., Zeng, Z. C., Zhang, F., Li, X., Ng, E., & Norford, L. K. (2018). Mapping sky, tree, and building view factors of street canyons in a high-density urban environment. Building and Environment, 134, 155-167. https://doi.org/10.1016/j.buildenv.2018.02.042

9. Gong, F. Y., Zeng, Z.-C., Ng, E., & Norford, L.K. (2019). Spatiotemporal patterns of street-level solar radiation estimated using Google Street View in a high-density urban environment. Building and Environment, 148, 547–566. https://doi.org/10.1016/j.buildenv.2018.10.025
10. Hämmerle, M., Gál, T., Unger, J., & Matzarakis, A. (2011). Comparison of models calculating the sky view factor used for urban climate investigations. Theoretical and Applied Climatology, 105(1-2), 521–527. https://doi.org/10.1007/s00704-011-0402-3
11. Hu, C.-B., Zhang, F., Gong, F.-Y., Ratti, C., & Li, X. (2020). Classification and mapping of urban canyon geometry using google street view images and deep multitask learning. Building and Environment, 167, 106424. https://doi.org/10.1016/j.buildenv.2019.106424
12. Jiao, Z. H., Ren, H., Mu, X., Zhao, J., Wang, T., & Dong, J. (2019). Evaluation of four sky view factor algorithms using digital surface and elevation model data. Earth and Space Science, 6(2), 222-237. https://doi.org/10.1029/2018EA000475
13. Kastendeuch, P. P. (2013). A method to estimate sky view factors from digital elevation models. International Journal of Climatology, 33(6), 1574-1578. https://doi.org/10.1002/joc.3523
14. Lee, K., & Levermore, G. J. (2020). Estimation of surface solar irradiation using sky view factor, sunshine factor and solar irradiation models according to geometry and buildings. Advances in Building Energy Research, 14(2), 189-201. https://doi.org/10.1080/17512549.2019.1591299
15. Li, G., Ren, Z., & Zhan, C. (2020). Sky view factor-based correlation of landscape morphology and the thermal environment of street canyons: A case study of Harbin, China. Building and Environment, 169, 106587. https://doi.org/10.1016/j.buildenv.2019.106587
16. Li, X., Yoshimura, Y., Tu, W., & Ratti, C. (2022). A pedestrian-level strategy to minimize outdoor sunlight exposure. Artificial intelligence, machine learning, and optimization tools for smart cities: Designing for sustainability, Springer Nature.
17. Liang, J., Gong, J., Sun, J., Zhou, J., Li, W., Li, Y., Liu, J., & Shen, S. (2017). Automatic sky view factor estimation from street view photographs—A big data approach. Remote Sensing, 9(5), 411. https://doi.org/10.3390/rs9050411
18. Liang, J., Gong, J., Zhang, J., Li, Y., Wu, D., & Zhang, G. (2020). GSV2SVF-an interactive GIS tool for sky, tree and building view factor estimation from street view photographs. Building and Environment, 168, 106475. https://doi.org/10.1016/j.buildenv.2019.106475
19. Liu, Y., Zhang, M., Li, Q., Zhang, T., Yang, L., & Liu, J. (2021). Investigation on the distribution patterns and predictive model of solar radiation in urban street canyons with panorama images. Sustainable Cities and Society, 75, 103275. https://doi.org/10.1016/j.scs.2021.103275
20. Matzarakis, A., & Matuschek, O. (2011). Sky view factor as a parameter in applied climatology-rapid estimation by the SkyHelios model. Meteorologische Zeitschrift, 20(1), 39.
21. Miao, C., Yu, S., Hu, Y., Zhang, H., He, X., & Chen, W. (2020a). Review methods used to estimate the sky view factor in urban street canyons. Building and Environment, 168, 106497. https://doi.org/10.1016/j.buildenv.2019.106497
22. Nishio, S., & Ito, F. (2019). Application of method for calculating sky view factor using google street view: relation between sky view factor and physical elements in urban space. Proceedings of the ICA, Göttingen, Germany.
23. Park, S., & Tuller, S. E. (2014). Advanced view factor analysis method for radiation exchange. International journal of biometeorology, 58, 161-178. https://doi.org/10.1007/s00484-013-0657-8
24. Rubio-Bellido, C., Pulido-Arcas, J. A., & Sánchez-Montañés, B. (2015). A simplified simulation model for predicting radiative transfer in long street canyons under high solar radiation conditions. Energies, 8(12), 13540-13558. https://doi.org/10.3390/en81212383
25. Sharma, A., Kumar, V., & Longchamps, L. (2024). Comparative performance of YOLOv8, YOLOv9, YOLOv10, YOLOv11 and Faster R-CNN models for detection of multiple weed species. Smart Agricultural Technology, 100648. https://doi.org/10.1016/j.atech.2024.100648
26. Shuai, L., Mu, J., Jiang, X., Chen, P., Zhang, B., Li, H., ... & Li, Z. (2023). An improved YOLOv5-based method for multi-species tea shoot detection and picking point location in complex backgrounds. Biosystems Engineering, 231, 117-132. https://doi.org/10.1016/j.biosystemseng.2023.06.007 https://doi.org/10.1016/j.biosystemseng.2023.06.007
27. Song, Y., Zhang, T., & Qi, F. (2024). A correction method for calculating sky view factor in urban canyons using fisheye images. Building and Environment, 262, 111834. https://doi.org/10.1016/j.buildenv.2024.111834
28. Souza, L. C. L., Rodrigues, D. S., & Mendes, J. F. (2003). Sky view factors estimation using a 3D-GIS extension. Proceeding Book of Eighth International IBPSA Conference, Eindhoven, Netherlands, 1227-1234.
29. Xia, Y., Yabuki, N., & Fukuda, T. (2021). Sky view factor estimation from street view images based on semantic segmentation. Urban Climate, 40, 100999. https://doi.org/10.1016/j.uclim.2021.100999
30. Xu, H., Lu, H., & Liu, S. (2024). Online street view-based approach for sky view factor estimation: A case study of Nanjing, China. Applied Sciences, 14(5), 2133. https://doi.org/10.3390/app14052133
31. Xu, Y., Imou, K., Kaizu, Y., & Saga, K. (2013). Two-stage approach for detecting slightly overlapping strawberries using HOG descriptor. Biosystems engineering, 115(2), 144-153. https://doi.org/10.1016/j.biosystemseng.2013.03.011
32. Yu, T., Chen, H., Li, Z., He, Q., & Lin, B. (2021, August). An efficient method of evaluating large scale urban residential skylight environment and an empirical study of Beijing main area. Building Simulation, 14, 871-883. https://doi.org/10.1007/s12273-020-0704-4
33. Zeng, L., Lindberg, F., Zhang, X., Pan, H., & Lu, J. (2023). Road surface temperature evaluated with streetview-derived parameters in a hot and humid megacity. Urban Climate, 51, 101585. https://doi.org/10.1016/j.uclim.2023.101585
34. Zeng, L., Lu, J., Li, W., & Li, Y. (2018). A fast approach for large-scale Sky View Factor estimation using street view images. Building and Environment, 135, 74-84. https://doi.org/10.1016/j.buildenv.2018.03.009
35. Zhao, X., Lu, Y., & Lin, G. (2024). An integrated deep learning approach for assessing the visual qualities of built environments utilizing street view images. Engineering Applications of Artificial Intelligence, 130, 107805.
36. Zou, J., Jiang, H., Ying, W., & Qiu, B. (2024). Scenic influences on walking preferences in urban forest parks from top-view and eye-level perspectives. Forests, 15(11), 2020. https://doi.org/10.3390/f15112020
37. Zou, X., & Hu, Y. (2024). Hidden danger detection and identification system of power transmission tower based on YOLOV11. Academic Journal of Science and Technology, 13(1), 224-231.