Building-integrated photovoltaics (BIPV) on facades are a significant but underutilized source of solar energy in urban environments. Automating the recognition of BIPV on facades through vision based systems can help guide design recommendations and extend solar asset maps. Unlike rooftop photovoltaic (PV) systems, facade BIPV recognition is difficult due to limited visibility in overhead imagery, high visual variability, and the absence of structured datasets. This study proposes a method based on deep learning (DL) for automated segmentation of BIPV panels on building facades using street-level and web images. A new benchmark data set comprising 400 annotated BIPV projects was created, including detailed pixel-level masks and project attributes. Two model architectures, Mask Region-based Convolutional Neural Network (Mask R-CNN) and SegFormer, as well as human baselines are evaluated. The SegFormer model outperforms Mask R-CNN in pixel-level metrics. A user study conducted with human annotators without domain-specific expertise provides comparative insight into human performance, revealing common challenges in recognizing facade BIPV. The results demonstrate that DL models, trained specifically for this task, can segment BIPV panels more accurately than mean of human annotators, with SegFormer achieving an IoU of 0.69 compared to 0.42. The user study suggests that BIPV with satin finishes, invisible cells, and a PV-to-facade ratio of more than half challenge human recognition and therefore can be prioritized in visually sensitive areas. The outputs of the segmentation model is also utilized to estimate the BIPV energy yield. The annotated dataset and models are made available to facilitate future research. • Compare deep learning models with human baselines for facade PV segmentation. • SegFormer achieves 0.69 IoU, outperforming Mask R-CNN and average human annotators. • Recent photovoltaic materials blend into facades, challenging human and model perception. • Release metadata for 400 facade PV projects with links to 665 images and PV masks. • Estimate potential energy yield of BIPV facades using deep learning and images.
Duran et al. (Sat,) studied this question.