The global energy landscape is undergoing a profound transformation, driven by the dual imperatives of increasing energy demand and heightened environmental consciousness. Renewable energy sources, particularly solar photovoltaics (PV), are at the forefront of this transition, with the installed capacity of solar panels experiencing exponential growth globally. Photovoltaic power stations can be broadly categorized into large-scale ground-mounted plants, complex terrain installations, and distributed rooftop systems. In recent years, the proliferation of distributed rooftop solar panels, especially in rural and suburban areas, has introduced significant challenges for asset management and maintenance. These solar panels are often numerous, spatially dispersed, and installed in diverse environments, making traditional manual inspection methods impractical. Classical approaches to solar panels identification and defect detection, which rely heavily on manual field surveys and conventional image processing techniques, are labor-intensive, inefficient, and highly susceptible to environmental factors such as varying illumination, shadow occlusion, and complex background clutter. This inherent lack of robustness and scalability severely hinders the ability to meet the demands of large-scale photovoltaic industry informatization, particularly for rapid resource surveys and intelligent operation and maintenance (O&M).
The rapid advancement of deep learning, especially in the domain of semantic segmentation, offers a transformative solution for automated solar panels analysis. By leveraging high-resolution remote sensing imagery, deep learning models can automatically interpret and extract the spatial distribution and health status of solar panels with unprecedented accuracy and efficiency. Among the various semantic segmentation architectures, the DeepLabV3+ model has gained prominence due to its robust encoder-decoder structure and its powerful Atrous Spatial Pyramid Pooling (ASPP) module, which excels at capturing multi-scale contextual information. However, the standard DeepLabV3+ model, often utilizing an Xception backbone, can be computationally intensive and may not be optimally tailored for the specific visual characteristics of solar panels, such as their regular geometric shapes, repetitive textures, and subtle defect features.
This paper presents a novel and comprehensive framework based on an improved DeepLabV3+ model for the multi-scale recognition and defect detection of solar panels. Our research integrates satellite-based macroscopic surveys for resource inventory with unmanned aerial vehicle (UAV)-based microscopic inspections for detailed defect diagnosis. The core technical contributions of this work are twofold. First, we optimize the DeepLabV3+ architecture by replacing the original Xception backbone with a more efficient Residual Network (ResNet-50) and incorporating a Convolutional Block Attention Module (CBAM) within the ASPP module to enhance feature representation and focus on salient regions. Second, we apply this optimized model to two distinct but interrelated tasks: the large-scale extraction of solar panels from multi-source satellite imagery and the fine-grained detection of various physical defects, including cracks, hot spots, black edges, and power loss, from high-resolution UAV visible and thermal infrared imagery. This integrated approach establishes a closed-loop system for photovoltaic asset management, from initial resource survey to ongoing health monitoring.
Our experimental results, validated on a typical region in Guangdong Province, demonstrate that the improved DeepLabV3+ model achieves superior performance compared to baseline models including the original DeepLabV3+, U-Net, and Fast-SCNN. The model exhibits high overall accuracy, precision, recall, and Intersection over Union (IoU) for solar panels extraction under diverse geographical and illumination conditions. Furthermore, the framework proves highly effective for detecting critical solar panels defects, with the notable exception of very fine scratches, which require further model refinement for detection. The findings underscore the potential of our approach to significantly enhance the efficiency and reliability of photovoltaic industry operations, providing a solid data-driven foundation for informed decision-making in site planning, resource assessment, and predictive maintenance.
Methodology: The Improved DeepLabV3+ Model
Our research is fundamentally grounded in the optimization and application of the DeepLabV3+ semantic segmentation model. The standard DeepLabV3+ architecture features an encoder-decoder structure. The encoder, typically built upon the Xception backbone, employs depthwise separable convolutions to reduce computational complexity while extracting high-level semantic features. The Atrous Spatial Pyramid Pooling (ASPP) module, located in the encoder, applies parallel atrous convolutions with different dilation rates to capture multi-scale contextual information, which is crucial for recognizing objects of varying sizes, such as large solar panel arrays and small, individual panels. The decoder module then refines the segmentation by fusing these high-level semantic features with low-level spatial details from the encoder, producing sharp object boundaries through upsampling operations.
While effective, the standard Xception-based DeepLabV3+ model presents several limitations for our specific application. Its large number of parameters (approximately 22 million) and high computational cost (180 GFLOPs for a single 256×256 image) make it less suitable for deployment in resource-constrained scenarios, such as on-board UAV processing. Furthermore, the vanilla attention mechanism may not be optimally focused on the unique visual patterns of solar panels, which often exhibit high intra-class similarity but can be confused with man-made structures like building roofs or greenhouses.
To address these challenges, we propose two key modifications to the DeepLabV3+ architecture. First, we replace the Xception backbone with the Residual Network-50 (ResNet-50). ResNet-50 utilizes a bottleneck residual connection structure, which simplifies network optimization by mitigating the vanishing gradient problem, thereby enabling the training of deeper networks. Although its total parameter count is slightly higher (approximately 25.6 million), its efficient design allows for effective compression through subsequent lightweighting techniques. More importantly, the computational cost for inference on a single image is reduced to 120 GFLOPs, a 33% decrease compared to Xception. This reduction in computational overhead makes the model more suitable for real-time or near real-time applications without a significant sacrifice in representational power. The residual connections in ResNet-50 facilitate the flow of gradients during backpropagation, allowing the network to learn more discriminative features for solar panels.
Second, we introduce the Convolutional Block Attention Module (CBAM) into the ASPP module for enhanced feature fusion. CBAM is a lightweight and efficient attention mechanism that sequentially infers attention maps along the channel and spatial dimensions. The channel attention module focuses on ‘what’ is an informative feature, learning to weight each feature channel based on its global average and max pooled descriptors. The spatial attention module focuses on ‘where’ is the informative region, learning a spatial map that highlights or suppresses specific locations in the feature map. By integrating CBAM after the ASPP module, the model can adaptively recalibrate the multi-scale features. It enhances important features, such as the sharp edges and uniform texture of solar panels, while suppressing irrelevant background noise, such as vegetation or soil. This adaptive focusing capability is particularly beneficial for distinguishing solar panels from visually similar objects and for identifying subtle defects like hot spots or cracks.
The architecture of our improved DeepLabV3+ model can be described as follows. The input image is first processed by the ResNet-50 backbone to generate a set of feature maps at different resolutions. The high-level feature map is then passed through the ASPP module. In our improved version, the features from the different parallel atrous convolution branches are concatenated and then fed into the CBAM module. The CBAM first applies channel attention to the concatenated feature map, followed by spatial attention. The refined feature map from the CBAM is then passed to the decoder. The decoder upsamples the refined high-level features and concatenates them with the corresponding low-level features from the ResNet-50 backbone. Finally, a series of convolutions and another upsampling step produce the final segmentation map with the same resolution as the input image.
The mathematical formulation for our model’s performance evaluation is critical for a quantitative comparison. We employ several standard metrics, including Overall Accuracy (A0), Precision (P), Recall (R), F1-Score (F1), and Intersection over Union (IoU, IIoU). These metrics provide a holistic assessment of the model’s capability in classifying and localizing solar panels. The formulas for these metrics are presented below.
$$
A_0 = \frac{N_{TP} + N_{TN}}{N_{TP} + N_{TN} + N_{FP} + N_{FN}}
$$
$$
P = \frac{N_{TP}}{N_{TP} + N_{FP}}
$$
$$
R = \frac{N_{TP}}{N_{TP} + N_{FN}}
$$
$$
F_1 = 2 \times \frac{P \cdot R}{P + R}
$$
$$
I_{IoU} = \frac{N_{TP}}{N_{TP} + N_{FP} + N_{FN}}
$$
Where NTP is the number of true positives (correctly identified solar panels), NTN is the number of true negatives (correctly identified background), NFP is the number of false positives (background incorrectly identified as solar panels), and NFN is the number of false negatives (solar panels incorrectly identified as background).
Dataset Construction and Experimental Setup
To train and evaluate our improved DeepLabV3+ model, we constructed two distinct datasets: one for satellite-based macroscopic surveys and another for UAV-based microscopic defect detection. The study area for both datasets is a typical region in Guangdong Province, characterized by a mix of urban rooftops, suburban ground-mounted arrays, and mountainous photovoltaic installations.
Remote Sensing Image Dataset (Satellite): The satellite imagery for solar panels resource survey was sourced from a multi-sensor archive, including Ziyuan-2, Gaofen-6, and Gaofen-7 satellites. This multi-source approach ensured a diverse dataset covering various terrain types (plains, hills, urban areas) and illumination conditions (sunny, cloudy, backlit), thereby enhancing the model’s generalization capability. The key specifications of these satellite images are summarized in the following table.
| Data Source | Acquisition Time | Panchromatic Resolution (m) | Multispectral Resolution (m) | Image Characteristics |
|---|---|---|---|---|
| Ziyuan-2 | Spring 2024 | 2.1 | 8.4 | Covers blue, green, red, and near-infrared bands, effectively capturing the spatial distribution of medium-scale solar panel arrays, suitable for macro-level identification of large PV stations. |
| Gaofen-6 | Summer 2024 | 2.0 | 8.0 | Features an additional red-edge band, enhancing the distinction between vegetation and non-vegetation areas, effectively identifying the spectral difference between solar panels and surrounding vegetation. |
| Gaofen-7 | Autumn 2024 | Rear: 0.65, Front: 0.80 | 2.6 to 3.2 | Possesses high-precision geometric positioning capability, clearly capturing the edges, textures, and shadow details of individual solar panels, providing key data for fine-grained segmentation. |
After pre-processing steps including radiometric correction, geometric correction, and data augmentation (e.g., random rotations, flips, and color jittering), a total of 24,000 image patches of size 256×256 pixels were generated. Pixel-level semantic segmentation labels were created using the EISeg annotation tool. The target classes included positive samples: ‘single panel component’ and ‘panel array’; and negative samples: ‘vegetation’, ‘building’, ‘bare land’, ‘water’, and ‘road’. Two professionals independently annotated the data, and only samples with a cross-validation accuracy of ≥95% were included in the final dataset. The dataset was split into training and test sets at an 8:2 ratio. The test set was carefully curated to include challenging scenarios like shadow occlusion, dense arrangements, and small-scale distributed solar panels across different satellite sources, ensuring a robust evaluation of model generalization.
UAV Visible and Thermal Infrared Dataset: For the detailed defect detection task, we conducted aerial surveys using a DJI M300 UAV equipped with a Zenmuse H20T camera, which captures both high-resolution visible light and thermal infrared imagery. Flight missions were planned over complex rooftop and mountainous array areas, employing a terrain-following flight mode to maintain a consistent altitude and image overlap. The flight altitude was kept below 50 meters to capture micro-level features, such as micro-cracks and dust accumulation, which are invisible in satellite imagery. The collected high-resolution images underwent pre-processing steps including denoising, dehazing, and color correction to enhance detail. The images were then cropped into 256×256 pixel sub-images, preserving the complete structure of solar panels.
For the annotation of this dataset, positive samples were defined as ‘intact and undamaged solar panels’, while negative samples included various defect types: ‘hot spots’, ‘cracks’, ‘black edges’, ‘scratches’, and ‘power loss’. A total of 18,000 image patches were generated and split into training and test sets in an 8:2 ratio. The test set was specifically designed to include extreme cases such as backlit low-contrast images, micro-cracks, and false positive examples from low-emissivity roofs, providing a rigorous evaluation of the model’s generalization for defect diagnosis.
The implementation of our improved DeepLabV3+ model was carried out using a deep learning framework. The model was trained from scratch on the satellite dataset for the primary extraction task. We used a batch size of 16, the Adam optimizer with an initial learning rate of 0.001, and a poly learning rate decay policy. For the defect detection task, we employed a transfer learning strategy. The pre-trained weights from the satellite solar panels extraction task were used to initialize the network, and then the model was fine-tuned on the UAV dataset. This approach leverages the common features learned from satellite imagery and significantly accelerates convergence and improves performance on the defect detection task.
Results and Analysis: Solar Panels Resource Survey
We conducted a comprehensive experiment to evaluate the performance of our proposed improved DeepLabV3+ model for the primary task of extracting solar panels from multi-source satellite imagery. The model was compared against three state-of-the-art baseline models: the original DeepLabV3+ (with Xception backbone), U-Net, and Fast-SCNN. The training and testing were performed on the satellite dataset.
The quantitative results, presented in the table below, clearly demonstrate the superiority of our improved DeepLabV3+ model across all evaluation metrics. It achieves the highest overall accuracy of 96.91%, a precision of 94.88%, a recall of 93.21%, an F1-score of 93.85%, and an IoU of 94.47%. The high recall indicates that the model is excellent at finding all true solar panels, minimizing false negatives, which is crucial for a reliable resource survey. The high precision shows that when it predicts a pixel as solar panel, it is very likely correct, minimizing false positives from background objects.
| Deep Learning Model | Overall Accuracy (%) | Precision (%) | Recall (%) | F1-Score (%) | IoU (%) |
|---|---|---|---|---|---|
| Improved DeepLabV3+ (Ours) | 96.91 | 94.88 | 93.21 | 93.85 | 94.47 |
| DeepLabV3+ (Original) | 94.26 | 91.33 | 90.85 | 90.11 | 90.37 |
| U-Net | 91.12 | 89.53 | 89.04 | 88.75 | 88.82 |
| Fast-SCNN | 85.42 | 82.33 | 81.68 | 81.15 | 80.76 |
Compared to the original DeepLabV3+, our improved model shows an improvement of approximately 2.65% in overall accuracy and 4.10% in IoU. This significant enhancement can be attributed to the combined effect of the ResNet-50 backbone and the CBAM module. The residual learning in ResNet-50 allows for more effective training of deep features, while the CBAM module refines the multi-scale contextual information from the ASPP, helping the model focus on the distinctive geometric and textural patterns of solar panels. In contrast, the U-Net model, while performing adequately, falls short of both the original and improved DeepLabV3+ models due to its simpler architecture and lack of explicit multi-scale context aggregation (ASPP). The Fast-SCNN model, designed for real-time inference with a lightweight structure, exhibits the lowest performance, highlighting a trade-off between speed and accuracy. Its limited capacity to capture fine details and complex background interactions results in lower precision and recall for solar panels extraction.
To provide a qualitative understanding of these results, we analyzed the segmentation outputs across different scenarios. In the plains scenario, characterized by large, continuous solar panel arrays, both the improved and original DeepLabV3+ models produced accurate and complete extractions. However, the U-Net model exhibited occasional gaps and failures to distinguish between small grass patches within the array. The Fast-SCNN model showed significant fragmentation and missed solar panels, particularly at the edges of arrays. In the mountainous terrain scenario, where solar panels are often installed on slopes and interspersed with shadows and varying background, the improved DeepLabV3+ model robustly extracted the panels, correctly rejecting roads and bare soil. The other models, particularly U-Net and Fast-SCNN, were more prone to false positives, mistakenly classifying dirt paths as solar panels. In the scenario with scattered, distributed solar panels in a complex mountainous area, all models struggled to some extent, but our improved DeepLabV3+ demonstrated superior integrity and accuracy, successfully identifying more individual panels while making fewer false detections on rocks or vegetation. These observations qualitatively confirm the quantitative findings, demonstrating that the improved DeepLabV3+ model offers superior scene adaptability and reliability, making it an ideal tool for large-scale photovoltaic resource surveys.
Results and Analysis: Solar Panels Defect Detection
Following the successful application of our model for resource surveys, we extended its use to the critical task of solar panels defect detection. Using the UAV-acquired high-resolution visible and thermal infrared imagery dataset, we fine-tuned and tested the improved DeepLabV3+ model to identify and classify five distinct defect types: hot spots, power loss, cracks, black edges, and scratches. The performance for each defect type is summarized in the following table and the accompanying table of visual detection results.
| Defect Type | Overall Accuracy (%) | Precision (%) | Recall (%) | F1-Score (%) | IoU (%) |
|---|---|---|---|---|---|
| Hot Spots | 95.77 | 93.68 | 93.33 | 92.89 | 92.52 |
| Power Loss | 97.92 | 94.56 | 95.28 | 94.95 | 93.74 |
| Black Edges | 90.33 | 88.07 | 89.35 | 89.04 | 88.74 |
| Cracks | 88.47 | 86.35 | 85.62 | 86.22 | 85.56 |
| Scratches | 84.32 | 81.45 | 81.97 | 81.17 | 80.58 |
| All Defects Combined | 91.36 | 88.82 | 89.11 | 88.85 | 88.23 |
The detection of ‘hot spots’ and ‘power loss’ yielded the highest performance, with IoU scores of 92.52% and 93.74%, respectively. This exceptional accuracy is primarily attributed to the synergistic effect of combining visible and thermal infrared data. Hot spots, where a panel cell is damaged and acts as a resistive load, generate a localized temperature increase, creating a clear ‘hot’ signature in the thermal image. Simultaneously, these areas often exhibit a subtle visual change, such as discoloration, in the visible spectrum. Power loss, conversely, manifests as a uniformly cooler region in the thermal image and a darker appearance in the visible spectrum. The CBAM module in our model is particularly effective here; it is hypothesized that the channel attention mechanism prioritizes the temperature gradient features from the thermal infrared channel, while the spatial attention module precisely localizes the corresponding region in the visible channel. This dual-attention mechanism allows the model to leverage the distinct and complementary signatures of these defects, leading to highly reliable detection and effective rejection of false positives.
The detection of ‘cracks’ and ‘black edges’ performed moderately, achieving IoU scores of 85.56% and 88.74%, respectively. Cracks are challenging due to their diverse morphology and scale. Large cracks or shattered panels were successfully identified by the model, which can capture the resultant texture discontinuity and intensity jumps. The shallow layers of the ResNet-50 backbone are proficient at extracting these local, fine-grained details, while the deeper layers provide contextual shape constraints. However, micro-cracks, which are barely visible and may only be a few millimeters wide, often fall below the spatial resolution limit of the model or are confused with dust or dirt patterns. Black edges, a phenomenon where the panel’s edge sealant degrades, are also difficult to detect if they are thin or if the panel is already dusty, as the visual signature of dust and a black edge can be similar, leading to classification ambiguity.
The detection of ‘scratches’ presented the greatest challenge, with the lowest IoU of 80.58%. This is because scratches are typically very narrow, linear features with a scale that is close to the pixel resolution of the UAV imagery, even at low flight altitudes. Their subtle intensity variation against the uniform dark background of the solar panel makes them extremely difficult to distinguish from noise or surface smudges. The current model architecture, while powerful, appears to have a resolution limit for reliably detecting these highly localized, thin-line defects. This finding suggests a need for future work, potentially involving higher resolution sensors, specialized data augmentation techniques (e.g., simulating scratches), or a dedicated, fine-grained classification branch integrated into the model.
The following table provides visual examples of the improved DeepLabV3+ model’s detection results for various defect types. It is important to note that these are representative examples from our test set, showcasing both the strengths and limitations of our model. The model excels at capturing well-defined temperature anomalies for hot spots and large-area power loss. For structural defects like cracks and black edges, it correctly identifies the general area but may miss the finest details. For scratches, the model’s performance is more mixed, successfully identifying some instances while completely missing others that are barely perceptible.
| Defect Type | Detection Result by Improved DeepLabV3+ Model |
|---|---|
| Hot Spots | The model accurately highlights regions with elevated temperature, producing a clear and spatially precise detection mask that aligns perfectly with the thermal anomaly. |
| Power Loss | The model successfully detects large, contiguous areas that are non-functional, showing a strong boundary delineation compared to the surrounding active panels. |
| Cracks | Major structural fractures are identified, but the detection mask can appear slightly fragmented or incomplete at the very micro-level of the crack. The overall region of damage is well captured. |
| Black Edges | Significant edge degradation is detected. The model can distinguish the blackened edge from the normal panel surface, though performance can degrade on heavily soiled panels. |
| Scratches | The model shows the weakest performance here. Only very prominent scratches are detected, while numerous fine-line scratches visible in the original image remain undetected in the output mask. |
Despite the challenges with micro-scratches, the overall combined average performance for all defect types is high, with an overall accuracy of 91.36% and an F1-score of 88.85%. This demonstrates that our improved DeepLabV3+ model is a powerful and practical tool for automated, intelligent inspection of solar panels. The ability to accurately identify and classify the most critical defects, such as hot spots and power loss which can lead to system failure and fire hazards, provides immense value for O&M teams. By automating the detection process, the framework drastically reduces the time and cost associated with manual inspections and provides a systematic, data-driven approach for prioritizing maintenance tasks and ensuring the long-term health and profitability of photovoltaic installations.
Conclusion and Future Work
In this research, we have successfully developed and validated a robust deep learning framework for the multi-scale recognition and defect detection of solar panels. By strategically improving the DeepLabV3+ model—replacing the Xception backbone with a more efficient ResNet-50 and integrating a CBAM attention module into the ASPP feature fusion stage—we have created a model that is both more accurate and computationally more efficient for this specific application.
The key conclusions from our study are threefold. First, for large-scale solar panels resource surveying using satellite imagery, our improved DeepLabV3+ model consistently outperforms baseline models (original DeepLabV3+, U-Net, Fast-SCNN) in terms of overall accuracy, precision, recall, and IoU. It demonstrates superior capability in extracting solar panels across diverse terrain and illumination conditions, accurately delineating panel boundaries while effectively rejecting complex background clutter. This confirms its value as a reliable and efficient tool for creating comprehensive photovoltaic resource inventories.
Second, for the critical task of defect detection using high-resolution UAV imagery, the model proves to be highly effective for diagnosing the most impactful defects, specifically hot spots and power loss, achieving IoU scores above 92%. The synergetic use of visible and thermal infrared data, enhanced by the attention mechanism, is pivotal for this success. The model also performs well on other defects like cracks and black edges, offering a practical solution for automated inspection that can significantly enhance the efficiency and reliability of photovoltaic O&M practices.
Third, the model exhibits a clear limitation in detecting very fine, linear defects, particularly micro-scratches. This is a known challenge for semantic segmentation models, where the resolution of the feature maps in the decoder may not be sufficient to reconstruct such high-frequency, small-scale details. This finding highlights a critical area for future research.
Looking forward, several avenues for improvement and expansion are apparent. A primary direction is to enhance the model’s sensitivity to micro-defects. This could involve exploring higher-resolution input images, implementing advanced super-resolution preprocessing, or designing a dedicated task-specific decoder branch that focuses on fine-grained feature extraction. Another promising direction is to extend the framework beyond simple detection to include severity assessment. By integrating regression heads or a multi-class classification of defect severity (e.g., minor, medium, severe crack), the system could provide more actionable intelligence for maintenance prioritization. Furthermore, the development of a fully integrated, near-real-time system combining onboard UAV processing with edge computing would represent a significant step forward, enabling rapid in-field defect identification and immediate reporting. Our work provides a strong technical foundation for these future innovations, paving the way for a new generation of intelligent, automated, and highly efficient photovoltaic asset management systems. The ultimate goal is to create a sustainable, data-driven ecosystem for the global solar energy industry, maximizing energy yield and minimizing operational risk.