We present a comprehensive study on solar panel recognition and defect detection using an improved DeepLabV3+ model. The research integrates satellite-based macro surveys and unmanned aerial vehicle (UAV)-based micro inspections to create a multi-scale detection framework for solar panels in a typical region of southern China. By replacing the original backbone network with ResNet-50 and incorporating the Convolutional Block Attention Module (CBAM) into the Atrous Spatial Pyramid Pooling (ASPP) module, we significantly enhance the model’s ability to extract spatial features and suppress irrelevant information. Experimental results on multi-source remote sensing images demonstrate that the improved model achieves superior performance in solar panel extraction compared to original DeepLabV3+, U-Net, and Fast-SCNN models, with an overall accuracy of 96.91% and an IoU of 94.47%. Furthermore, using UAV-collected visible and thermal infrared images, the model is applied to detect multiple defect types of solar panels, including cracks, hot spots, black edges, scratches, and power-off cells. The detection accuracy for hot spots and power-off cells exceeds 95%, while scratch detection remains challenging due to small scale and similarity to dirt. Our work provides an efficient and reliable solution for solar panel resource surveys and intelligent maintenance, supporting the sustainable development of photovoltaic energy.
1. Introduction
The global shift towards renewable energy has accelerated the deployment of photovoltaic (PV) systems, particularly solar panels installed on rooftops, ground-mounted arrays, and complex terrains. As the number of solar panels increases rapidly, especially in distributed PV systems located in rural areas, the need for efficient and accurate methods to inventory solar panel locations, measure areas, and detect defects becomes critical. Traditional approaches rely on manual inspection and conventional image processing, which are labor-intensive, sensitive to illumination variations and shadow occlusion, and poorly adaptable to complex backgrounds. These limitations hinder large-scale PV industry information acquisition and intelligent maintenance.
Recent advances in deep learning, especially semantic segmentation models, offer new opportunities for automatic solar panel recognition from remote sensing imagery. DeepLabV3+, a widely used encoder-decoder architecture with atrous convolutions and multi-scale feature fusion, has demonstrated strong performance in object extraction tasks. However, standard DeepLabV3+ using Xception as backbone suffers from high computational costs and large parameter counts, making it less suitable for resource-constrained deployment. In our study, we improve DeepLabV3+ by adopting ResNet-50 as the backbone and integrating CBAM to refine feature attention, thereby achieving better accuracy and efficiency for solar panel identification.
Moreover, beyond spatial mapping, the health status of solar panels is crucial for operational efficiency. Defects such as cracks, hot spots, black edges, and power-off cells reduce energy yield and may lead to safety hazards. We extend our framework to defect detection using high-resolution UAV imagery, combining visible and thermal infrared data to capture subtle anomalies. The improved DeepLabV3+ model is trained on a dataset of solar panel defects and evaluated on various test scenarios. Our contributions include: (1) a lightweight yet accurate model for solar panel segmentation from satellite images; (2) a systematic defect classification approach for UAV-based inspection; and (3) a multi-scale monitoring system for PV resource management.
2. Methodology
2.1 Improved DeepLabV3+ Architecture
DeepLabV3+ is a state-of-the-art semantic segmentation model consisting of an encoder-decoder structure. The encoder employs a backbone network to extract hierarchical features, followed by the ASPP module that applies atrous convolutions with different dilation rates to capture multi-scale contextual information. The decoder then upsamples the high-level features and fuses them with low-level details to produce precise segmentation maps. Traditionally, Xception serves as the backbone due to its depthwise separable convolutions, but it still involves around 22 million parameters and high floating-point operations (180 GFLOPS for a 256×256 image).
We replace Xception with ResNet-50, which uses bottleneck residual connections. Although ResNet-50 has about 25.6 million parameters (slightly more than Xception), its deeper bottleneck structure allows effective training and feature extraction. By incorporating subsequent lightweight designs, we reduce the inference complexity to 120 GFLOPS per image—a 33% reduction compared to Xception—making it more suitable for scenarios with limited computational resources. Additionally, we introduce the Convolutional Block Attention Module (CBAM) after the ASPP module. CBAM sequentially applies channel attention and spatial attention to adaptively recalibrate feature maps, emphasizing important regions and suppressing irrelevant responses. This integration enhances the model’s focus on solar panel features while ignoring background clutter.
The improved DeepLabV3+ network is defined as follows. Let \(X\) be the input image. The encoder with ResNet-50 backbone produces multi-level feature maps \(F_l\) (low-level) and \(F_h\) (high-level). The ASPP module with CBAM takes \(F_h\) and generates refined multi-scale features \(F_{aspp}\). The decoder upsamples \(F_{aspp}\) and concatenates with \(F_l\) to obtain the final segmentation map \(Y\). The loss function is categorical cross-entropy with L2 regularization.
2.2 Evaluation Metrics
To comprehensively assess model performance, we compute several metrics: Overall Accuracy (\(A_0\)), Precision (\(P\)), Recall (\(R\)), F1 Score (\(F_1\)), and Intersection over Union (\(I_{IoU}\)). The formulas are:
\[
A_0 = \frac{TP + TN}{TP + TN + FP + FN}
\]
\[
P = \frac{TP}{TP + FP}
\]
\[
R = \frac{TP}{TP + FN}
\]
\[
F_1 = \frac{2 \cdot P \cdot R}{P + R}
\]
\[
I_{IoU} = \frac{TP}{TP + FP + FN}
\]
where \(TP\) is true positive, \(TN\) true negative, \(FP\) false positive, and \(FN\) false negative. \(A_0\) reflects overall classification ability, while \(I_{IoU}\) measures localization accuracy of solar panel regions.
3. Experiments
3.1 Dataset Construction
Satellite Remote Sensing Dataset: We collected multi-source satellite images (high-resolution optical data) covering the study area with diverse terrain (plains, hills, urban areas) and illumination conditions (clear, cloudy, backlight). Images were radiometrically corrected, geometrically corrected, and augmented. Pixel-level semantic segmentation annotation was performed using EISeg tool, with positive samples (solar panels) including single modules and arrays, and negative samples including vegetation, buildings, bare land, water, and roads. Two independent annotators cross-validated samples with agreement ≥95%. A total of 24,000 patches (256×256 pixels) were generated, split into training (80%) and testing (20%). The test set retained challenging cases such as shadow occlusion, dense arrays, and small-scale distributed solar panels to evaluate generalization.
UAV Image Dataset: For defect detection, we used a commercial UAV equipped with a dual camera capturing visible and thermal infrared images. Flight altitude was kept below 50 m to capture fine details like micro-cracks and dust accumulation. Images were preprocessed (denoising, dehazing, color correction) and cropped into 256×256 patches, keeping only those with at least 5% solar panel area. Thermal data were augmented with temperature noise simulation and local contrast stretching. The final dataset comprised 18,000 patches, split 80% training and 20% testing. Defect categories included intact panels (negative) and defective panels with cracks, hot spots, black edges, scratches, and power-off cells (positive). Test samples included low-contrast backlight images, micro-cracks, and false-positive rooftop panels to rigorously evaluate model robustness.
3.2 Training Settings
All models were implemented using PyTorch and trained on a single NVIDIA RTX 3090 GPU. The improved DeepLabV3+ was trained with a batch size of 8, initial learning rate 0.001, using Adam optimizer with cosine annealing schedule. Data augmentation included random rotation, flipping, and color jitter. Early stopping with patience of 10 epochs based on validation IoU was applied. For comparison, we trained original DeepLabV3+ (Xception backbone), U-Net, and Fast-SCNN under identical conditions.
4. Results and Discussion
4.1 Solar Panel Extraction from Satellite Imagery
We evaluated the improved DeepLabV3+ against baseline models on the satellite test set. Table 1 summarizes the comparison results.
| Model | Overall Accuracy (%) | Precision (%) | Recall (%) | F1 Score (%) | IoU (%) |
|---|---|---|---|---|---|
| Improved DeepLabV3+ | 96.91 | 94.88 | 93.21 | 93.85 | 94.47 |
| Original DeepLabV3+ | 94.26 | 91.33 | 90.85 | 90.11 | 90.37 |
| U-Net | 91.12 | 89.53 | 89.04 | 88.75 | 88.82 |
| Fast-SCNN | 85.42 | 82.33 | 81.68 | 81.15 | 80.76 |
As shown, the improved DeepLabV3+ achieves the highest values across all metrics, surpassing the original DeepLabV3+ by approximately 2.6% in accuracy and 4.1% in IoU. U-Net performs moderately, while Fast-SCNN lags significantly due to its limited feature extraction capability in complex scenes. In qualitative analysis on three typical scenarios—plain ground-mounted arrays, hillside arrays, and scattered rooftop panels—the improved model consistently produced segmentation maps closest to ground truth. For plain areas, it accurately delineated panel boundaries without merging adjacent grassland. In hilly terrain, it distinguished road and bare land from solar panels, whereas other models frequently misclassified these backgrounds. For scattered panels, the improved model maintained high integrity and low false positive rates, demonstrating robustness to complex topography and variable lighting.
4.2 Solar Panel Defect Detection with UAV Imagery
We applied the improved DeepLabV3+ to detect six defect types from UAV images. Table 2 reports the per-class and overall detection performance.
| Defect Type | Overall Accuracy (%) | Precision (%) | Recall (%) | F1 Score (%) | IoU (%) |
|---|---|---|---|---|---|
| Cracks | 88.47 | 86.35 | 85.62 | 86.22 | 85.56 |
| Hot Spots | 95.77 | 93.68 | 93.33 | 92.89 | 92.52 |
| Black Edges | 90.33 | 88.07 | 89.35 | 89.04 | 88.74 |
| Scratches | 84.32 | 81.45 | 81.97 | 81.17 | 80.58 |
| Power-Off Cells | 97.92 | 94.56 | 95.28 | 94.95 | 93.74 |
| All Defects | 91.36 | 88.82 | 89.11 | 88.85 | 88.23 |
Hot spots and power-off cells achieve the highest accuracy (above 95% and 97% respectively), benefiting from the synergistic use of thermal anomalies and visible texture changes. The CBAM module effectively activates thermal gradient features in the channel attention branch and spatially focuses on corresponding visible regions. Crack detection accuracy is slightly lower (around 88%) because cracks exhibit diverse morphologies (fissures, fragmentation, missing parts). The improved model successfully captures larger breaks but struggles with micro-cracks. Black edge defects are susceptible to dust interference, leading to occasional false positives. Scratches, being extremely thin and similar to stains, present the greatest challenge, with an IoU of only 80.58%. Overall, the system demonstrates high reliability for routine inspection, with the caveat that tiny scratches may require higher-resolution imagery or specialized attention mechanisms.
5. Conclusion
In this work, we developed an improved DeepLabV3+ model by replacing the backbone with ResNet-50 and integrating CBAM attention for solar panel recognition and defect detection. The model was validated on multi-source satellite imagery for large-scale solar panel mapping and on UAV imagery for fine-grained defect diagnosis. Key findings include:
- The improved model outperforms original DeepLabV3+, U-Net, and Fast-SCNN in semantic segmentation of solar panels, achieving 96.91% overall accuracy and 94.47% IoU. It effectively handles diverse terrain and illumination conditions, providing reliable spatial distribution for PV resource surveys.
- For defect detection, the model achieves high accuracy for hot spots (95.77%) and power-off cells (97.92%), moderate accuracy for cracks (88.47%) and black edges (90.33%), and lower accuracy for scratches (84.32%). The overall detection accuracy is 91.36%, demonstrating practical utility for PV maintenance.
- The combination of remote sensing and UAV data with an optimized deep learning framework enables efficient, automated monitoring of solar panel health, reducing reliance on manual inspection and supporting timely interventions to prevent energy losses.
Future work will focus on improving scratch detection by incorporating fine-grained feature extraction modules or using higher-resolution sensors. Additionally, we plan to deploy the model on edge devices for real-time UAV inspection, further enhancing the scalability of solar farm management.