In the context of global energy transformation, solar energy has emerged as one of the most promising renewable energy sources. The widespread deployment of photovoltaic systems has led to an increasing demand for efficient and accurate fault detection methods for solar panels. Traditional manual inspection approaches are time-consuming and unreliable, while deep learning-based methods have demonstrated remarkable potential in automating this process. However, a critical challenge remains: accurately segmenting solar panels from infrared images, particularly when dealing with complex backgrounds, varying lighting conditions, and subtle boundary details. In this research, I propose an improved semantic segmentation model based on DeepLabV3+, specifically designed to address the limitations of existing methods when applied to solar panel segmentation tasks. The proposed model incorporates several key innovations, including a lightweight backbone network, edge detection enhancement, attention-guided feature recalibration, and multi-channel feature propagation, all aimed at achieving superior segmentation accuracy and robustness for solar panels in real-world scenarios.
Semantic segmentation, at its core, aims to assign a class label to every pixel in an image, thereby enabling precise delineation of objects of interest. When applied to solar panels, this technique allows for the extraction of panel regions from infrared imagery, effectively removing background interference and providing clean inputs for subsequent fault detection algorithms. The primary motivation behind this work stems from the observation that conventional semantic segmentation models often struggle with the unique characteristics of solar panel infrared images. Issues such as wavy boundaries,粘连 between adjacent panels, and misclassification of background regions as solar panels are common. These problems are particularly pronounced in distributed photovoltaic installations, where solar panels may be installed on metal roofs, ground-mounted structures, or other complex settings that produce similar infrared signatures. To address these challenges, I have developed a comprehensive改进方案 that enhances the feature extraction capability, boundary sensitivity, and contextual understanding of the DeepLabV3+ framework.
The foundation of my proposed approach lies in the DeepLabV3+ architecture, which has established itself as a state-of-the-art semantic segmentation model due to its encoder-decoder structure and atrous spatial pyramid pooling (ASPP) module. The encoder component extracts multi-scale contextual features through a series of convolutional layers, while the decoder progressively upsamples these features to recover spatial details. The ASPP module, with its parallel atrous convolutions at different dilation rates, captures information at multiple scales, enabling the model to handle objects of varying sizes. However, when applied to solar panel segmentation, the original DeepLabV3+ exhibits several shortcomings, particularly in boundary precision and the handling of fine details. To overcome these limitations, I propose a series of targeted modifications that significantly improve the model’s performance on solar panel infrared images.
The first major modification involves replacing the original Xception backbone with MobileNetV2. While Xception offers powerful feature extraction capabilities, it comes with substantial computational overhead, making it less suitable for real-time or resource-constrained applications. MobileNetV2, a lightweight convolutional neural network architecture, employs depthwise separable convolutions to dramatically reduce the number of parameters and computational cost while maintaining competitive accuracy. The core building block of MobileNetV2 is the inverted residual structure with linear bottlenecks, which consists of an expansion layer, a depthwise convolution layer, and a projection layer. This design allows the network to learn rich feature representations efficiently. In my implementation, I utilize the output from the fourth stage of MobileNetV2, which corresponds to bottleneck layer 3 with an output depth of 32 channels, as the source for generating shallow feature semantic information. This choice strikes a balance between computational efficiency and the preservation of spatial details crucial for accurate boundary delineation of solar panels.
The mathematical formulation of depthwise separable convolution can be expressed as follows. For a standard convolution with kernel size \(K \times K\), input channels \(C_{in}\), and output channels \(C_{out}\), the computational cost is:
$$ \text{Cost}_{\text{standard}} = K \times K \times C_{in} \times C_{out} \times H \times W $$
where \(H\) and \(W\) are the spatial dimensions of the output feature map. In contrast, depthwise separable convolution decomposes this operation into two steps: depthwise convolution and pointwise convolution. The computational cost becomes:
$$ \text{Cost}_{\text{depthwise}} = K \times K \times C_{in} \times H \times W + C_{in} \times C_{out} \times H \times W $$
The ratio of computational costs between depthwise separable convolution and standard convolution is:
$$ \frac{\text{Cost}_{\text{depthwise}}}{\text{Cost}_{\text{standard}}} = \frac{1}{C_{out}} + \frac{1}{K^2} $$
This significant reduction in computational complexity makes MobileNetV2 an ideal backbone for practical applications involving solar panel segmentation, especially when deployed on edge devices or drones used for inspecting photovoltaic installations.
To further enhance the boundary detection capability of the model, I introduce a Canny edge detection module that processes the shallow features extracted from the MobileNetV2 backbone. The Canny algorithm is a multi-stage edge detection operator that identifies edges by looking for local maxima of the gradient magnitude. The process begins with Gaussian smoothing to reduce noise, followed by gradient computation using Sobel operators. The gradient magnitude \(G\) and direction \(\theta\) at each pixel are computed as:
$$ G = \sqrt{G_x^2 + G_y^2} $$
$$ \theta = \arctan\left(\frac{G_y}{G_x}\right) $$
where \(G_x\) and \(G_y\) are the gradients in the horizontal and vertical directions, respectively. After non-maximum suppression to thin the edges, double thresholding is applied to identify strong, weak, and non-edge pixels. A pixel with gradient magnitude above the high threshold is classified as a strong edge pixel, while a pixel with magnitude between the low and high thresholds is classified as a weak edge pixel and retained only if it is connected to a strong edge pixel. By integrating the edge information from Canny detection into the shallow feature stream, the model gains improved sensitivity to the boundaries of solar panels, enabling more precise segmentation even in challenging cases where panels are closely spaced or partially occluded.
Another critical innovation in my proposed model is the SE-ASPP module, which combines the strengths of squeeze-and-excitation (SE) attention mechanisms with the multi-scale feature extraction capability of ASPP. The original ASPP module in DeepLabV3+ uses parallel atrous convolutions with dilation rates of 6, 12, and 18, along with image-level features, to capture contextual information at different scales. However, this configuration may not be optimal for solar panel segmentation, where fine-grained details at object boundaries are crucial. In my SE-ASPP module, I adopt a modified set of dilation rates: 3, 6, and 9. The dilation rate of 3 is well-suited for capturing small-scale details and texture information, such as the细微 variations at the edges of solar panels. The dilation rate of 6 expands the receptive field to encompass larger edge contours, while the dilation rate of 9 further extends the context to capture broader background information that aids in accurate localization. After each parallel atrous convolution branch, I insert an SE module that recalibrates the feature channels adaptively.
The SE mechanism operates in three stages: squeeze, excitation, and rescaling. In the squeeze stage, global average pooling is applied to each feature channel to generate a channel-wise descriptor. For a feature map \(U \in \mathbb{R}^{H \times W \times C}\), the squeeze operation produces a vector \(z \in \mathbb{R}^{C}\) where each element is given by:
$$ z_c = \frac{1}{H \times W} \sum_{i=1}^{H} \sum_{j=1}^{W} u_c(i,j) $$
In the excitation stage, two fully connected layers with a reduction ratio \(r\) are used to model channel-wise dependencies. The first layer reduces the dimensionality to \(C/r\) with a ReLU activation, and the second layer restores the original dimensionality with a sigmoid activation to produce channel weights:
$$ s = \sigma(W_2 \delta(W_1 z)) $$
where \(W_1 \in \mathbb{R}^{\frac{C}{r} \times C}\) and \(W_2 \in \mathbb{R}^{C \times \frac{C}{r}}\) are the weight matrices of the two fully connected layers, \(\delta\) denotes the ReLU function, and \(\sigma\) represents the sigmoid function. Finally, in the rescaling stage, the learned weights are applied to the original feature maps through element-wise multiplication:
$$ \tilde{x}_c = s_c \cdot u_c $$
By incorporating SE modules into each branch of the ASPP, my model learns to emphasize informative feature channels while suppressing less relevant ones, leading to more discriminative feature representations for solar panel segmentation. This channel-wise recalibration is particularly beneficial for handling the variability in infrared signatures of solar panels under different environmental conditions.
To further strengthen the contribution of shallow features in the segmentation process, I modify the decoder architecture by replacing the single 1×1 convolution pathway from the encoder to the decoder with a parallel dual-channel configuration. In the original DeepLabV3+, shallow features from the backbone are passed through a single 1×1 convolution before being fused with the upsampled deep features in the decoder. While this single pathway provides some spatial detail, it may not be sufficient to fully leverage the rich edge and texture information contained in the shallow layers. My proposed dual-channel approach creates two parallel 1×1 convolution branches that independently transform the shallow feature maps into two distinct feature representations. These two representations are then concatenated before fusion with the deep features, providing the decoder with richer and more diverse spatial information. This augmentation enhances the model’s ability to recover fine details and improves the overall segmentation quality for solar panels, particularly at boundaries and in regions with complex texture patterns.
The overall architecture of my improved DeepLabV3+ model can be summarized as follows. The input infrared image of solar panels is first processed by the MobileNetV2 backbone, which extracts hierarchical features at multiple scales. The output from the fourth stage of MobileNetV2, with an output depth of 32 channels, is fed into the Canny edge detection module to generate edge-enhanced shallow feature semantic information. This edge-enhanced information follows two parallel paths. In the first path, it is passed through two parallel 1×1 convolution channels directly to the decoder, bypassing the deeper processing stages. In the second path, it is combined with deep features extracted from later stages of the backbone and fed into the SE-ASPP module, where multi-scale contextual features are extracted and channel-wise recalibrated. The output from SE-ASPP is then subjected to a 1×1 convolution and 4× upsampling before entering the decoder. In the decoder, the two streams of information are fused through concatenation, followed by a 3×3 convolution and another 4× upsampling to produce the final segmentation map. This multi-pathway design ensures that both fine-grained spatial details and high-level semantic context are effectively utilized for accurate segmentation of solar panels.
To evaluate the effectiveness of my proposed model, I conducted comprehensive experiments using a dataset of infrared images of solar panels. The dataset was acquired using a DJI M300 RTK drone equipped with an HT20 infrared thermal imaging camera, capturing images of photovoltaic power stations in mountainous terrain under various lighting conditions and weather scenarios. The original resolution of the images was 640×512 pixels. A total of 1,820 images containing solar panels were selected from 1,865 captured images, and data augmentation through rotation expanded the dataset to 3,640 images, containing 11,273 individual solar panel instances. The dataset was randomly split into training, validation, and testing sets with a ratio of 7:2:1. Each image was manually annotated using the Labelme software to generate JSON files, which were then converted into PNG format mask images for training and evaluation.
The evaluation metrics used in my experiments include precision (P), mean intersection over union (mIoU), recall (R), and F1 score. These metrics are defined as follows:
$$ P = \frac{TP}{TP + FP} \times 100\% $$
$$ \text{mIoU} = \frac{1}{N} \sum_{i=1}^{N} \frac{TP}{TP + FP + FN} \times 100\% $$
$$ R = \frac{TP}{TP + FN} \times 100\% $$
$$ F1 = 2 \times \frac{P \times R}{P + R} \times 100\% $$
where \(TP\) represents the number of pixels correctly predicted as solar panels, \(FP\) is the number of pixels incorrectly predicted as solar panels, \(FN\) is the number of solar panel pixels incorrectly predicted as background, and \(N\) is the number of classes (in this case, \(N=2\) for background and solar panels).
I conducted comparative experiments with several state-of-the-art semantic segmentation models, including PSPNet, HRNet, U-Net, and the original DeepLabV3+. All models were trained and evaluated under identical experimental conditions using the same dataset and hardware environment. The experimental setup consisted of a system running Windows 10 with PyTorch 2.1.2 and Python 3.8.18, powered by a 13th Gen Intel Core i9-13900K CPU and an NVIDIA GeForce RTX 4090 GPU. During training, the input image size was set to 512×512 pixels, the initial learning rate was 0.0001, the Adam optimizer was used, the batch size was 16, and the total number of training epochs was 100.
The results of the comparative experiments are summarized in the following table:
| Model | P (%) | mIoU (%) | R (%) | F1 (%) |
|---|---|---|---|---|
| PSPNet | 96.31 | 96.83 | 97.37 | 96.84 |
| HRNet | 97.01 | 97.29 | 97.62 | 97.31 |
| U-Net | 96.92 | 97.15 | 97.46 | 97.08 |
| Original DeepLabV3+ | 97.26 | 97.63 | 98.04 | 97.83 |
| Proposed Improved Model | 99.50 | 99.21 | 99.61 | 99.55 |
The results clearly demonstrate the superiority of my proposed model over the competing approaches. The original DeepLabV3+ achieved a precision of 97.26%, mIoU of 97.63%, recall of 98.04%, and F1 score of 97.83%, which already outperformed PSPNet, HRNet, and U-Net. My improved DeepLabV3+ model further elevated these metrics to 99.50%, 99.21%, 99.61%, and 99.55%, respectively, representing substantial improvements of 2.24%, 1.58%, 1.57%, and 1.72% over the original DeepLabV3+. These results indicate that my proposed enhancements significantly boost the model’s ability to accurately segment solar panels from infrared images, with particular improvements in boundary precision and overall segmentation consistency.
While the quantitative metrics provide compelling evidence of the model’s performance, I also conducted qualitative analysis by examining the segmentation outputs on representative images from the test set. The visual inspection revealed that PSPNet frequently produced coarse segmentation results with noticeable boundary errors and misclassifications of background regions as solar panels. HRNet and U-Net showed improved performance but still struggled with scenarios involving closely spaced solar panels, where粘连 between adjacent panels was observed. The original DeepLabV3+ demonstrated better overall segmentation quality but occasionally exhibited wavy boundaries, internal holes within solar panel regions, and misclassification of background objects such as metal buildings and vehicles that have similar infrared characteristics to solar panels. In contrast, my proposed model consistently produced clean, precise segmentation results with sharp boundaries, complete panel regions without internal holes, and accurate rejection of background objects. The improvements were particularly evident in challenging cases, such as images containing metal structures, vehicles, or panels with irregular shapes.
To rigorously validate the contribution of each proposed component, I conducted ablation experiments by incrementally adding the different modules to the baseline DeepLabV3+ model. The results of the ablation study are presented in the following table:
| Group | MobileNetV2 | Canny Edge | SE-ASPP | Dual Channel | P (%) | mIoU (%) | R (%) | F1 (%) |
|---|---|---|---|---|---|---|---|---|
| 1 | ✗ | ✗ | ✗ | ✗ | 97.26 | 97.63 | 98.04 | 97.83 |
| 2 | ✓ | ✗ | ✗ | ✗ | 98.10 | 98.95 | 99.07 | 98.94 |
| 3 | ✓ | ✓ | ✗ | ✗ | 98.54 | 99.01 | 99.52 | 99.01 |
| 4 | ✓ | ✓ | ✓ | ✗ | 98.84 | 99.03 | 99.52 | 99.02 |
| 5 | ✓ | ✓ | ✓ | ✓ | 99.50 | 99.21 | 99.61 | 99.55 |
The ablation results provide valuable insights into the individual contributions of each component. Group 1 represents the baseline DeepLabV3+ model. In Group 2, replacing the Xception backbone with MobileNetV2 resulted in improvements of 0.84%, 1.32%, 1.03%, and 1.11% in precision, mIoU, recall, and F1 score, respectively. This substantial improvement demonstrates that MobileNetV2, despite being a lightweight architecture, provides powerful feature extraction capabilities that are well-suited for the task of solar panel segmentation. The reduction in model complexity also contributes to more efficient training and inference, which is advantageous for practical deployment in photovoltaic inspection systems.
In Group 3, the addition of the Canny edge detection module further improved the metrics by 0.44%, 0.06%, 0.45%, and 0.07% compared to Group 2. While the improvement in mIoU was modest, the noticeable gains in precision and recall indicate that edge information from Canny detection enhances the model’s ability to correctly identify boundary pixels of solar panels, reducing false positives and false negatives at the edges. This is particularly important for applications where accurate delineation of solar panel boundaries is critical for subsequent fault detection.
Group 4 incorporated the SE-ASPP module, which brought an additional improvement of 0.30% in precision compared to Group 3, while maintaining similar recall rates. The SE-ASPP module’s ability to recalibrate feature channels adaptively helps the model focus on the most informative features for solar panel segmentation, leading to more accurate positive predictions. The precision gain of 0.30% demonstrates the effectiveness of the attention mechanism in filtering out irrelevant or misleading features that may arise from complex background variations in infrared images of solar panels.
Finally, Group 5 integrated the parallel dual-channel module for shallow features, achieving the best overall performance with precision of 99.50%, mIoU of 99.21%, recall of 99.61%, and F1 score of 99.55%. Compared to Group 4, the addition of the dual-channel module improved precision by 0.66%, mIoU by 0.18%, recall by 0.09%, and F1 score by 0.53%. These results confirm that enriching the shallow feature information through parallel transformation pathways significantly enhances the model’s ability to capture and utilize fine-grained spatial details, leading to superior segmentation quality for solar panels. The dual-channel module effectively increases the diversity and richness of the feature representations available to the decoder, enabling more precise boundary reconstruction and better handling of intricate panel shapes.
The ablation study unequivocally demonstrates that all four proposed modules contribute positively to the overall performance, and their synergistic combination yields the best results. The MobileNetV2 backbone provides a strong foundation with efficient feature extraction, the Canny edge detection enhances boundary sensitivity, the SE-ASPP module improves channel-wise feature discrimination, and the dual-channel shallow feature pathway enriches spatial detail information. Together, these innovations form a comprehensive solution to the challenges of semantic segmentation for solar panels in infrared imagery.
The practical implications of this research are significant for the photovoltaic industry. Accurate semantic segmentation of solar panels from infrared images forms the basis for reliable fault detection, including the identification of hot spots, diode failures, cracks, and other anomalies that can compromise the performance and safety of photovoltaic systems. By effectively removing background interference and precisely delineating panel boundaries, my improved DeepLabV3+ model enables subsequent fault detection algorithms to focus exclusively on the panel regions, thereby reducing false positives and improving detection accuracy. This is particularly valuable in distributed photovoltaic installations, where solar panels are often installed in complex environments with diverse background elements such as metal roofs, concrete structures, vegetation, and vehicles, all of which can produce confusing infrared signatures.
Furthermore, the lightweight nature of the MobileNetV2 backbone makes the proposed model suitable for deployment on resource-constrained platforms, such as the onboard computers of inspection drones. Real-time or near-real-time segmentation of solar panels during flight can enable immediate identification of potential faults, allowing for targeted inspections and timely maintenance interventions. This capability is crucial for large-scale solar farms where manual inspection is impractical and costly. The computational efficiency of my model, combined with its high segmentation accuracy, strikes an optimal balance between performance and practicality for real-world applications in the photovoltaic industry.
The robustness of my proposed model across different imaging conditions is another notable advantage. The infrared images in the dataset were captured under varying lighting conditions, weather scenarios, and camera angles, reflecting the diverse operational environments encountered in actual photovoltaic installations. The strong segmentation performance achieved across these varied conditions demonstrates the model’s generalization capability and its potential for reliable deployment in real-world settings. The integration of Canny edge detection and SE-ASPP contributes to this robustness by enhancing the model’s sensitivity to edge features and its ability to adapt to different feature distributions.
Looking forward, there are several promising directions for future research. One avenue is to further optimize the model architecture to achieve even faster inference speeds while maintaining or improving accuracy, which would be beneficial for real-time applications. Another direction is to explore the integration of temporal information from video sequences to improve segmentation stability and consistency across consecutive frames. Additionally, extending the model to handle multi-class segmentation tasks, such as distinguishing between different types of panel defects or differentiating panels from other solar equipment, could provide more comprehensive information for fault diagnosis. Finally, investigating the transferability of the model to other types of photovoltaic panels and imaging modalities would broaden its applicability and impact in the renewable energy sector.
In conclusion, I have presented an improved DeepLabV3+ semantic segmentation model specifically designed for accurate and robust segmentation of solar panels from infrared images. The proposed model incorporates four key innovations: the replacement of the Xception backbone with the lightweight MobileNetV2, the integration of Canny edge detection for enhanced boundary feature extraction, the design of the SE-ASPP module for adaptive channel-wise feature recalibration, and the introduction of a parallel dual-channel pathway for enriched shallow feature propagation. Comprehensive experiments on a real-world dataset of infrared images of solar panels demonstrate that the proposed model achieves superior performance compared to existing state-of-the-art segmentation methods, with precision of 99.50%, mIoU of 99.21%, recall of 99.61%, and F1 score of 99.55%. Ablation studies confirm the individual and collective contributions of each proposed component to the overall performance improvement. The model effectively addresses the common challenges in solar panel segmentation, including wavy boundaries, panel粘连, and background misclassification, providing clean and accurate segmentation results that can significantly enhance the reliability of downstream fault detection systems. With its combination of high accuracy, computational efficiency, and robustness to diverse imaging conditions, the proposed model represents a valuable tool for advancing the automation of photovoltaic inspection and maintenance, contributing to the broader goal of sustainable and efficient solar energy utilization.