Solar energy plays an increasingly critical role in the global transition toward clean and sustainable power generation. As a key component of renewable energy systems, solar panels are deployed in large-scale photovoltaic farms and distributed rooftop installations. However, the long-term outdoor operation of solar panels inevitably leads to various surface defects, including dust accumulation, bird droppings, snow coverage, physical damage, and electrical degradation. These defects can significantly reduce the power conversion efficiency and even cause permanent damage to the photovoltaic modules. Traditional manual inspection methods are labor-intensive, time-consuming, and prone to human error, especially in large-scale solar farms. Therefore, developing an intelligent and automated surface defect detection system for solar panels is of great practical importance.
With the rapid advancement of deep learning, convolutional neural networks (CNNs) have become the backbone of modern object detection algorithms. Among these, the YOLO (You Only Look Once) family has gained widespread popularity due to its excellent balance between detection accuracy and inference speed. However, applying standard YOLO models to solar panel defect detection still faces several challenges, such as the small size of defects, complex background interference, and the requirement for real-time processing on edge devices. To address these issues, this paper presents a series of improvements to YOLOv5s and YOLOv8s, aiming at enhancing the detection performance of solar panel surface defects while maintaining high efficiency. The proposed methods are validated on a self-constructed dataset containing six typical solar panel states, and a complete detection system is developed based on the improved model.
1. Introduction
The global energy landscape is undergoing a profound transformation due to the pressing need to reduce greenhouse gas emissions and mitigate climate change. Solar energy, as one of the most abundant and clean renewable energy sources, has experienced phenomenal growth in recent decades. According to the International Energy Agency, photovoltaic installations have expanded dramatically, and solar panels now contribute to a substantial share of renewable power generation. Nevertheless, the reliable operation of solar panels is challenged by various environmental factors. Dust, bird droppings, snow, and leaves can cover the panel surface, causing partial shading and leading to the widely known “hot spot” effect. Physical damages such as cracks and electrical failures can further degrade the performance of solar panels and even pose safety risks.
Efficient operation and maintenance (O&M) of solar power plants are essential to maximize energy yield and ensure long-term reliability. Traditional inspection approaches rely on manual visual inspection or periodic electrical measurements, which are often inadequate for large-scale photovoltaic farms. The need for accurate, fast, and cost-effective defect detection has motivated the development of image-based automated inspection systems. In this context, object detection algorithms based on deep learning have shown remarkable success in identifying and localizing defects in solar panel images. However, most existing methods are either too slow for real-time applications or not sufficiently accurate for small and subtle defects. The work presented in this paper focuses on developing improved detection models that achieve both high precision and high speed, specifically tailored for solar panel surface defect detection.
The contributions of this work are threefold. First, we construct a comprehensive solar panel defect dataset containing six categories: clean, physical damage, electrical damage, dust, snow, and bird droppings. The dataset is augmented using geometric transformations and the Mosaic technique to improve model generalization. Second, we propose two-stage model enhancement strategies. In the first stage, we modify YOLOv5s by introducing DynamicConv, iRMB attention, and HATHead to improve feature extraction and detection accuracy. In the second stage, we develop a lightweight improved YOLOv8s model that incorporates SENetV2 channel attention, EMA attention, and the Inner-ShapeIoU loss function to achieve a better balance between accuracy and efficiency. Finally, we design and implement a PyQt5-based detection system that integrates the improved model, providing a user-friendly interface for solar panel defect inspection. The experimental results demonstrate that the proposed methods outperform baseline models in terms of both detection accuracy and inference speed.
2. Related Work
Solar panel defect detection has been approached from two main directions: electrical characteristics-based methods and visual image-based methods. Electrical characteristics-based methods analyze current-voltage (I-V) curves, power output, and other electrical parameters to diagnose abnormal conditions. For instance, some researchers developed threshold-based error analysis techniques to classify faults by comparing I-V curves. Others used artificial neural networks to detect short-circuit faults based on parameters at maximum power point. Although these methods can be effective, they require large-scale installation of sensors and wiring, which increases cost and maintenance complexity.
Visual image-based methods, particularly those relying on visible light images, have gained traction due to their lower cost and ease of deployment. Early works used infrared thermography to identify hot spots and thermal anomalies. More recent studies utilize deep convolutional neural networks to automatically extract features and classify defects. The YOLO series has become a favorite among practitioners for its state-of-the-art performance. However, standard YOLO models often struggle with small objects and complex backgrounds, which are common in solar panel images. Many improvements have been proposed, such as multi-scale feature fusion, attention mechanisms, and custom loss functions. Despite these advances, there remains a need for models that are both lightweight and highly accurate for real-time solar panel inspection. The current paper builds upon these prior works by introducing carefully designed modifications to YOLOv5s and YOLOv8s, focusing on the unique characteristics of solar panel defects.
3. Proposed Methods
3.1 Improved YOLOv5s Architecture
The base YOLOv5s model is a lightweight yet powerful object detector. Its architecture consists of an input stage, a backbone network, a neck module with PANet, and a detection head. To enhance the detection of solar panel defects, we introduce three improvements: DynamicConv, iRMB attention, and HATHead.
DynamicConv replaces the standard stationary convolutions in the backbone. Standard convolution uses a fixed kernel weight, whereas DynamicConv dynamically generates a convolution kernel conditioned on the input feature map. This allows the network to adapt to different defect patterns. Given an input feature map \( X \in \mathbb{R}^{C_{in}\times H\times W} \), the dynamic kernel \( W’ \) is computed by a weighted sum of \( M \) expert kernels \( W_i \):
\[
W’ = \sum_{i=1}^{M} \alpha_i(X) \cdot W_i
\]
where the attention coefficients \( \alpha_i(X) \) are obtained through a lightweight sub-network that performs global average pooling followed by two fully connected layers and a softmax activation. The output feature map is then computed by the standard convolution with the dynamic kernel:
\[
Y = X * W’
\]
This mechanism substantially increases the model’s capacity without a proportional increase in computational cost, because the dynamic kernel is shared across spatial locations.
iRMB attention is inserted in the neck network to enhance the extraction of fine-grained defect features. The iRMB module combines an expanded multi-head self-attention (EW-MHSA) and depth-wise convolution to capture both global context and local details. The input feature \( X \) is first expanded with a linear layer: \( X_e = MLP_e(X) \). Then the attention is computed as:
\[
Attn(X_e) = \text{Softmax}\left(\frac{QK^T}{\sqrt{d}}\right) V
\]
where \( Q = K = Linear(X) \) and \( V = X_e \). The output is passed through a depth-wise convolution and a channel contraction layer, followed by a residual connection:
\[
Y = X + MLP_s(DWConv(Attn(X_e)))
\]
This design allows the network to focus on relevant regions while keeping computational complexity manageable.
HATHead replaces the original detection head with a hybrid attention transformer head. The head leverages channel attention and spatial self-attention to refine the enriched features. The core structure is composed of hybrid attention blocks (HAB) and overlapping cross-attention blocks (OCAB). Through these blocks, the model adaptively reweights features along channel and spatial dimensions, leading to more accurate localization of solar panel defects, especially for small objects.
3.2 Lightweight Improved YOLOv8s
YOLOv8s inherently offers a better tradeoff between speed and accuracy due to its anchor-free design and decoupled head. To further boost its performance for solar panel defect detection, we introduce three key improvements: C2f-SENetV2, EMA attention, and Inner-ShapeIoU loss.
C2f-SENetV2 integrates a SENetV2 attention module into the C2f block of the backbone. SENetV2, which stands for Squeeze-and-Excitation Network version 2, uses a Squeeze aggregated Excitation (SaE) unit to enhance the modeling of channel dependencies. Given an input \( x \), the output is computed as:
\[
Y = F(x \cdot Ex(\sum Sq(x)))
\]
where \( Sq \) is the squeeze operation, \( Ex \) is the excitation operation, and \( F \) represents the residual-branch operations. The SENetV2 module selectively enhances critical channels while suppressing irrelevant ones, which is particularly beneficial when distinguishing solar panel defects from complex backgrounds.
EMA attention is incorporated into the neck network to improve multi-scale feature representation. The efficient multi-scale attention mechanism divides input features into groups and processes them via three parallel branches. The horizontal and vertical branches capture long-range dependencies, while a 3×3 convolution branch captures cross-channel interactions. The output features are fused using cross-space learning. The mathematical formulation for the horizontal attention is:
\[
Z_c^H(H) = \frac{1}{W}\sum_{i=0}^{W} X_c(H,i)
\]
and for the vertical: \( Z_c^W(W) = \frac{1}{H}\sum_{j=0}^{H} X_c(j,W) \). The combined feature map \( D \) is obtained by concatenation and a 1×1 convolution. The EMA module helps the network emphasize important features and suppress background noise, leading to more robust solar panel defect detection.
Inner-ShapeIoU loss is designed to replace the standard CIoU loss for bounding box regression. The original CIoU loss suffers from gradient vanishing when the aspect ratios are identical but sizes differ. The proposed loss incorporates both shape similarity and inner region overlap. The ShapeIoU part is defined as:
\[
L_{shape} = 1 – IoU + distance_{shape} + 0.5 \cdot \Omega_{shape}
\]
where the distance term is weighted by width and height scale factors, and \( \Omega_{shape} \) represents the shape consistency. Then, the InnerIoU term introduces an auxiliary bounding box to refine the regression:
\[
IoU_{inner} = \frac{inter}{union}
\]
Finally, the composite loss is given by:
\[
L_{Inner-ShapeIoU} = L_{shape} + IoU – IoU_{inner}
\]
This loss function significantly improves the localization accuracy for solar panel defects with varying shapes and sizes.
4. Experimental Setup
4.1 Dataset Construction
Due to the scarcity of public labeled data for solar panel surface defects, we constructed a dedicated dataset by collecting images from both open-source resources and real photovoltaic stations. The dataset covers six typical categories: clean, physical damage, electrical damage, dust, snow, and bird droppings. First, we extracted the effective solar panel region from the original images to remove irrelevant background. Then, we applied Gaussian filtering to reduce noise. Each defect was manually annotated using the LabelImg tool, producing bounding box coordinates and category labels. To improve the generalization capability, we performed geometric transformations including horizontal and vertical flips, random scaling, random shearing, and perspective transformation. In addition, we used the Mosaic data augmentation technique, which randomly combines four training images into one composite image. The final dataset consists of 14,478 images, which were divided into training, validation, and testing sets with an 8:1:1 ratio, i.e., 11,582 training images, 1,448 validation images, and 1,448 test images.
4.2 Implementation Details
All experiments were conducted on a workstation equipped with an NVIDIA RTX 4060 GPU with 8GB memory, running Windows 11. The deep learning framework was Python 3.8.5 and PyTorch 1.9.0 with CUDA 11.6. The input image size was set to 640×640. We used SGD optimizer with an initial learning rate of 0.01, momentum of 0.937, weight decay of 0.0005, batch size of 16, and 200 training epochs. The hyperparameters are summarized in the following table.
| Hyperparameter | Value |
|---|---|
| Image size | 640 |
| Epochs | 200 |
| Batch size | 16 |
| Optimizer | SGD |
| Initial learning rate | 0.01 |
| Momentum | 0.937 |
| Weight decay | 0.0005 |
5. Results and Discussion
5.1 Evaluation Metrics
We used precision (P), recall (R), mean average precision (mAP), and frames per second (FPS) to evaluate the detection models. Precision and recall are defined as:
\[
P = \frac{TP}{TP+FP}, \quad R = \frac{TP}{TP+FN}
\]
where TP, FP, and FN are true positives, false positives, and false negatives, respectively. The mAP values are computed at IoU thresholds of 0.5 and 0.5:0.95.
5.2 Comparison of Conv Modules
We first evaluated the effect of replacing the standard convolution in the improved YOLOv5s with different dynamic convolution modules. The results are presented in Table 1. The DynamicConv module achieved the best mAP (87.7%) and the highest FPS (143), indicating its excellent tradeoff between accuracy and efficiency.
| Conv Module | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) | FPS |
|---|---|---|---|---|---|
| ODConv | 80.5 | 79.2 | 84.4 | 84.3 | 133 |
| SAConv | 83.9 | 80.7 | 86.5 | 86.4 | 111 |
| RFAConv | 82.7 | 77.2 | 84.7 | 84.6 | 89 |
| DynamicConv | 84.4 | 79.2 | 87.7 | 87.7 | 143 |
5.3 Comparison of Detection Heads
We compared HATHead with other popular detection heads, including RepHead, SAHead, and FRMHead. HATHead achieved the highest recall (85.4%) and mAP (89.4%) while maintaining a high FPS (141), as shown in Table 2.
| Head | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) | FPS |
|---|---|---|---|---|---|
| RepHead | 85.2 | 80.2 | 88.5 | 88.4 | 135 |
| SAHead | 79.9 | 83.6 | 87.9 | 87.9 | 133 |
| HATHead | 81.9 | 85.4 | 89.4 | 89.4 | 141 |
| FRMHead | 88.1 | 75.5 | 87.3 | 87.2 | 92 |
5.4 Ablation Study for Improved YOLOv5s
Table 3 shows the contribution of each module added to the baseline YOLOv5s. The full model with DynamicConv, HATHead, and iRMB achieves the best mAP@0.5 of 90.2% and recall of 87.5%, demonstrating that the three modules complement each other.
| YOLOv5s | DynamicConv | HATHead | iRMB | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) | FPS |
|---|---|---|---|---|---|---|---|---|
| ✓ | – | – | – | 85.5 | 77.9 | 86.6 | 86.5 | 147 |
| ✓ | ✓ | – | – | 84.4 | 79.2 | 87.7 | 87.7 | 143 |
| ✓ | – | ✓ | – | 81.9 | 85.4 | 89.4 | 89.4 | 141 |
| ✓ | – | – | ✓ | 80.7 | 83.4 | 87.4 | 87.3 | 101 |
| ✓ | ✓ | ✓ | – | 89.1 | 79.7 | 88.4 | 88.4 | 137 |
| ✓ | – | ✓ | ✓ | 88.5 | 81.9 | 89.6 | 89.6 | 91 |
| ✓ | ✓ | – | ✓ | 88.0 | 81.6 | 87.5 | 87.5 | 96 |
| ✓ | ✓ | ✓ | ✓ | 81.3 | 87.5 | 90.2 | 90.1 | 90 |
5.5 Comparison of Loss Functions
We evaluated different regression loss functions in the improved YOLOv8s framework. As shown in Table 4, the proposed Inner-ShapeIoU loss yields the highest mAP@0.5 of 90.5% and precision of 87.2%, while maintaining competitive speed.
| Loss Function | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) | FPS |
|---|---|---|---|---|---|
| CIoU | 82.6 | 83.6 | 89.4 | 89.3 | 135 |
| WIoU | 82.2 | 85.1 | 89.0 | 89.0 | 217 |
| EIoU | 83.1 | 80.6 | 90.2 | 90.2 | 222 |
| ShapeIoU | 86.7 | 84.1 | 89.5 | 89.5 | 135 |
| InnerIoU | 85.5 | 80.5 | 89.1 | 89.0 | 137 |
| Inner-ShapeIoU | 87.2 | 83.6 | 90.5 | 90.5 | 137 |
5.6 Ablation Study for Improved YOLOv8s
We conducted an ablation study for the YOLOv8s improvements, as summarized in Table 5. The full model combining C2f-SENetV2, EMA, and Inner-ShapeIoU achieves the best mAP@0.5 of 92.1% and mAP@0.5:0.95 of 91.8%, while maintaining an FPS of 116.
| YOLOv8s | C2f-SENetV2 | Inner-ShapeIoU | EMA | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) | FPS |
|---|---|---|---|---|---|---|---|---|
| ✓ | – | – | – | 82.6 | 83.6 | 89.4 | 89.3 | 137 |
| ✓ | ✓ | – | – | 81.2 | 88.1 | 90.2 | 90.2 | 135 |
| ✓ | – | ✓ | – | 87.2 | 83.6 | 90.5 | 90.5 | 137 |
| ✓ | – | – | ✓ | 90.1 | 80.4 | 90.1 | 90.1 | 119 |
| ✓ | ✓ | ✓ | – | 84.0 | 85.3 | 90.1 | 90.0 | 135 |
| ✓ | – | ✓ | ✓ | 86.5 | 80.4 | 89.8 | 89.8 | 119 |
| ✓ | ✓ | – | ✓ | 80.5 | 86.3 | 90.8 | 90.8 | 116 |
| ✓ | ✓ | ✓ | ✓ | 88.3 | 81.7 | 92.1 | 91.8 | 116 |
5.7 Overall Model Comparison
We compared the proposed improved YOLOv8s with a wide range of state-of-the-art detectors, including YOLOv5 variants, YOLOv6, YOLOv8 variants, YOLOv9s, YOLOv11s, and the improved YOLOv5s from the previous stage. Table 6 presents the comprehensive results. The improved YOLOv8s model achieves the best mAP@0.5 (92.1%) and mAP@0.5:0.95 (91.8%), outperforming all baselines while retaining an FPS of 116, which is sufficient for real-time solar panel inspection.
| Model | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) | FPS |
|---|---|---|---|---|---|
| YOLOv5n | 80.5 | 78.8 | 84.8 | 84.7 | 294 |
| YOLOv5s | 85.5 | 77.9 | 86.6 | 86.5 | 147 |
| YOLOv5m | 82.9 | 79.2 | 86.3 | 86.3 | 68 |
| YOLOv5l | 82.5 | 83.3 | 86.9 | 86.8 | 52 |
| YOLOv6n | 76.8 | 82.5 | 85.6 | 85.6 | 217 |
| YOLOv6s | 77.8 | 84.5 | 86.5 | 86.5 | 141 |
| YOLOv6m | 74.5 | 83.6 | 84.7 | 84.7 | 52 |
| YOLOv6l | 74.5 | 80.1 | 84.3 | 84.0 | 34 |
| YOLOv8n | 83.4 | 81.8 | 89.0 | 88.9 | 270 |
| YOLOv8s | 82.6 | 83.6 | 89.4 | 89.3 | 137 |
| YOLOv8m | 84.1 | 81.1 | 89.3 | 89.3 | 62 |
| YOLOv8l | 81.6 | 82.2 | 89.2 | 89.2 | 45 |
| YOLOv8s-p2 | 81.3 | 83.1 | 89.0 | 88.9 | 85 |
| YOLOv8s-p6 | 81.8 | 73.7 | 82.2 | 82.2 | 141 |
| YOLOv8s-ghost | 83.0 | 81.1 | 87.5 | 87.4 | 159 |
| YOLOv9s | 84.8 | 74.4 | 85.1 | 85.0 | 125 |
| YOLOv11s | 83.7 | 80.4 | 88.5 | 88.0 | 138 |
| Improved YOLOv5 | 81.3 | 87.5 | 90.2 | 90.1 | 90 |
| Improved YOLOv8 (ours) | 88.3 | 81.7 | 92.1 | 91.8 | 116 |
The experimental results clearly demonstrate that the proposed improved YOLOv8s model achieves superior detection accuracy for solar panel defects while maintaining a high inference speed. The improvements introduced in both the backbone and neck are effective in capturing subtle defects, and the customized loss function further refines the regression quality.
6. Solar Panel Defect Detection System
To bridge the gap between algorithm research and practical application, we developed a complete solar panel defect detection system based on the improved YOLOv8s model. The system is built using the PyQt5 framework and integrates three major functional modules: the detection module, the result visualization module, and the result saving module. The system supports multiple input sources, including still images, image folders, video files, and real-time camera streams. After loading the model, the user can click the “Run” button to perform detection. The interface displays the annotated image with bounding boxes and class labels, along with a structured table listing each detection result, including category, confidence score, and coordinates. The detection results can be saved to local storage for future analysis and record keeping.
The system was tested on our solar panel defect dataset and successfully identified all six categories. The user-friendly interface allows maintenance personnel to quickly assess the condition of solar panels and locate defects. The integration of the improved model ensures both high accuracy and real-time performance, making the system practical for deployment in real photovoltaic power stations.
7. Conclusion and Future Work
In this paper, we presented a comprehensive study on solar panel surface defect detection using improved YOLO-based models. We first constructed a carefully curated dataset covering six typical solar panel states with extensive data augmentation. Then, we proposed two-stage model improvements: an enhanced YOLOv5s architecture with DynamicConv, iRMB attention, and HATHead, and a lightweight improved YOLOv8s model with C2f-SENetV2, EMA attention, and Inner-ShapeIoU loss. Extensive experiments showed that the improved YOLOv8s model achieves the best performance, with a mAP@0.5 of 92.1% and a FPS of 116, outperforming all baseline models. Finally, we developed a PyQt5-based detection system that integrates the improved model, providing a practical tool for solar panel inspection.
Future work will focus on enhancing the model’s robustness under extreme weather conditions, such as heavy rain, fog, and snowstorms. Additionally, we plan to optimize the model for deployment on edge devices such as drones and embedded platforms, enabling an integrated air-ground inspection system for large-scale solar farms. The continued development of accurate and efficient solar panel defect detection technologies will play a vital role in the intelligent operation and maintenance of photovoltaic power plants.