In recent years, photovoltaic power generation has developed rapidly because of its low cost and environmentally friendly characteristics. The demand and scale of photovoltaic systems have also increased accordingly. However, during the operation of photovoltaic power plants, solar panels often suffer from various types of faults due to long-term exposure to harsh environmental conditions, which directly reduces the power generation efficiency and economic benefits. Traditional fault detection methods for solar panels mainly rely on electrical characteristic analysis or manual visual inspection, which are often inefficient, unsafe, and cannot meet the real-time requirements of modern power plants. With the rise of deep learning, fault detection approaches based on convolutional neural networks have attracted tremendous attention. In this paper, I focus on the research of fault detection algorithms for solar panels using deep learning, with the goal of achieving high accuracy and high speed detection of multiple fault types.
My research is organized as follows. First, I constructed a high-quality infrared image dataset of solar panels by addressing issues such as image noise, sample imbalance, and insufficient sample size. Second, I proposed an improved YOLOv7 algorithm named YOLOv7-PV to enhance detection accuracy and reduce missed detections. Third, I designed an improved YOLOv8 algorithm to achieve a better balance between detection precision and inference speed. Extensive experiments demonstrate that the proposed methods outperform several mainstream baseline detectors on the constructed dataset.
1. Background and Motivation
Solar energy is one of the most promising renewable energy sources due to its cleanliness, abundance, and sustainability. Large-scale photovoltaic plants are often deployed in remote areas with extreme weather conditions, such as deserts, mountains, and coastal regions. These environments expose solar panels to strong winds, hailstorms, dust accumulation, snow loads, and high ultraviolet radiation, which lead to various defects including cracks, hot spots, dust shading, cell string failures, and vegetation occlusion. These faults not only reduce the output power but also pose safety risks such as short circuits and fire hazards. Therefore, reliable and timely fault detection is critical for maintaining the stable operation of photovoltaic systems.
The adoption of unmanned aerial vehicles (UAVs) equipped with infrared cameras has become a popular solution for photovoltaic plant inspection. Infrared thermography captures the thermal radiation emitted by solar panels, and faults such as hot spots or cracks often exhibit distinct temperature patterns compared to normal areas. However, the raw infrared images usually suffer from low contrast, noise, and complex backgrounds. Moreover, the number of fault samples is often limited and imbalanced, which makes it challenging to train deep learning models with good generalization.
Deep learning-based object detection methods have shown superior performance in various industrial vision tasks. In particular, the YOLO family of single-stage detectors offers a favorable trade-off between accuracy and speed. I therefore selected YOLOv7 and YOLOv8 as baseline models and introduced several architectural improvements to adapt them to the specific characteristics of solar panel fault detection.
2. Related Work
2.1 Electrical characteristic based methods
These methods monitor parameters such as output voltage, current, power, irradiance, and temperature to identify faults. For example, artificial neural networks and fuzzy clustering have been used to classify different fault types. Although these approaches are low-cost, they require the installation of many sensors and cannot localize faults precisely.
2.2 Image feature based methods
Visible light images, electroluminescence (EL) images, and infrared images have been exploited for fault detection. Visible light images can reveal surface cracks and dirt, while EL images are effective for micro-crack detection but require specialized equipment. Infrared thermography allows non-contact inspection and can identify thermal anomalies such as hot spots. Traditional image processing techniques, such as edge detection, thresholding, and support vector machines, have been applied but often fail to generalize to complex scenes.
2.3 Deep learning based methods
Convolutional neural networks (CNNs) have revolutionized object detection. Two-stage detectors like Faster R-CNN provide high accuracy but are relatively slow. Single-stage detectors, especially the YOLO series, are preferred for real-time applications. Several studies have applied YOLO variants to solar panel fault detection. However, the detection performance is still limited by the small size of fault regions and the imbalance of fault categories. To address these challenges, I propose specialized modules and loss functions that enhance feature extraction and small-target detection.
3. Construction of the Solar Panel Fault Dataset
3.1 Data acquisition
I used a DJI M300RT quadrotor UAV equipped with an HT20 thermal camera with a resolution of 640 × 512 to collect infrared images from 13 photovoltaic plants located in Shanxi, Inner Mongolia, Jiangsu, and Heilongjiang provinces. The UAV flew at altitudes of 10–15 meters and 20–30 meters. For flat terrain, I used an automatic path planning algorithm to generate uniform coverage routes. For rugged terrain, I manually selected the starting and ending points of the photovoltaic arrays. During flight, a line-following strategy corrected deviations from the planned path.
In total, 4593 raw infrared images were captured. After removing images without solar panels or with too blurry contents, 2035 valid images were retained for further processing.
3.2 Fault types and characteristics
The collected dataset contains five types of faults commonly observed in photovoltaic plants:
- Fragmentation (碎裂): The broken areas appear as bright, mosaic-like blocks with high intensity in infrared images.
- Hot spot (热斑): Localized short-circuit regions exhibit one or more bright spot-like or square features.
- Dust shading (蒙尘): Dust-covered areas have uneven color distribution and blurred edges due to light scattering.
- Cell string failure (电池串): A bright rectangular strip appears with a height similar to that of the solar panel.
- Vegetation occlusion (植物遮挡): Occluded regions appear as dark, irregular shadows with plant-like contours, usually located near panel edges or gaps.
3.3 Image enhancement
Raw infrared images often contain noise and have poor contrast. I applied both spatial domain and frequency domain enhancement techniques. In the spatial domain, histogram equalization redistributes pixel intensities to improve contrast. Spatial filtering, such as median filtering, reduces salt-and-pepper noise. In the frequency domain, I performed a Fourier transform, multiplied the spectrum by a high-pass or homomorphic filter, and then applied the inverse transform to enhance edges and details. The enhanced images provide clearer fault features and facilitate model training.
The enhancement process can be expressed as:
$$G(u,v) = H(u,v) \cdot F(u,v)$$
where \(F(u,v)\) is the Fourier transform of the input image \(f(x,y)\), \(H(u,v)\) is the filter function, and \(G(u,v)\) is the filtered spectrum. The enhanced image is obtained by the inverse Fourier transform \(g(x,y) = \mathcal{F}^{-1}\{G(u,v)\}\).
3.4 Sample balancing and augmentation
In the original dataset, fragmentation samples accounted for only 2% and vegetation occlusion samples for 5% of the total, causing a severe class imbalance. To balance the number of samples, I used different strategies for different fault types.
Fragmentation augmentation: Since fragmentation often covers a large area of a single solar panel, I divided each panel into four equal sub-images and randomly recombined them. If \(n\) panels produce \(4n\) sub-images, the number of possible recombination results is \(256n^4\), which greatly increases the number of fragmentation samples while preserving the essential texture and brightness information.
Vegetation occlusion augmentation: I extracted the contours of typical plant occlusions and drew different irregular shapes resembling leaves and branches. These synthetic occlusions were then pasted onto normal solar panel images at realistic positions (e.g., edges and gaps between panels). This process generated diverse vegetation occlusion samples that mimic natural conditions.
General image diversification: To simulate UAV flights at different heights and angles, I performed random scaling and random rotation on the images. Scaling simulates different altitudes, while rotation simulates different camera orientations. These augmentations increase the diversity of the dataset and reduce the risk of overfitting.
After augmentation, the total number of infrared images reached 7864 samples. Table 1 summarizes the number of samples for each fault type after processing.
| Fault type | Original samples | Augmented samples |
|---|---|---|
| Fragmentation | 41 | 1532 |
| Hot spot | 689 | 1720 |
| Dust shading | 758 | 1856 |
| Vegetation occlusion | 102 | 1478 |
| Cell string failure | 445 | 1278 |
3.5 Annotation
I used the LabelImg tool to manually annotate all images in YOLO format. Each annotation file contains a single line per object with class id and normalized bounding box coordinates \(x_{center}\), \(y_{center}\), \(width\), \(height\). This annotation format is directly compatible with the YOLO training pipeline.
3.6 Impact of dataset processing
To verify the effectiveness of the dataset processing, I trained several baseline detectors on the raw dataset and the processed dataset. The results are shown in Table 2.
| Model | mAP before (%) | mAP after (%) | Precision P (%) | Recall R (%) |
|---|---|---|---|---|
| YOLOv5s | 71.10 | 84.60 | 84.36 | 82.96 |
| YOLOv7 | 75.43 | 88.58 | 80.71 | 88.54 |
| YOLOv8 | 73.59 | 87.56 | 87.51 | 83.79 |
It can be observed that after filtering, enhancement, augmentation, and balancing, the mAP of all models increased significantly. For example, YOLOv7 improved by 13.15 percentage points, and YOLOv8 improved by 13.97 percentage points. This demonstrates the importance of constructing a high-quality dataset for solar panel fault detection.
4. Improved YOLOv7 for Solar Panel Fault Detection
4.1 Architecture of YOLOv7
YOLOv7 consists of an input preprocessing module, a backbone network, and a head network. The backbone uses the ELAN (Efficient Layer Aggregation Network) structure to extract rich features. The head network employs a feature pyramid network (FPN) with an SPPCSPC module and an ELAN-H module for multi-scale prediction. The model also uses reparameterization and auxiliary training heads to improve performance.
The standard YOLOv7 model has strong detection capability, but when applied to solar panel faults, it suffers from several issues: small fault regions are easily missed, the background is complex, and the detection accuracy is insufficient for industrial requirements. Therefore, I propose the YOLOv7-PV architecture with the following improvements.
4.2 ResNet-PV module
The ResNet-PV module is based on the residual bottleneck structure but substitutes the standard \(3\times3\) convolution with partial convolution (PConv). PConv applies regular convolution to only a subset of input channels, reducing redundant computation and memory access. The number of floating-point operations (FLOPs) for PConv is:
$$FLOPs = h \times w \times k^2 \times c_p^2$$
where \(c_p = r \times c\) is the number of sampled channels. When \(r=1/4\), the FLOPs is one-sixteenth of a regular convolution. The memory access is approximately:
$$M \approx h \times w \times c_p \times 2 = \frac{1}{4} \times h \times w \times c \times 2$$
The ResNet-PV module preserves the original input through a skip connection, which helps avoid information loss and enhances gradient flow. This module is inserted before the prediction heads to refine the features and improve localization.
4.3 ELAN-PV module
The ELAN-PV module modifies the original ELAN structure in two ways. First, all \(3\times3\) convolutions are replaced with PConv to reduce computation and remove redundant features. Second, the SiLU activation function is replaced with the RReLU (Randomized ReLU) activation. RReLU introduces a random negative slope \(a\) during training:
$$f(x) = \begin{cases} ax, & x>0 \\ x, & x \le 0 \end{cases}$$
Here \(a\) is sampled from a uniform distribution during training and fixed during inference. RReLU alleviates the dead neuron problem and improves model stability. The combination of PConv and RReLU in ELAN-PV enhances feature extraction efficiency and makes the model more robust to small target variations.
4.4 BiFormer attention mechanism
BiFormer is a vision transformer with bi-level routing attention, which dynamically selects a small set of key regions for each query token. It combines the advantages of global attention and local attention while reducing computational cost. I inserted BiFormer modules after certain convolution blocks in the backbone and head to strengthen the extraction of discriminative features for solar panel faults. The bi-level routing operation enables the model to focus on relevant spatial regions, which is beneficial for detecting small hot spots and thin crack lines.
4.5 NWD loss function
The original YOLOv7 uses CIoU loss for bounding box regression. CIoU is sensitive to small object localization and does not perform well when bounding boxes do not overlap. I replaced CIoU with the Normalized Wasserstein Distance (NWD) loss, which models bounding boxes as 2D Gaussian distributions and computes the Wasserstein distance between them. The NWD loss is:
$$NWD(N_a, N_b) = \exp\left(-\frac{\sqrt{W_2^2(N_a, N_b)}}{C}\right)$$
where \(W_2^2(N_a, N_b)\) is the second-order Wasserstein distance between two Gaussian distributions \(N_a\) and \(N_b\) derived from the bounding boxes \(A=(cx_a, cy_a, w_a, h_a)\) and \(B=(cx_b, cy_b, w_b, h_b)\):
$$W_2^2(N_a, N_b) = \left( \left(cx_a – cx_b\right)^2 + \left(cy_a – cy_b\right)^2 \right) + \frac{1}{2}\left( \left(w_a – w_b\right)^2 + \left(h_a – h_b\right)^2 \right)$$
Here \(C\) is a constant related to the dataset. NWD loss is insensitive to scale and can measure similarity even for non-overlapping boxes, which greatly improves the detection of tiny fault regions.
4.6 Experimental results of YOLOv7-PV
I conducted experiments on the constructed dataset with an input size of \(640\times640\). The evaluation metrics include mean Average Precision (mAP) and frames per second (FPS). The comparison with state-of-the-art detectors is shown in Table 3.
| Model | Fragmentation (%) | Hot spot (%) | Dust (%) | Vegetation (%) | Cell string (%) | FPS | mAP (%) |
|---|---|---|---|---|---|---|---|
| YOLOv5s | 96.1 | 60.7 | 89.5 | 83.7 | 93.0 | 60 | 84.60 |
| RetinaNet | 95.3 | 61.6 | 88.3 | 84.1 | 92.7 | 57 | 84.03 |
| T-DINO | 94.8 | 56.7 | 83.1 | 80.4 | 90.3 | 49 | 81.13 |
| YOLOv7 | 97.2 | 76.6 | 87.6 | 85.7 | 95.8 | 62 | 88.58 |
| YOLOv7-tiny | 98.1 | 75.4 | 89.3 | 80.3 | 93.1 | 65 | 87.24 |
| YOLOv7-PV | 98.7 | 80.1 | 93.2 | 91.2 | 95.6 | 69 | 91.76 |
YOLOv7-PV achieves a mAP of 91.76%, which is 3.18% higher than YOLOv7, 4.52% higher than YOLOv7-tiny, and 7.16% higher than YOLOv5s. The improvements are particularly significant for hot spot and vegetation occlusion, which are small and irregular targets. The FPS of YOLOv7-PV is 69, which is higher than all compared models, indicating that the proposed modifications not only improve accuracy but also accelerate inference.
4.7 Ablation study for YOLOv7-PV
I performed ablation experiments to evaluate the contribution of each component. The results are shown in Table 4.
| YOLOv7 | ResNet-PV | ELAN-PV | BiFormer | NWD Loss | FPS | mAP (%) |
|---|---|---|---|---|---|---|
| ✓ | 62 | 88.58 | ||||
| ✓ | ✓ | 60 | 90.71 | |||
| ✓ | ✓ | 75 | 89.65 | |||
| ✓ | ✓ | 67 | 90.34 | |||
| ✓ | ✓ | 62 | 89.27 | |||
| ✓ | ✓ | ✓ | ✓ | ✓ | 69 | 91.76 |
Adding ResNet-PV increases mAP by 2.13% but slightly reduces FPS due to the additional parameters. The ELAN-PV module significantly increases FPS by 13 frames/s while improving mAP by 1.07%. BiFormer contributes a 1.76% mAP improvement. NWD loss provides a 0.69% mAP increase. The combination of all four modules yields the best mAP of 91.76% at 69 FPS, achieving a good balance between accuracy and speed.
5. Improved YOLOv8 for Efficient Solar Panel Fault Detection
5.1 Overview of YOLOv8
YOLOv8 is the latest iteration of the YOLO series, featuring an anchor-free design, a decoupled head for classification and regression, and a C2f module in the backbone. It offers excellent inference speed and relatively high accuracy. However, when directly applied to solar panel fault detection, the model still struggles with small objects and complex backgrounds. To further improve both accuracy and speed, I introduced three modifications to YOLOv8.
5.2 D_C2f module
In the backbone network, I replaced the original C2f modules (except the first one) with D_C2f modules. The D_C2f module uses deformable convolutions (DConv) in its bottleneck blocks instead of standard convolutions. Deformable convolutions add learnable offsets to the sampling grid, allowing the convolution kernel to adaptively focus on relevant regions. The output feature map is computed as:
$$y(p_0) = \sum_{n=1}^{N} w(p_n) \cdot x(p_0 + p_n + \Delta p_n)$$
where \(p_n\) is the predefined offset for the n-th sampling point, and \(\Delta p_n\) is the learned offset. This adaptive receptive field improves feature extraction for objects with varying shapes and sizes, which is common in solar panel faults such as cracks and hot spots.
The D_C2f module also employs a split operation to separate feature maps into multiple branches, and then fuses them to enrich the feature representation. This design reduces redundant gradient information and speeds up the model fitting process.
5.3 SE_Conv module
To suppress background interference and emphasize important channel information, I introduced the SE_Conv module, which is a standard convolution followed by a Squeeze-and-Excitation (SE) attention block. The SE block performs global average pooling to generate a channel descriptor \(z \in \mathbb{R}^{C}\), then uses two fully-connected layers to produce channel-wise weights \(s\):
$$s = \sigma\left( W_2 \delta\left( W_1 z \right) \right)$$
where \(\delta\) is the ReLU activation, \(\sigma\) is the sigmoid function, \(W_1 \in \mathbb{R}^{C/r \times C}\), and \(W_2 \in \mathbb{R}^{C \times C/r}\). The final output is obtained by rescaling the original feature map \(x\) with the weights:
$$\tilde{x}_c = s_c \cdot x_c$$
I placed the SE_Conv module in the neck network to refine the fused features before they are passed to the detection head. This helps the model focus on the most informative channels and improves the discrimination of fault regions from the background.
5.4 WIoU v3 loss function
The original YOLOv8 uses CIoU as the bounding box regression loss. CIoU has some drawbacks in small object scenarios. I replaced it with Wise IoU v3 (WIoU v3), which introduces a dynamic non-monotonic focusing mechanism to evaluate the quality of anchor boxes. The loss is defined as:
$$L_{WIoUv3} = r \cdot L_{WIoUv1}$$
where \(L_{WIoUv1}\) is the base loss:
$$L_{WIoUv1} = R_{WIoU} \cdot L_{IoU}$$
and
$$R_{WIoU} = \exp\left( \frac{(x – x_{gt})^2 + (y – y_{gt})^2}{(W_g^2 + H_g^2)} \right)$$
The focusing factor \(r\) is:
$$r = \frac{\beta}{\delta \alpha^{\beta – \delta}}$$
\(\beta\) is the outlier degree, defined as \(\beta = \frac{L_{IoU}^*}{L_{IoU}} \in [0, +\infty)\), where \(L_{IoU}^*\) is the moving average of \(L_{IoU}\). The dynamic non-monotonic focus mechanism allocates smaller gradient gains to low-quality samples and larger gains to high-quality ones, thus improving the robustness of the model.
5.5 Experimental results of improved YOLOv8
I evaluated the improved YOLOv8 on the same dataset and compared it with YOLOv7 and YOLOv7-tiny. Table 5 presents the results.
| Model | Fragmentation (%) | Hot spot (%) | Dust (%) | Vegetation (%) | Cell string (%) | FPS | mAP (%) |
|---|---|---|---|---|---|---|---|
| YOLOv7 | 97.2 | 76.6 | 87.6 | 85.7 | 95.8 | 62 | 88.58 |
| YOLOv7-tiny | 98.1 | 75.4 | 89.3 | 80.3 | 93.1 | 65 | 87.24 |
| YOLOv8 | 97.8 | 77.5 | 86.4 | 85.9 | 94.9 | 73 | 87.56 |
| Improved YOLOv8 | 98.5 | 81.2 | 91.2 | 90.5 | 95.0 | 79 | 91.04 |
The improved YOLOv8 achieves a mAP of 91.04%, which is 3.48% higher than the original YOLOv8. It also surpasses YOLOv7 by 2.46% and YOLOv7-tiny by 3.80%. More importantly, the FPS is 79, which is significantly faster than YOLOv7 (62 FPS), YOLOv7-tiny (65 FPS), and even faster than YOLOv8 itself by 6 FPS. These results demonstrate that the proposed improvements effectively enhance both accuracy and inference speed.
5.6 Ablation study for improved YOLOv8
Table 6 shows the ablation results.
| YOLOv8 | D_C2f | WIoU v3 | SE_Conv | FPS | mAP (%) |
|---|---|---|---|---|---|
| ✓ | 73 | 87.56 | |||
| ✓ | ✓ | 81 | 87.71 | ||
| ✓ | ✓ | 76 | 88.75 | ||
| ✓ | ✓ | 73 | 89.60 | ||
| ✓ | ✓ | ✓ | ✓ | 79 | 91.04 |
D_C2f improves mAP by 0.15% and increases FPS by 8. WIoU v3 adds 1.19% mAP. SE_Conv adds 2.04% mAP but does not change FPS. The full model achieves the best mAP of 91.04% at 79 FPS. The improved YOLOv8 is particularly suitable for real-time fault detection tasks in large photovoltaic plants.
6. Discussion
Comparing the two proposed models, YOLOv7-PV achieves a slightly higher mAP (91.76%) than the improved YOLOv8 (91.04%), but the improved YOLOv8 is faster (79 FPS vs 69 FPS). Both models outperform their respective baselines by significant margins. The choice between them depends on the specific requirements: YOLOv7-PV is preferable when maximum accuracy is critical, while the improved YOLOv8 is better suited for real-time applications where inference speed is essential.
The key innovation of my work lies in the combination of lightweight modules (PConv, deformable convolutions), attention mechanisms (BiFormer, SE), and advanced loss functions (NWD, WIoU v3) tailored to the characteristics of solar panel faults. These modules can be easily integrated into other YOLO-based detectors and potential applications beyond photovoltaic inspection, such as defect detection in other industrial surfaces.
7. Conclusion and Future Work
In this paper, I have presented a comprehensive study on deep learning-based fault detection algorithms for solar panels. I built a high-quality infrared image dataset by enhancing images, augmenting scarce samples, and balancing fault categories. Based on this dataset, I proposed two improved detection models:
- YOLOv7-PV, which incorporates ResNet-PV, ELAN-PV, BiFormer attention, and NWD loss, achieving a mAP of 91.76% and an inference speed of 69 FPS.
- Improved YOLOv8, which introduces D_C2f, SE_Conv, and WIoU v3 loss, achieving a mAP of 91.04% and a speed of 79 FPS.
The experimental results demonstrate that both models significantly outperform the original YOLOv5s, YOLOv7, YOLOv7-tiny, and YOLOv8 baselines in terms of detection accuracy and speed. The improved YOLOv8 offers a better trade-off between accuracy and speed, making it highly practical for real-time industrial inspection.
There are several directions for future research. First, I plan to explore more advanced transformer-based backbones to further improve the detection of extremely small faults. Second, model compression techniques such as pruning and quantization can reduce the model size and enable deployment on embedded UAV platforms. Third, semi-supervised learning and domain adaptation can reduce the annotation effort and improve generalization to new photovoltaic plants. Finally, the integration of visible and infrared image fusion may provide complementary information and enhance the overall detection robustness.
In conclusion, the proposed deep learning-based methods provide an effective solution for automatic solar panel fault detection, contributing to the safe and efficient operation of photovoltaic power stations.