In the rapidly expanding field of solar energy, the efficiency of photovoltaic (PV) modules is critically dependent on the cleanliness of their surfaces. As a researcher deeply involved in AI-based inspection systems, I have observed that dust accumulation on solar panels is a pervasive issue that significantly reduces power output. Studies have shown that when dust density reaches 0.64 g/m², the output power of a solar panel can drop by 14.26%. The degradation follows an exponential decay pattern with increasing dust coverage. Moreover, severe dust can induce hot-spot effects, further damaging the solar panel’s lifespan. Therefore, accurate and robust detection of dust on solar panels is essential for proactive maintenance and optimal energy harvesting.
Traditional dust detection methods rely on manual inspection or threshold-based image processing. These approaches, however, are highly sensitive to varying illumination, weather conditions, and the complex textures of solar panel surfaces. With the advent of deep learning, object detection and segmentation have achieved remarkable success. Yet, the performance of deep learning models is heavily dependent on the quantity and quality of annotated data. In real-world photovoltaic plants, collecting diverse dust images under different environmental conditions is challenging, leading to high false-negative or false-positive rates when models encounter unseen scenarios.
To address these limitations, I propose a hybrid method that integrates YOLO11 with traditional histogram analysis and dual-light (visible and thermal) image fusion. The core idea is to leverage YOLO11’s powerful feature extraction to first localize solar panel strings and individual solar panels, then apply both a deep learning-based dust segmentation and a traditional histogram-based verification. For further validation and severity classification, thermal images are used to detect hot spots and measure temperatures corresponding to dusty regions. This approach effectively reduces the reliance on large annotated datasets while improving detection accuracy under complex conditions.
System Overview and Methodology
Figure 1 illustrates the overall architecture of the proposed dust detection system. The process begins with the acquisition of dual-light images via an unmanned aerial vehicle (UAV) equipped with RTK positioning and a dual-spectrum camera. After preprocessing (such as distortion correction), the visible and thermal images are fed into two parallel YOLO11-OBB (oriented bounding box) modules to extract solar panel strings and individual solar panels.
The visible image path performs dust detection using YOLO11-SEGMENT and traditional histogram analysis. For each solar panel detected with low confidence (below 0.65), the histogram pipeline provides a secondary check. Panels with high histogram scores (above 0.7) are retained as positive dust cases. Additionally, panels not detected by the deep model but with histogram scores above 0.8 are also flagged. The thermal image path detects hot spots, and a coordinate mapping (pixel-to-geographic) links the visible dust bounding boxes with thermal hot spots. Finally, the severity of dust is classified based on the temperature of the corresponding hot spot: mild (40–60 °C) or severe (above 60 °C).
YOLO11-OBB with Improvements
YOLO11, the latest YOLO model from Ultralytics, provides an excellent balance between accuracy and efficiency. For solar panel detection, I employ the oriented bounding box (OBB) variant, which can precisely localize solar panels that are often tilted in UAV images due to pitch or terrain. To further enhance performance, I introduce two key improvements to the YOLO11 architecture: the C3K2-T module incorporating Triple Attention, and the C2PSA-S module integrating SEAM with repulsion loss.
Triple Attention in C3K2
The C3K2 module, though effective, sometimes struggles with detailed features under complex weather. By integrating channel, spatial, and contextual attention, the C3K2-T module improves sensitivity. The final output feature map after Triple Attention is given by:
$$
F_{out} = X \cdot C \cdot S \cdot S_{context}
$$
where $X$ is the input feature, $C$ is the channel attention weight, $S$ is the spatial attention map, and $S_{context}$ is the context attention map. The channel attention weights are computed as:
$$
C = \sigma(W_c \cdot X)
$$
with $\sigma$ as the activation function. Spatial attention uses:
$$
S = \sigma(W_s \cdot X)
$$
Context attention aggregates global information:
$$
S_{context} = \text{Softmax}(W_{context} \cdot X)
$$
SEAM with Repulsion Loss in C2PSA
To handle occlusion and background interference (e.g., shadows on solar panels), I embed the SEAM module into the C2PSA block of YOLO11. SEAM’s loss function consists of three components: classification loss $L_{cls}$, equivariant regularization loss $L_{ER}$, and equivariant cross-regularization loss $L_{ECR}$. The total loss is:
$$
L = L_{cls} + L_{ER} + L_{ECR}
$$
The classification loss is defined as:
$$
L_{cls} = \frac{1}{2}\left( l_{cls}(Z^0, l) + l_{cls}(Z^t, l) \right)
$$
where $Z^0$ and $Z^t$ are predictions from two augmented views, and $l$ is the ground truth label. The equivariant regularization loss measures the $L_1$ distance between the transformed prediction $\bar{y}^0$ and the target $\bar{y}^t$:
$$
L_{ER} = \left\| A \bar{y}^0 – \bar{y}^t \right\|_1
$$
Similarly, the equivariant cross-regularization loss is:
$$
L_{ECR} = \left\| A \bar{y}^0 – \bar{y}^t \right\|_1 + \left\| A \bar{y}^t – \bar{y}^0 \right\|_1
$$
These improvements enable the model to better distinguish dust from shadows and reduce false positives on solar panel surfaces.
Histogram-Based Dust Verification
After the deep learning model detects a solar panel region, a traditional image processing pipeline is applied to estimate dust coverage independently. This serves as a secondary check, especially for low-confidence predictions.
Grayscale Conversion
The RGB region of the solar panel is converted to grayscale using the weighted sum:
$$
I_{gray} = 0.299 \cdot R + 0.587 \cdot G + 0.114 \cdot B
$$
Gaussian Filtering
To remove high-frequency noise while preserving edges, a Gaussian filter is applied:
$$
G(x, y) = \frac{1}{2\pi\sigma^2} e^{-\frac{x^2 + y^2}{2\sigma^2}}
$$
Morphological Operations
Erosion removes small white lines (e.g., grid lines on solar panels), and dilation enhances dust regions:
Erosion: $A \ominus B = \{ z | (B)_z \subseteq A \}$
Dilation: $A \oplus B = \{ z | (\hat{B})_z \cap A \neq \emptyset \}$
Histogram Analysis and Thresholding
After morphological processing, the histogram of the grayscale image is computed and smoothed with a Gaussian kernel. Two dominant peaks are identified in the intensity range [100, 150] (or globally if not found). The minimum intensity between the two peaks is taken as the threshold $T$. Pixels with intensity greater than $T$ (but below 235) are classified as dust; pixels above 235 are solar panel edges and are ignored. The binary image is then cleaned by removing connected components smaller than 8 pixels in either dimension.
The dust coverage ratio $R_{dust}$ is calculated as:
$$
R_{dust} = \frac{\text{number of dust pixels}}{\text{total solar panel pixels}} \times 100\%
$$
This ratio is used to assign a histogram score: a ratio above 30% generally indicates significant dust. In the hybrid pipeline, if the deep model’s confidence is low but the histogram score exceeds 0.7 (i.e., dust ratio > 30%), the panel is taken as positive. If the deep model detects nothing but the histogram score exceeds 0.8, it is also flagged.
Experimental Setup
Dataset
I collected a comprehensive dataset from multiple photovoltaic plants across China, including ground-mounted and hillside installations. The dataset contains about 100,000 dual-light image pairs (visible + thermal). Among these, approximately 5,000 visible images are annotated for dust defects, and about 8,000 thermal images are annotated for hot spots. The data covers various weather conditions (sunny, cloudy, rainy) and illumination angles to ensure diversity. Table 1 summarizes the dataset statistics.
| Category | Number of Images | Annotations |
|---|---|---|
| Visible – solar panel strings | 100,000 | Oriented bounding boxes for panels |
| Visible – dust defects | 5,000 | Instance segmentation masks |
| Thermal – hot spots | 8,000 | Oriented bounding boxes |
Implementation Details
Experiments were conducted on an Ubuntu system with Intel Xeon Gold 5218 CPU, 45 GB RAM, and two NVIDIA Quadro RTX 8000 GPUs (each 48 GB). The framework is PyTorch 1.12.1 with CUDA 10.2 and CuDNN 7.6.5, using Python 3.8. The batch size was set to 16, input image size 640×640, and training lasted 300 epochs with SGD optimizer (learning rate 0.01, momentum 0.937, weight decay 5e-4). Data augmentation included mosaic, mixup, random affine, and HSV jitter.
Evaluation Metrics
I used the following metrics:
- mean Average Precision (mAP) at IoU thresholds 0.5 and 0.5:0.95.
- Precision (P) = TP / (TP + FP)
- Recall (R) = TP / (TP + FN)
- Number of parameters (Param) in millions (M)
- Computational complexity in GFLOPs
Experimental Results and Comparison
Solar Panel Detection
Table 2 compares the proposed method (YOLO11s-OBB with C2PSA-S and C3K2-T) against several baseline detectors on a test set of 2,000 images from unseen solar farms. The proposed method achieves the highest mAP@0.5 of 0.88 and mAP@0.5:0.95 of 0.83, with 15.0M parameters and 21 GFLOPs. This represents a 7.3% improvement over the vanilla YOLO11s-OBB.
| Method | Recall | mAP@0.5 | mAP@0.5:0.95 | Param (M) | GFLOPs |
|---|---|---|---|---|---|
| Faster-RCNN | 0.76 | 0.77 | 0.78 | 370.0 | 220 |
| SSD | 0.73 | 0.76 | 0.72 | 12.3 | 62 |
| YOLOv5s | 0.73 | 0.75 | 0.75 | 7.2 | 17 |
| YOLOv8s | 0.76 | 0.78 | 0.77 | 11.2 | 28 |
| YOLO11s-OBB | 0.80 | 0.82 | 0.79 | 13.5 | 20 |
| Ours | 0.85 | 0.88 | 0.83 | 15.0 | 21 |
Ablation Study
Table 3 shows the contributions of each proposed module. Adding C2PSA-S individually improves mAP@0.5 by 1.2%, C3K2-T provides 2.4% improvement, and their combination yields 7.3% improvement over the baseline. The gains are consistent across all metrics while keeping computational overhead modest.
| Method | C2PSA-S | C3K2-T | Recall | mAP@0.5 | Param (M) | GFLOPs |
|---|---|---|---|---|---|---|
| YOLO11s-OBB | ✗ | ✗ | 0.80 | 0.82 | 13.5 | 20 |
| ① | ✓ | ✗ | 0.81 | 0.83 | 15.5 | 22 |
| ② | ✗ | ✓ | 0.83 | 0.85 | 13.5 | 20 |
| ③ (Ours) | ✓ | ✓ | 0.85 | 0.88 | 15.0 | 21 |
Dust Defect Detection
Because dust data is scarce, I employed a two-stage strategy: first detect the solar panel string, then segment individual panels and run dust detection only on each panel. This reduces background interference. Table 4 compares dust detection performance. The proposed method (YOLO11s-OBB + histogram verification) achieves a recall of 0.62 and mAP@0.5 of 0.65, outperforming all single-stage baselines. The hybrid approach reduces both false positives and false negatives.
| Method | Recall | mAP@0.5 | mAP@0.5:0.95 | Param (M) | GFLOPs |
|---|---|---|---|---|---|
| Faster-RCNN | 0.53 | 0.55 | 0.55 | 370.0 | 220 |
| SSD | 0.51 | 0.52 | 0.52 | 12.3 | 62 |
| YOLOv5s | 0.55 | 0.56 | 0.58 | 7.2 | 17 |
| YOLOv8s | 0.56 | 0.58 | 0.58 | 11.2 | 28 |
| YOLO11s-OBB | 0.56 | 0.59 | 0.61 | 13.5 | 20 |
| Ours (hybrid) | 0.62 | 0.65 | 0.63 | 7.8 | 20 |
Hot-Spot Detection in Thermal Images
Thermal hot-spot detection is critical for dust severity grading. Table 5 shows that the improved YOLO11s-OBB achieves a recall of 0.73 and mAP@0.5 of 0.75, significantly better than other methods. The integration of thermal data allows severity classification: a solar panel with a hot-spot temperature between 40–60 °C is classified as mild dust, above 60 °C as severe dust.
| Method | Recall | mAP@0.5 | mAP@0.5:0.95 | Param (M) | GFLOPs |
|---|---|---|---|---|---|
| Faster-RCNN | 0.64 | 0.61 | 0.59 | 370.0 | 220 |
| SSD | 0.55 | 0.57 | 0.53 | 12.3 | 62 |
| YOLOv5s | 0.66 | 0.62 | 0.57 | 7.2 | 17 |
| YOLOv8s | 0.65 | 0.67 | 0.62 | 11.2 | 28 |
| YOLO11s-OBB | 0.66 | 0.69 | 0.64 | 13.5 | 20 |
| Ours | 0.73 | 0.75 | 0.71 | 7.8 | 20 |
Discussion
The experimental results validate the effectiveness of the proposed hybrid dust detection method for solar panels. By combining YOLO11’s deep learning strength with traditional histogram analysis, I achieved robust detection even with limited annotated dust data. The thermal imaging layer adds an orthogonal signal: dust causes temperature anomalies, which are less dependent on visual appearance. This multi-modal approach significantly reduces false alarms from shadows or dirt-like textures.
One limitation is that the histogram threshold may be influenced by extreme illumination (e.g., overexposed solar panels). However, the dual-check mechanism ensures that only cases with high histogram scores (0.7 or 0.8) are accepted, providing a conservative filter. In future work, I plan to incorporate an adaptive threshold based on local contrast or use a lightweight CNN to estimate dust coverage directly from histogram features.
Conclusion
In this paper, I presented a novel dust detection method for solar panels that synergizes YOLO11-based deep learning with traditional histogram analysis and dual-light thermal imaging. The approach addresses the critical challenge of data scarcity in real-world photovoltaic inspections. Through careful architectural improvements (C3K2-T and C2PSA-S modules) and a three-stage verification pipeline (deep model, histogram, thermal), the system achieves high accuracy in detecting dust and classifying its severity. Extensive experiments on diverse datasets confirm that the proposed method outperforms existing state-of-the-art detectors in both solar panel localization and dust defect recognition. The integration of thermal hot-spot analysis provides a practical severity grading that directly guides cleaning schedules. This work offers a reliable and efficient solution for automated solar panel maintenance, contributing to the sustainable operation of photovoltaic power plants.