Solar energy, as one of the most commercially valuable green energy sources, holds significant competitive advantages in the field of power generation. Photovoltaic modules, commonly referred to as solar panels, are the core components for photoelectric conversion. During operation, the surfaces of these solar panels are prone to dust deposition, which blocks solar radiation and leads to a decline in photoelectric conversion efficiency and output power. Existing dust detection methods based on deep learning algorithms heavily rely on the richness and quality of annotated data. To address this issue, I propose a dust detection algorithm based on YOLO11 combined with histogram analysis and dual-light image fusion. First, I utilize the YOLO11-OBB rotating object detection algorithm to accurately extract photovoltaic strings and individual solar panels from both visible light and thermal imaging images. Second, for each photovoltaic string, I employ the YOLO11-SEGMENT algorithm to detect dust areas in visible light images and to identify hot spots in thermal imaging images, analyzing their temperatures. Finally, I apply traditional image processing methods to analyze visible light images and predict the dust coverage ratio by leveraging histogram features. This method leverages the powerful feature extraction and generalization capabilities of the YOLO11 algorithm, incorporates traditional image processing for secondary verification, and classifies the severity of dust accumulation based on abnormal temperature data from corresponding components in thermal imaging images, significantly improving the accuracy of dust detection in complex scenarios.
With the rapid advancement of solar power technology, solar panels serve as the core components of photovoltaic power generation systems. Their operational efficiency directly determines the overall energy output of the entire system. However, in real-world operation, the surfaces of solar panels are easily covered by contaminants such as dust and dirt. These pollutants block solar radiation, reduce the photoelectric conversion efficiency, and consequently cause a notable decrease in power generation. Studies have shown that when the dust density on solar panels exposed to natural environments reaches $0.64\ \mathrm{g/m^2}$, the output power drops by $14.26\%$. Controlled experiments further reveal that dust accumulation significantly inhibits the energy output of solar panels, and the power generation declines exponentially with increasing dust severity. In severe cases, dust can even trigger hot-spot effects, further damaging the lifespan of solar panels. Therefore, timely detection and cleaning of dust on solar panels are vital for maintaining efficient system operation.
Currently, dust detection methods mainly rely on manual inspection or traditional image processing techniques based on threshold segmentation. Although these approaches can identify dust to some extent, their detection accuracy and efficiency are limited by environmental factors such as lighting conditions and weather changes, making it difficult to adapt to complex and dynamic real-world scenarios. In recent years, deep learning techniques have made remarkable progress in object detection and image segmentation, providing new solutions for dust detection. However, the performance of deep learning models heavily depends on the richness and quality of annotated data. In practical business scenarios, it is challenging to acquire sufficiently diverse and abundant annotated images of dust-covered solar panels. Moreover, the uneven distribution of dust on solar panel surfaces and complex lighting conditions further increase the difficulty of dust detection, leading to issues such as missed detections and false positives in real applications.
To tackle these problems, I propose a dust detection method that integrates deep learning with traditional image processing techniques. By employing the YOLO11 rotating object detection algorithm, I first accurately extract photovoltaic strings and individual solar panels from the entire image. Then, using a limited dataset of dust defect images on solar panels, I train a relatively effective dust defect detection model. However, due to the complexity of photovoltaic station scenes and the limitations in data volume and diversity, the model cannot achieve high generalization ability. Through observation and analysis of the characteristics of dust-covered solar panels versus clean ones, I find that dust tends to accumulate at the bottom edges of solar panels, forming a clear contrast with dust-free areas. Based on this feature, traditional image processing methods can distinguish dust areas from clean areas to a certain extent, but their adaptability is constrained by lighting and weather conditions. Therefore, I combine deep learning with traditional methods to create a dual-verification mechanism: for dust detected by the deep learning model with low confidence, I use image processing for secondary verification to reduce false positives; for solar panels where the model fails to detect any dust, I compute a dust score using image processing, and if the score is sufficiently high, I classify them as dust areas, thereby reducing missed detections. At the same time, for dust components with high scores verified by both deep learning and image processing, I cross-check with the hot-spot temperature from the thermal imaging perspective to further validate and classify the severity of dust accumulation. This effectively meets the dust detection requirements in complex scenes, providing an efficient and practical solution for the cleaning and maintenance of solar panels.
The main contributions of this work are as follows: (1) I propose a dust detection method that combines deep learning with traditional image processing, improving detection accuracy even when the model is trained with insufficient data or has limited generalization ability; (2) By integrating the improved YOLO11 rotating object detection method with image processing techniques, I effectively overcome the dependency of traditional methods on lighting and weather conditions, significantly reducing missed detections and false positives, and enhancing algorithm robustness and adaptability; (3) I leverage the powerful feature extraction and generalization capabilities of YOLO11, combined with traditional image processing for secondary verification to confirm whether solar panels have dust, and then utilize hot-spot temperature data from the thermal imaging perspective to grade the severity of dust accumulation. The effectiveness of the proposed method is validated in real-world photovoltaic inspection scenarios, providing an efficient and practical solution for dust detection on solar panels.
System Overview of YOLO11-OBB Dual-Light Dust Defect Detection
In this section, I describe the overall system workflow that combines deep learning and traditional image processing to detect dusty solar panels, and uses both thermal and visible light images to determine the severity of dust accumulation. The overall architecture includes input images, extraction of strings and components, dust detection via deep learning and traditional methods, hot-spot detection in thermal images, and temperature analysis. This method accurately determines and classifies the dust contamination level on solar panels.
The system processes as follows:
Step S1: A drone equipped with precise RTK positioning and a dual-light camera flies over a photovoltaic station at an appropriate angle and altitude. It captures visible light and thermal images at fixed positions or time intervals, transmits the dual-light data to an AI backend, and performs conventional image preprocessing such as distortion correction.
Step S2: I use an improved YOLO11-OBB algorithm to extract all photovoltaic strings and individual solar panels from the visible light image. Then, within the extracted string images, I detect dust defects.
Step S3: I use the same improved YOLO11-OBB algorithm to extract hot-spot defects from the thermal images.
Step S4: Dust components detected by the deep learning model with confidence greater than 0.65 are retained as positive examples. Dust components with confidence below 0.65 are fed into a traditional detection pipeline that uses histogram processing to determine whether they are dusty; those with a histogram score above 0.7 are retained as positive examples. Additionally, solar panels not detected by the algorithm are also processed by the traditional method, and if the histogram score exceeds 0.8, they are classified as dusty.
Step S5: I match the visible light dust bounding box data to the hot-spot defects in the thermal image via pixel-to-geo-coordinate transformation.
Step S6: By analyzing the dust size in the visible light image and the hot-spot temperature corresponding to the dust in the thermal image, I classify hot spots into three categories: mild, moderate, and severe. According to industry experience, if the maximum hot-spot temperature caused by dust ranges from 40 to 60 °C, it is classified as mild dust accumulation; above 60 °C, it is severe dust accumulation. This classification guides cleaning and maintenance plans, prioritizing heavily dusty solar panels.
YOLO11 Algorithm and Improvements
YOLO11 is the latest YOLO model developed by Ultralytics. It offers significant advantages in solar panel detection, balancing model accuracy and efficiency with notable architectural improvements over previous versions.
YOLO11-OBB Model
Due to inevitable tilting of drone cameras and the terrain of photovoltaic stations, solar panels often appear rotated in images. Traditional axis-aligned bounding boxes introduce background redundancy and degrade detection accuracy. The YOLO11-OBB rotating object detection model, with robust angle prediction, outputs oriented bounding boxes, providing a distinct advantage for solar panel detection. Compared with YOLOv10, YOLO11 reduces redundant computation and improves efficiency by enhancing the PSA module and introducing depthwise separable convolutions. Additionally, residual structures and attention mechanisms optimize network training, boosting overall performance, making it the state-of-the-art for solar panel detection.
Algorithm Improvements
Integration of Triple Attention into C3K2 Module
The C3K2 module performs reasonably well in solar panel detection but has limitations in capturing fine details under complex weather conditions. To improve detection accuracy, I introduce a Triple Attention mechanism into the C3K2 module, forming the C3K2-T module. This module combines channel, spatial, and contextual attention to enhance sensitivity to complex features, improving the accuracy and robustness of solar panel detection.
The Triple Attention weighted feature is formulated as:
$$
F_{out} = X \cdot C \cdot S \cdot S_{context}
$$
Channel attention strengthens key feature channels by adjusting channel weights:
$$
C = \sigma(W_c \cdot X)
$$
Spatial attention weights pixel positions to focus on key regions:
$$
S = \sigma(W_s \cdot X)
$$
Contextual attention captures long-range dependencies by integrating global information:
$$
S_{context} = \text{Softmax}(W_{context} \cdot X)
$$
where $X$ is the input feature map, $\sigma$ is an activation function, and $W$ are learnable weights.
Incorporating Repulsion Loss into C2PSA-S Module
Although YOLO11 performs well in feature extraction and object detection, it is susceptible to false positives and missed detections due to lighting shadows and background clutter, especially when prediction boxes overlap with real targets. The SEAM module enhances inter-channel correlations by optimizing convolutional structures, combining spatial attention and feature enhancement to improve detection under occlusion. I therefore embed the SEAM module into the PSA attention layer of the C2PSA module, forming the C2PSA-S module.
The SEAM loss comprises three components: classification loss $L_{cls}$, equivariant regularization (ER) loss $L_{ER}$, and equivariant cross regularization (ECR) loss $L_{ECR}$. The total loss is:
$$
L = L_{cls} + L_{ER} + L_{ECR}
$$
The classification loss is computed after global average pooling of feature maps:
$$
L_{cls} = \frac{1}{2}(l_{cls}(Z^0, l) + l_{cls}(Z^t, l))
$$
where $Z^0$ and $Z^t$ are two different predictions, and $l$ is the ground truth label.
The equivariant regularization loss measures similarity between the CAM of the original image and the transformed image:
$$
L_{ER} = \left\| A(\hat{y}^0) – \hat{y}^t \right\|_1
$$
The equivariant cross regularization loss further enforces consistency:
$$
L_{ECR} = \left\| A(\hat{y}^0) – y^t \right\|_1 + \left\| A(\hat{y}^t) – y^0 \right\|_1
$$
The improved network architecture is shown schematically (not displayed here, but the modifications are implemented in our code).
Histogram-Based Dust Contamination Assessment
In this section, I detail the image processing pipeline applied after YOLO detects solar panels. This pipeline extracts dust-covered regions on the panel surface and removes noise and interference, providing reliable data for subsequent dust coverage calculation.
Grayscale Conversion
I convert the detected solar panel region image to grayscale, which simplifies subsequent steps while retaining structural information. The grayscale value is computed using the weighted average method:
$$
I_{gray} = 0.299 \times R + 0.587 \times G + 0.114 \times B
$$
where $R$, $G$, and $B$ are the red, green, and blue channel values of the color image.
Gaussian Filtering
To remove noise, I apply Gaussian filtering to the grayscale image. Gaussian filtering is a linear smoothing filter that suppresses high-frequency noise while preserving main edge information. The Gaussian kernel is:
$$
G(x, y) = \frac{1}{2\pi\sigma^2} e^{-\frac{x^2 + y^2}{2\sigma^2}}
$$
where $\sigma$ controls the smoothing degree.
Morphological Operations
After denoising, I use morphological operations. First, erosion removes small interferences such as white stripes on the panel surface. Erosion shrinks bright regions, eliminating small noise points. Then, dilation enlarges fine dust areas, enhancing connectivity. The erosion and dilation operations are expressed as:
$$
\text{Erosion: } A \ominus B = \{ z | (B)_z \subseteq A \}
$$
$$
\text{Dilation: } A \oplus B = \{ z | (\hat{B})_z \cap A \neq \emptyset \}
$$
where $A$ is the input image and $B$ is the structuring element.
Histogram Analysis and Gaussian Smoothing
To determine the gray-level threshold separating dust areas from clean areas, I analyze the histogram of the processed image. I compute the grayscale histogram, then smooth it with a Gaussian filter to highlight major peaks. The steps are:
1. Compute the grayscale histogram of the image.
2. Smooth the histogram with a Gaussian filter.
3. Search for two major peaks in the gray range of 100–150. If two peaks exist, find the minimum gray value between them; otherwise, find the minimum between the two largest peaks globally.
Binarization
Based on the minimum gray value obtained from histogram analysis, I perform threshold-based binarization:
1. Pixels with gray value above 235 are set to 100 (considered white edges of the panel).
2. Pixels below the threshold are set to 100 (considered normal panel area).
3. Pixels above the threshold are set to 200 (considered dust-covered area).
4. Finally, I remove connected regions with fewer than 8 rows or columns to avoid edge expansion effects.
This step clearly separates normal and dust-covered regions, providing an accurate binary image for dust coverage calculation.
Experimental Results and Analysis
Dataset Construction
Detection of photovoltaic strings, individual solar panels, and dust is a crucial part of my proposed method. I conduct multiple comparative experiments to demonstrate the effectiveness of the improved YOLO11 model for solar panel detection, dust detection, and hot-spot detection. The experimental data come from drone vertical-angle images captured at multiple photovoltaic stations in China, including centralized stations and mountain PV stations. The dataset covers various lighting conditions, weather conditions, and dust distributions, ensuring the algorithm’s efficacy and generalizability in complex scenes. The dataset contains approximately 100,000 visible light and thermal images, including about 5,000 real dust images.
Experimental Environment and Configuration
The experiments were conducted on an Ubuntu system with an Intel Xeon Gold 5218 processor, 45 GB of GPU memory (2× Nvidia Quadro RTX 8000), using PyTorch 1.12.1, CUDA 10.2, cuDNN 7.6.5, and Python 3.8.
Evaluation Metrics
I use the following metrics:
- Mean Average Precision (mAP): mAP@0.5 is the average detection precision at IoU threshold 0.5; mAP@0.5:0.95 averages across IoU thresholds from 0.5 to 0.95. Higher mAP indicates better performance.
- Precision (P): $P = \frac{TP}{TP+FP}$, where TP = true positives, FP = false positives.
- Recall (R): $R = \frac{TP}{TP+FN}$, where FN = false negatives.
- Parameters (Param): Total number of trainable parameters, indicating model complexity.
- GFLOPs: Giga floating-point operations per second, measuring computational cost.
Comparison Experiments for Solar Panel Detection
Table 1 shows the comparison between my improved model and several popular algorithms on the same dataset. Tests were performed on data from different PV stations under various weather conditions to evaluate generalization. My method shows significant advantages in detection accuracy, robustness, and adaptability.
| Algorithm | Recall | mAP@0.5 | mAP@0.5:0.95 | Param (M) | GFLOPs |
|---|---|---|---|---|---|
| Faster-RCNN | 0.76 | 0.77 | 0.78 | 370.0 | 220 |
| SSD | 0.73 | 0.76 | 0.72 | 12.3 | 62 |
| YOLOv5s | 0.73 | 0.75 | 0.75 | 7.2 | 17 |
| YOLOv8s | 0.76 | 0.78 | 0.77 | 11.2 | 28 |
| YOLO11s-OBB | 0.80 | 0.82 | 0.79 | 13.5 | 20 |
| Ours | 0.85 | 0.88 | 0.83 | 7.8 | 20 |
Table 2 presents the ablation study. The combinations (C2PSA-S, C3K2-T) improve mAP@0.5 by 1.2%, 2.4%, and 7.3% respectively over the baseline YOLO11s-OBB. The full model achieves the highest mAP@0.5 of 0.88 with minimal increase in computational load, demonstrating the superiority of the proposed improvements.
| Method | C2PSA-S | C3K2-T | Recall | mAP@0.5 | Param (M) | GFLOPs |
|---|---|---|---|---|---|---|
| YOLO11s-OBB | ✗ | ✗ | 0.80 | 0.82 | 13.5 | 20 |
| ① | ✓ | ✗ | 0.81 | 0.83 | 15.5 | 22 |
| ② | ✗ | ✓ | 0.83 | 0.85 | 13.5 | 20 |
| ③ (Ours) | ✓ | ✓ | 0.85 | 0.88 | 15.0 | 21 |
Dust Detection Comparison Experiments
Due to scarcity of annotated dust data on solar panels, traditional single-component dust detection methods struggle with accuracy. I adopt a strategy: first detect the entire string using the improved YOLO11s-OBB model, then segment each solar panel, and finally perform dust detection on each segmented component. This reduces background interference and improves detection performance. Table 3 shows the dust detection comparison results.
| Algorithm | Recall | mAP@0.5 | mAP@0.5:0.95 | Param (M) | GFLOPs |
|---|---|---|---|---|---|
| Faster-RCNN | 0.53 | 0.55 | 0.55 | 370.0 | 220 |
| SSD | 0.51 | 0.52 | 0.52 | 12.3 | 62 |
| YOLOv5s | 0.55 | 0.56 | 0.58 | 7.2 | 17 |
| YOLOv8s | 0.56 | 0.58 | 0.58 | 11.2 | 28 |
| YOLO11s-OBB | 0.56 | 0.59 | 0.61 | 13.5 | 20 |
| Ours | 0.62 | 0.65 | 0.63 | 7.8 | 20 |
Hot-Spot Detection Comparison Experiments
Hot spots on solar panels often result from dust accumulation, causing local overheating that can reduce performance or even pose fire risks. Accurate hot-spot detection and correlation with visible-light dust areas are critical. I use the improved YOLO11s-OBB model for hot-spot detection in thermal images, then match these hot spots with visible-light dust regions via pixel-to-geo-coordinate transformation. Finally, temperature data from hot spots are used to grade dust severity. Table 4 shows the hot-spot detection comparison results.
| Algorithm | Recall | mAP@0.5 | mAP@0.5:0.95 | Param (M) | GFLOPs |
|---|---|---|---|---|---|
| Faster-RCNN | 0.64 | 0.61 | 0.59 | 370.0 | 220 |
| SSD | 0.55 | 0.57 | 0.53 | 12.3 | 62 |
| YOLOv5s | 0.66 | 0.62 | 0.57 | 7.2 | 17 |
| YOLOv8s | 0.65 | 0.67 | 0.62 | 11.2 | 28 |
| YOLO11s-OBB | 0.66 | 0.69 | 0.64 | 13.5 | 20 |
| Ours | 0.73 | 0.75 | 0.71 | 7.8 | 20 |
Comprehensive Performance Analysis
Based on experiments conducted on self-collected multi-scenario data, the improved YOLO11s-OBB model combined with histogram equalization significantly enhances rotating object detection performance. It resolves issues such as false positives caused by regular arrangement of solar panels and the difficulty of distinguishing dust from similar-texture backgrounds using traditional segmentation methods, providing a reliable technical path for high-precision industrial surface defect detection.
The dust detection results from the improved model and the histogram analysis are summarized. The histogram analysis outputs a dust coverage percentage, which, together with the hot-spot temperature, enables severity classification.
Conclusion
I proposed a dust detection method for solar panels that combines YOLO11 object detection with histogram analysis. By integrating deep learning and traditional image processing, the method achieves zero-shot high-precision dust coverage estimation without requiring surface dust annotated data. Experimental results demonstrate that this method effectively overcomes the dependence of traditional approaches on lighting and weather conditions, significantly reduces missed detections and false positives, and exhibits high detection accuracy and robustness in real photovoltaic inspection scenarios. This research provides an efficient and practical solution for dust detection on solar panels, with important engineering application value. Future work may focus on further improving model generalization and exploring dynamic dust monitoring and intelligent cleaning systems to support efficient operation and sustainable development of photovoltaic power generation systems.