During the inspection of photovoltaic stations, solar panels installed outdoors inevitably face issues of dust accumulation and hot spots. The dust problem leads to reduced power generation efficiency and poor power quality, while the new photovoltaic panels can experience acceleration in aging due to local heat accumulation. Traditional image processing algorithms commonly employed in intelligent operation and maintenance systems suffer from low recognition accuracy, poor illumination robustness, and slow recognition speed. This dissertation addresses the recognition of dust accumulation and hot spots on solar panels by proposing specific algorithmic solutions.
For solar panel dust accumulation, a traditional image processing algorithm is applied to identify the dust state. An adaptive image segmentation recognition algorithm is proposed. According to the spatial characteristics of dust and solar panel background in the YCbCr color space, dust regions are separated from the panel background. The globally invariant unified threshold is replaced with a local threshold adaptively adjusted by local information to accommodate uneven dust distribution. Texture features of the segmented image are extracted, and the feature vector is used to recognize the dust density. Experimental analysis shows that the algorithm accuracy reaches 88.56%, with fast computing speed, simple structure, and strong real-time performance.
Although Traditional image recognition algorithms own advantages in computing speed, their accuracy is limited relative to deep learning methods. Therefore, an improved dense convolutional neural network is proposed. For the small-sample problem, a parameter-free attention mechanism is introduced into the network to enhance feature extraction capability. Because the original convolution is too sensitive to illumination, leading to degraded model performance under uneven lighting conditions, the original convolution is replaced with central difference convolution, which enhances illumination robustness and improves generalization ability. Experiments indicate that the improved network model achieves 95.01% recognition accuracy on a small sample dataset, with fast convergence and strong generalization capacity.
For the hot spot recognition challenge, where small hot spot targets cause poor detection precision, the SSD network algorithm is proposed. SSD networks possess powerful feature extraction abilities and perform well in small target detection. Experiments are conducted on hot spots with different target sizes to validate the feasibility of the SSD algorithm. Results show that the SSD algorithm maintains a good recognition rate for small targets and can fully identify hot spots along with their corresponding positions.
1. Introduction and Background
Energy scarcity has become a major obstacle to technological progress. Conventional fossil fuels are finite and generate significant environmental pollution. Renewable energy sources, especially solar energy, are increasingly important. By the end of 2021, the national statistical data indicated that the installed capacity of photovoltaic power grew by 20.9% year-on-year, representing the fastest growth among all forms of power generation. The photovoltaic (PV) industry has assumed strategic importance in China’s energy structure, particularly in poverty alleviation and carbon neutrality goals.
Solar power stations are typically established in regions with long sunshine duration. PV panels are exposed to harsh outdoor environments, causing gradual deterioration. Dust deposition is the most common issue. Beyond lowering power output, dust accelerates panel degradation and triggers severe faults such as hot spots. Hot spots produce sufficient heat to damage packaging materials and even create safety hazards. Therefore, a robust inspection method is essential for maintaining solar panels.
Traditional inspection involves manual patrols, which are inefficient, potentially unsafe, and error-prone. Intelligent operation and maintenance (O&M) systems employ unmanned aerial vehicles (UAVs) and image processing algorithms. However, existing image processing techniques, when applied to solar panel fault diagnosis, face several challenges: low recognition accuracy, poor illumination robustness, slow processing, and small target detection difficulties. This dissertation investigates the specific characteristics of dust accumulation and hot spot faults on solar panels and presents targeted recognition algorithms.
1.1 Literature Review
Research on solar panel dust accumulated through multiple phases. Early investigations established physical models linking dust thickness to power loss. Klugmann-Radziemska demonstrated that dust thickness correlates linearly with decreased power generation. Hegazy discovered that installation tilt angle affects dust accumulation and established a linear relationship between dust and transmittance. However, these physical models are limited by interference from multiple environmental factors, making accurate dust assessment difficult.
With the advancement of image processing, computer vision techniques gained widespread adoption. Kamal Adel Abuqaaud applied gray-level co-occurrence matrices (GLCM) to extract textural features from dust images and utilized classifiers for detection. Tsanakas employed Canny edge detection on infrared thermal images to identify hot spot regions. Yu Ruei Tzeng combined line detection and edge detection to locate PV modules and hot spots within them. Chinese researchers also contributed extensively. Zhang Yubo proposed a desktop dust detection algorithm using gray-gradient co-occurrence matrices. Zhang Shuwei categorized dust density into five levels by analyzing gray values after preprocessing. Wang Peizhen studied infrared array image characteristics and feature extraction methods. Sheng Xiaochun applied eight-direction Sobel edge detection and SVM classifiers for fault diagnosis.
As deep learning matured, convolutional neural networks (CNNs) were increasingly applied in industrial inspection. Zhao Bo applied ResNet to recognize eleven dust states on solar panels, achieving 84% accuracy. Wei Shuquan combined Faster-RCNN with transfer learning for hot spot detection. Xiao Yuhong proposed a small kernel dense convolutional neural network for small-sample industrial datasets. Zitong Yu introduced central difference convolution to enhance illumination invariance in feature extraction.
The literature reveals several persisting challenges: (1) data scarcity in industrial settings, (2) inadequate illumination robustness, (3) low accuracy in traditional methods, and (4) difficulty in detecting small targets. This dissertation directly addresses these issues through three research thrusts: an adaptive segmentation algorithm for dust recognition, an improved dense network with attention and central difference convolution, and an SSD-based hot spot detector.
2. Intelligent Inspection of Photovoltaic Stations
2.1 Structure of Photovoltaic Stations
A photovoltaic station primarily consists of PV panels, inverters, combiner boxes, and monitoring systems. Solar panels convert sunlight directly into electricity through the photovoltaic effect. When photons strike the semiconductor material, electron-hole pairs are generated at the P-N junction, creating a voltage difference. The photoelectric effect can be represented as follows.
$$\begin{equation} V_{oc} = \frac{kT}{q}\ln\left(\frac{I_L}{I_0} + 1\right) \tag{1} \end{equation}$$
where \(V_{oc}\) is the open-circuit voltage, \(k\) is Boltzmann’s constant, \(T\) is temperature, \(q\) is the elementary charge, \(I_L\) is the light-generated current, and \(I_0\) is the saturation current. The arrangement of PV panels in series forms strings, and multiple strings connect to combiner boxes, then to inverters, which feed the grid. The structure of centralized PV stations differs from distributed ones in scale, component configuration, and deployment preference.
Centralized stations often adopt polycrystalline panels due to economic considerations, while distributed small-capacity systems are more likely to use monocrystalline panels. Recent advances have increased the market share of monocrystalline panels even in large installations, owing to their superior efficiency and lower degradation rates.
2.2 Intelligent Inspection Modes
Traditional manual inspection suffers from several disadvantages: high labor costs, safety risks associated with high-voltage equipment, inconsistent fault detection, and incomplete assessment of equipment conditions. In contrast, intelligent O&M leverages embedded high-speed processors, wireless data transmission, and GPS positioning to enable automated UAV-based patrols.
UAV inspection workflow consists of three primary stages. First, electrical parameter acquisition and analysis determine approximate fault locations across the array. Second, UAV cruises over the identified area and captures RGB and thermal images. Third, images are transmitted to ground stations, where intelligent algorithms recognize and classify faults, providing precise localization and decision recommendations. The integration of UAV platforms with ground stations enables both online and offline processing, supported by user-friendly interfaces that display flight status, trajectory, speed, and battery level.
UAVs carry both RGB and infrared cameras. RGB cameras detect visible faults such as dust, structural damage, and wiring issues. Infrared cameras capture thermal information, revealing temperature anomalies indicative of internal defects. Hot spots appear as bright regions in infrared images, easily visible to trained algorithms. Despite the advantages, UAV inspection faces environmental challenges. Terrain complexity, weather conditions, uneven illumination, and glass reflections complicate image acquisition and analysis. These factors necessitate robust algorithms capable of handling illumination variation, noise, and geometric distortion.
2.3 Complexity Factors Affecting Solar Panels
Outdoor PV panels accumulate dust with different morphologies: loose dry powder, point-like deposition, and saline-like crust. These forms arise from aging dust that has been subjected to rain and condensation. The physical characteristics of dust significantly influence its adhesion and optical properties. Dust consists of inorganic particles, predominantly silica, alumina, and calcium oxide, varying in size from micrometers to hundreds of micrometers. These particles scatter, absorb, and reflect light, reducing the intensity reaching the photovoltaic cells.
Hot spot formation occurs when a cell within a module becomes reverse-biased and consumes power generated by other cells. This phenomenon is prevalent when partial shading occurs, such as from accumulated dirt, bird droppings, fallen leaves, or structural shadowing. The affected cell heats up, potentially exceeding safe operating temperatures. The heat can degrade the encapsulation and lead to permanent module damage or fire hazards.
In the context of intelligent inspection, the illumination condition is one of the most critical factors. Images captured outdoors under different solar elevation angles, cloud coverage, and weather conditions display significant brightness variations. Infrared images are also affected by emissivity differences and reflection from glass surfaces. These factors hinder fault detection algorithms and call for preprocessing and algorithmic strategies that are robust to illumination changes.
3. Dust Recognition via Image Segmentation
3.1 Dust Characteristics and Color Analysis
Dust is a complex mixture of particulates with sizes in the micrometer range. In the laboratory, dust samples are collected and sieved using a standard 200-mesh sieve to select particles below 100 μm. Particle size distribution varies across urban districts. Table 1 presents the particle size distribution for different districts of Urumqi.
| District | 2.5–5 μm | 5–10 μm | 10–50 μm | 50–100 μm | >100 μm |
|---|---|---|---|---|---|
| Xinshi | 4.08 | 6.42 | 50.82 | 16.05 | 22.63 |
| Midong | 4.39 | 6.97 | 52.16 | 17.58 | 20.90 |
| Tianshan | 3.79 | 6.07 | 50.65 | 15.95 | 23.54 |
The dust particles exhibit a certain degree of opacity and color, typically appearing grayish-yellow. When deposited on solar panels, dust particles create a distinct color contrast against the dark, often bluish-black, surface of monocrystalline panels. This color distinction forms the basis for the segmentation algorithm.
Images are acquired in RGB format. However, RGB space combines color and intensity information, making it difficult to effectively separate regions based solely on color differences. The YCbCr color space separates luminance (Y) from chrominance (Cb and Cr). This separation facilitates more robust segmentation based on color properties while maintaining the ability to account for intensity variations. For a color image \(I(x,y)\) with RGB components, the YCbCr conversion is given by:
$$\begin{equation} \begin{bmatrix} Y \\ Cb \\ Cr \end{bmatrix} = \begin{bmatrix} 0.299 & 0.587 & 0.114 \\ -0.169 & -0.331 & 0.500 \\ 0.500 & -0.419 & -0.081 \end{bmatrix} \begin{bmatrix} R \\ G \\ B \end{bmatrix} + \begin{bmatrix} 16 \\ 128 \\ 128 \end{bmatrix} \tag{2} \end{equation}$$
In the YCbCr space, the dust regions observed in the Y channel exhibit higher brightness values than the non-dust regions. For the Cb channel, dust regions show lower values than the background average. For the Cr channel, dust regions show higher values. To segment the dust regions effectively, a global thresholding rule is initially used as follows.
$$\begin{equation} R(x,y) = \begin{cases} I(x,y) & \text{if } Y(x,y) > Y_{mean} \quad \text{and} \quad Cb(x,y) < Cb_{mean} \\ 0 & \text{otherwise} \end{cases} \tag{3} \end{equation}$$
where \(R(x,y)\) represents the segmented output pixel value, \(I(x,y)\) is the original RGB value, \(Y_{mean}\) is the mean Y value, and \(Cb_{mean}\) is the mean Cb value. This global approach works when dust distribution is uniform. However, in practice, dust often accumulates unevenly, resulting in local variations in color characteristics. A purely global threshold may misclassify regions with lower dust density as background.
3.2 Adaptive Segmentation Algorithm
To overcome the limitations of global thresholding, an adaptive segmentation algorithm is developed. The method computes the threshold for each row based on the previous row’s statistical properties. The first row uses the global mean as an initial threshold. Subsequent thresholds are updated using the following relation.
$$\begin{equation} Cb_{i,mean} = Cb_{mean} + \frac{1}{M}\sum_{j} Cb(i-1,j) \tag{4} \end{equation}$$
where \(Cb_{i,mean}\) is the threshold for the \(i\)-th row, \(M\) is the number of pixels in the \((i-1)\)-th row that satisfy the segmentation rule, and \(Cb(i-1,j)\) represents the Cb value at position \(j\) of the \((i-1)\)-th row satisfying the threshold condition. The first threshold is set to the global mean: \(Cb_{1,mean} = Cb_{mean}\). The complete adaptive segmentation rule is given by:
$$\begin{equation} R(x,y) = \begin{cases} I(x,y) & \text{if } Y(x,y) > Y_{mean} \quad \text{and} \quad Cb(x,y) < Cb_{i,mean} \\ 0 & \text{otherwise} \end{cases} \tag{5} \end{equation}$$
This row-wise adaptive thresholding allows the algorithm to adjust to varying dust density across the image. For the first test dataset, the global threshold yields high accuracy for uniformly dusted panels. However, for non-uniform dust, the adaptive method improves segmentation quality. Figure 3-4 (referenced in the original) illustrates the segmentation results, demonstrating that non-dust images remain largely black with thin white lines corresponding to metallic interconnects on the panel surface. In dusty images, the adaptive algorithm effectively isolates the dust while suppressing the white metallic lines.
3.3 Texture Feature Extraction with Local Binary Patterns
After segmentation, the image consists of dust regions with distinct textural patterns and a black background. Texture features are effective for distinguishing different dust densities. The Local Binary Pattern (LBP) algorithm, widely used for texture classification, exhibits good illumination invariance and computational simplicity. The basic LBP operator for a pixel \(c\) with neighborhood \(P\) at radius \(R\) is defined as:
$$\begin{equation} LBP_{P,R} = \sum_{i=0}^{P-1} s(g_i – g_c) 2^i \tag{6} \end{equation}$$
where \(g_c\) is the center pixel value, \(g_i\) are the neighboring pixel values, and
$$\begin{equation} s(x) = \begin{cases} 1 & x \geq 0 \\ 0 & \text{otherwise} \end{cases} \tag{7} \end{equation}$$
The LBP value is computed by comparing the center pixel with its neighbors and constructing a binary sequence. To achieve rotation invariance, the rotation-invariant LBP is defined as:
$$\begin{equation} LBP_{P,R}^{ri} = \min \{ ROR(LBP_{P,R}, i) | i = 0, 1, \ldots, P-1 \} \tag{8} \end{equation}$$
where \(ROR(x, i)\) denotes the circular right-rotation of \(x\) by \(i\) positions. To reduce feature dimensionality while preserving discriminative power, the rotation-invariant uniform pattern is preferred. This pattern counts transitions between 0 and 1 in the binary sequence:
$$\begin{equation} LBP_{P,R}^{riu2} = \begin{cases} \sum_{i=0}^{P-1} s(g_i – g_c) & \text{if } U(LBP_{P,R}) \leq 2 \\ P + 1 & \text{otherwise} \end{cases} \tag{9} \end{equation}$$
where the uniformity measure \(U\) is:
$$\begin{equation} U(LBP_{P,R}) = |s(g_{P-1} – g_c) – s(g_0 – g_c)| + \sum_{i=1}^{P-1} |s(g_i – g_c) – s(g_{i-1} – g_c)| \tag{10} \end{equation}$$
This formulation reduces the LBP encoding from 256 possible values to only 10 distinct patterns, significantly reducing computational cost while preserving descriptive capability.
To further improve the LBP performance, the image is divided into non-overlapping blocks. For each block, the LBP histogram is computed, and the histograms from all blocks are concatenated to form the final feature vector. This block-based approach preserves spatial information while maintaining the robustness of LBP features. In this research, the rotation-invariant uniform LBP with \(R=1\) and \(P=8\) yields the best balance between accuracy and computational efficiency.
3.4 Dataset Preparation and Experimental Results
The dataset was constructed by simulating natural dust deposition on a monocrystalline silicon solar panel with dimensions 1580 mm × 810 mm and a rated output of 200 W. Dust was collected outdoors, sieved below 100 μm, and weighed using a precision electronic balance. Dust was spread uniformly over the panel with the assistance of a humidifier to enhance adhesion. Five dust density levels were selected: 0, 5, 10, 15, and 20 g/m². The choice was based on the observation that power output drops by 30% at 30 g/m², and cleaning is recommended before reaching that level. RGB images were captured for each density level, totaling 2000 images, resized to 300 × 400 pixels. 1400 images served as training samples and 600 as testing samples.
Experiments were performed on a Windows platform with an Intel i7-10750 processor, 8 GB RAM, and NVIDIA GeForce RTX 2060 GPU. The algorithm was implemented in MATLAB 2018b and Python 3.6. The LBP parameters were optimized by comparing different radii and modes. The results are summarized in Table 2.
| LBP Mode | Feature Dimension | Accuracy (%) | Time (s) |
|---|---|---|---|
| Rotation-invariant (R=1,P=8) | 324 | 74.23 | 70.73 |
| Rotation-invariant (R=2,P=8) | 324 | 83.56 | 62.92 |
| Rotation-invariant uniform (R=1,P=8) | 90 | 88.56 | 70.38 |
| Rotation-invariant uniform (R=2,P=8) | 90 | 82.61 | 61.33 |
The best accuracy of 88.56% was achieved with the rotation-invariant uniform mode at \(R=1\), \(P=8\). This configuration retains the smallest feature dimension while providing the highest accuracy. The feature histograms for different dust densities display clear separation, particularly at small feature values, where the number of pixels increases with dust density. At feature value 9, the trend reverses due to the interference from segmented background structures, but the pattern is restored at feature value 10. This discrimination capability enables the classifier to accurately identify the dust state.
The proposed algorithm was compared with GLCM and ResNet50 in terms of accuracy and running time. The results are presented in Table 3.
| Algorithm | Accuracy (%) | Recognition Time (s) |
|---|---|---|
| Proposed method | 88.56 | 70.38 |
| GLCM | 86.73 | 138.09 |
| ResNet50 | 82.53 | 615.23 |
The results confirm that the adaptive segmentation algorithm achieves higher accuracy than both GLCM and ResNet50 in the tested dataset. Moreover, the recognition time is significantly shorter than GLCM and dramatically faster than ResNet50. This efficiency is particularly important for real-time monitoring applications, where processing speed is critical. The algorithm simplicity and low computational cost make it a practical choice for photovoltaic station deployment, especially in resource-constrained edge devices.
However, at 88.56%, the accuracy remains insufficient for fully autonomous O&M systems. The primary cause is the shallow feature extraction capability of LBP, which lacks the hierarchical semantic understanding that deep learning networks provide. Consequently, the next chapter explores deep learning approaches to improve recognition accuracy.
4. Deep Learning for Dust State Recognition
4.1 Fundamentals of Convolutional Neural Networks
Convolutional neural networks (CNNs) represent a dominant class of deep learning architectures for image analysis. A CNN typically consists of an input layer, convolution layers, pooling layers, activation functions, fully connected layers, and an output classifier. The convolution operation extracts local features by sliding a kernel over the input feature map:
$$\begin{equation} y(p_0) = \sum_{p_n \in \mathcal{R}} w(p_n) \cdot x(p_0 + p_n) \tag{11} \end{equation}$$
where \(x\) and \(y\) are input and output feature maps, \(w\) is the kernel weight, \(p_0\) is the current position, and \(\mathcal{R}\) denotes the receptive field, typically a 3×3, 5×5, or 7×7 region.
Pooling layers reduce spatial dimensions and provide translation invariance. Commonly used pooling operations include max pooling, average pooling, and overlapping pooling. Max pooling selects the maximum value within a local window, reducing redundant information and simplifying calculations. The architecture of a standard CNN includes multiple convolution and pooling blocks, followed by one or more fully connected layers. The output layer typically employs a Softmax activation to produce probability distributions over classes.
Training a CNN involves optimizing a loss function, commonly the cross-entropy loss:
$$\begin{equation} L(x, y) = -\sum_i y_i \log f(x_i, w) \tag{12} \end{equation}$$
where \(y_i\) is the true label for the \(i\)-th sample, and \(f(x_i, w)\) is the predicted probability. Gradient-based methods, particularly the Adaptive Moment Estimation (Adam) optimizer, are widely adopted due to their efficiency and stability.
Adam combines the first-order and second-order moments of gradients. The update rule is:
$$\begin{equation} m_t = \beta_1 m_{t-1} + (1-\beta_1)\nabla_\theta f(\theta_t) \tag{13} \end{equation}$$
$$\begin{equation} v_t = \beta_2 v_{t-1} + (1-\beta_2)(\nabla_\theta f(\theta_t))^2 \tag{14} \end{equation}$$
$$\begin{equation} \hat{m}_t = \frac{m_t}{1-\beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1-\beta_2^t} \tag{15} \end{equation}$$
$$\begin{equation} \theta_{t+1} = \theta_t – \frac{\alpha \hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon} \tag{16} \end{equation}$$
where \(\alpha\) is the learning rate, \(\beta_1\) and \(\beta_2\) are the exponential decay rates, and \(\epsilon\) is a small constant preventing division by zero.
4.2 Densely Connected Convolutional Networks
DenseNet introduces dense connectivity patterns to enhance feature propagation and reuse. In a DenseNet block, each layer receives the feature maps of all preceding layers. The output of the \(l\)-th layer in the \(k\)-th dense block can be expressed as:
$$\begin{equation} x_{l}^{k} = H_k([x_0^k, x_1^k, \ldots, x_{l-1}^k]) \tag{17} \end{equation}$$
where \(H_k(\cdot)\) is a composite function including batch normalization, ReLU activation, and convolution operations. This dense connectivity enables direct gradient flow from the loss function to each layer, mitigating the vanishing gradient problem. DenseNet reduces the number of parameters because each layer has a small growth rate \(k\) (e.g., \(k=32\)), representing the number of feature maps generated per layer. The dense block structure with bottleneck layers (1×1 convolution followed by 3×3 convolution) further reduces computational cost. Transition layers between dense blocks adjust feature map sizes.
This research employs DenseNet-121, which comprises four dense blocks with 6, 12, 24, and 16 layer groups, respectively. The overall architecture includes an initial convolutional layer, a max pooling layer, four dense blocks separated by transition layers, a global average pooling layer, and a fully connected layer with Softmax activation for classification. DenseNet’s narrow structure and feature reuse properties make it particularly suitable for small-sample datasets, which are common in industrial domains.
4.3 Parameter-Free Attention Mechanism
Attention mechanisms enhance neural networks by allowing the model to focus on informative features while suppressing irrelevant ones. The parameter-free attention mechanism, denoted SimAM, is inspired by neuroscience findings about neural activation patterns. It computes a 3-D attention weight map without introducing additional parameters.
For a given neuron, the energy function is defined as:
$$\begin{equation} e(w_t, b_t, y, x_i) = (y_t – \hat{t})^2 + \frac{1}{M-1}\sum_{i=1}^{M-1} (y_o – \hat{x}_i)^2 \tag{18} \end{equation}$$
where \(\hat{t} = w_t t + b_t\) and \(\hat{x}_i = w_t x_i + b_t\) are transformed inputs, \(t\) is the target neuron, \(x_i\) are other neurons in the same channel, and \(M = H \times W\) is the number of neurons per channel. By applying binary labels and regularization, the energy function becomes:
$$\begin{equation} e(w_t, b_t, y, x_i) = \frac{1}{M-1}\sum_{i=1}^{M-1} (-1 – (w_t x_i + b_t))^2 + (1 – (w_t t + b_t))^2 + \lambda w_t^2 \tag{19} \end{equation}$$
Minimizing this energy function yields the analytic solution for \(w_t\) and \(b_t\):
$$\begin{equation} w_t = -\frac{2(t – \mu_t)}{(t – \mu_t)^2 + 2\sigma_t^2 + 2\lambda} \tag{20} \end{equation}$$
$$\begin{equation} b_t = -\frac{1}{2}(t + \mu_t) w_t \tag{21} \end{equation}$$
where \(\mu_t\) and \(\sigma_t^2\) are the mean and variance of all neurons except \(t\). The minimum energy is:
$$\begin{equation} e_t^* = \frac{4(\sigma^2 + \lambda)}{(t – \mu)^2 + 2\sigma^2 + 2\lambda} \tag{22} \end{equation}$$
Lower energy values indicate that a neuron is more distinct from its surrounding neurons and thus more informative. The attention weight map is computed as:
$$\begin{equation} \tilde{X} = \text{sigmoid}\left(\frac{1}{E}\right) \odot X \tag{23} \end{equation}$$
where \(E\) is the energy matrix collecting all \(e_t^*\) values, and \(\odot\) denotes element-wise multiplication.
This parameter-free attention mechanism was integrated into each dense block of DenseNet-121. After obtaining the feature maps from the convolutional layers, the energy function is calculated to derive the weight matrix, which is then reapplied to the feature maps to strengthen important features before passing to the next dense block.
4.4 Central Difference Convolution
Standard convolution operates as a simple weighted sum of neighboring pixels, making it sensitive to illumination changes. To enhance illumination robustness, central difference convolution (CDC) combines the ideas of LBP and conventional convolution. CDC includes both a conventional aggregation term and a gradient term:
$$\begin{equation} y(p_0) = \sum_{p_n \in \mathcal{R}} w(p_n) \cdot \left[ (1-\theta) \cdot x(p_0+p_n) + \theta \cdot \left( x(p_0+p_n) – x(p_0) \right) \right] \tag{24} \end{equation}$$
where \(\theta\) is a hyperparameter controlling the contribution of the gradient component. When \(\theta=0\), CDC reduces to standard convolution. When \(\theta\) is larger, the gradient information becomes more dominant, providing stronger illumination invariance. In this work, \(\theta\) was set to 0.3, experienced to balance between robustness and discriminative capacity.
By incorporating both intensity and gradient information, CDC enables the network to learn more detailed and illumination-robust features. This property is essential for outdoor solar panel images, where lighting conditions vary substantially across different times of day and weather conditions.
4.5 Proposed Network Architecture
The improved network model combines DenseNet-121, SimAM attention, and CDC. All convolutional layers in the dense blocks are replaced with central difference convolutions. The SimAM attention module is inserted after each dense block. The overall architecture is depicted in the original Figure 4-12. The dense blocks use bottleneck structures, each containing a 1×1 convolution followed by a 3×3 CDC. The growth rate is set to 32, and the compression factor following transition layers is 0.5.
The network hyperparameters are listed as follows:
| Parameter | Value |
|---|---|
| Input image size | 300 × 400 × 3 |
| Initial learning rate | 0.1 |
| Batch size | 32 |
| Number of epochs | 100 |
| Dense blocks layer groups | 6, 12, 24, 16 |
| Growth rate \(k\) | 32 |
| \(\theta\) in CDC | 0.3 |
| \(\lambda\) in SimAM | 10⁻⁴ |
4.6 Data Augmentation and Experimental Results
Since the original dataset contained only 2000 images, data augmentation was necessary to prevent overfitting. Geometric transformations (horizontal and vertical flips, rotations), noise injection (salt-and-pepper noise), and brightness/contrast adjustments were applied. Color-space transformations were avoided because color information is critical for dust segmentation. The augmentation increased the dataset tenfold, yielding 20,000 images. 14,000 for training and 6,000 for testing.
The improved model was compared with the original DenseNet-121 on the augmented dataset. The comparison is presented in Table 5.
| Model | Params | Flops | Accuracy | Loss | EER | TPR |
|---|---|---|---|---|---|---|
| DenseNet-121 | 7.351M | 2.524G | 0.9273 | 0.1082 | 1.0115 | 0.6424 |
| Improved model | 7.351M | 2.524G | 0.9501 | 0.0974 | 0.9057 | 0.8156 |
The improved model achieves 95.01% accuracy, a 2.28% improvement over the baseline DenseNet-121. Significantly, the parameter count and FLOPs remain unchanged, confirming that the attention and CDC enhancements do not introduce additional computational burden. The equal error rate (EER) decreases from 1.0115 to 0.9057, and the true positive rate (TPR) increases from 0.6424 to 0.8156, demonstrating better classification performance.
The improved model was also compared with VGG19 and ResNet50. The accuracy curves and loss curves across epochs are presented in Figures 4-14 and 4-15 (referenced in the original). The improved model achieves higher accuracy and faster convergence. The ROC curves, shown in Figure 4-16, further confirm the superiority of the proposed model, as its curve lies closer to the (0,1) corner.
These results demonstrate that combining dense connectivity, central difference convolution, and parameter-free attention provides an effective approach for small-sample photovoltaic dust recognition. The model converges rapidly, generalizes well, and maintains high accuracy under uneven illumination conditions.
5. Hot Spot Detection on Solar Panels
5.1 Hot Spot Formation and Traditional Detection
Hot spots occur when a photovoltaic cell in a module becomes reverse-biased and dissipates power as heat. This typically happens when a cell is partially shaded or damaged. The affected cell acts as a load, consuming current generated by other cells in the series string. The dissipated power is converted into heat, raising the cell temperature dangerously. Commercial photovoltaic cells have a maximum power dissipation limit, typically 25 W per cell. Exceeding this limit causes thermal damage to the cell and its encapsulation.
Hot spots are invisible to the naked eye until extensive damage occurs. Infrared thermography is the most effective method for detecting hot spots, as the abnormal temperatures appear as bright regions in infrared images. UAV-based infrared inspection has been widely adopted. However, when the UAV flies at high altitudes, the acquired images contain many photovoltaic panels, and hot spot targets become extremely small, challenging traditional algorithms.
Traditional hot spot detection methods include template matching and edge detection. Template matching compares the target image with a standard template using similarity measures:
$$\begin{equation} R(x,y) = \sum_{x’,y’} \left[ T(x’, y’) – I(x+x’, y+y’) \right]^2 \tag{25} \end{equation}$$
Template matching is sensitive to image deformations and target size variations, which are common in UAV inspection. High altitude results in small target sizes, significantly degrading matching performance.
Edge detection, particularly Canny edge detection, is another traditional approach. The Canny operator applies Gaussian filtering, gradient computation, non-maximum suppression, and double-threshold detection. The gradient magnitude and direction are:
$$\begin{equation} G = \sqrt{G_x^2 + G_y^2}, \quad \theta = \arctan\left(\frac{G_y}{G_x}\right) \tag{26} \end{equation}$$
Canny edge detection provides cleaner edges compared to Sobel, as demonstrated in the original Figure 5-4. However, edge-based methods suffer from false detections caused by panel boundaries, illumination gradients, and reflections. These limitations necessitate deep learning-based detection methods.
5.2 SSD Network for Hot Spot Detection
Single Shot Multibox Detector (SSD) is a prominent deep learning object detection framework. SSD performs dense anchor-based prediction at multiple feature map scales, enabling detection of objects of varying sizes. The architecture modifies VGG16 by converting fully connected layers FC6 and FC7 into convolutional layers (Conv6 and Conv7). Additional convolutional layers are appended to extract higher-level semantic features. The network produces default boxes at each position of multiple feature maps, each with predefined scales and aspect ratios. The default box scales are computed by:
$$\begin{equation} S_k = S_{min} + \frac{S_{max} – S_{min}}{m-1} (k-1), \quad k \in [1, m] \tag{27} \end{equation}$$
where \(S_{min}=0.2\), \(S_{max}=0.9\), and \(m\) is the number of feature maps. Aspect ratios are specified as \(a_r \in \{1, 2, 3, \frac{1}{2}, \frac{1}{3}\}\), producing default boxes with widths \(w_k^a = S_k \sqrt{a_r}\) and heights \(h_k^a = S_k / \sqrt{a_r}\).
The SSD loss function combines localization and confidence losses:
$$\begin{equation} L(x, c, l, g) = \frac{1}{N} \left( L_{conf}(x, c) + \alpha L_{loc}(x, l, g) \right) \tag{28} \end{equation}$$
where \(N\) is the number of matched default boxes, \(\alpha\) is the weight factor (set to 1), and the localization loss is:
$$\begin{equation} L_{loc}(x, l, g) = \sum_{i \in Pos \; m \in \{cx,cy,w,h\}} x_{ij}^k \text{smooth}_{L1}(l_i^m – \hat{g}_j^m) \tag{29} \end{equation}$$
The confidence loss is a softmax loss over multiple class confidences:
$$\begin{equation} L_{conf}(x, c) = -\sum_{i \in Pos} x_{ij}^p \log(\hat{c}_i^p) – \sum_{i \in Neg} \log(\hat{c}_i^0) \tag{30} \end{equation}$$
SSD performs well in detecting small objects due to its multi-scale anchor design, making it suitable for hot spot detection in UAV-captured images. In this research, the input images were resized to 300 × 300 × 3, the initial learning rate was \(10^{-4}\), and the network was trained for 150 epochs.
5.3 Dataset Preparation and Evaluation Metrics
Infrared images of solar panel arrays were collected at a photovoltaic station in Xinjiang. In total, 1500 images were captured, of which 1000 formed the training set and 500 the testing set. Annotation was performed using LabelImg software. Two object categories were defined: “board” for solar panels and “hot_spot” for hot spot regions. Annotations comprised rectangular bounding boxes tightly enclosing each object.
The evaluation metrics included Intersection over Union (IoU), Precision, Recall, and Average Precision (AP). IoU measures the overlap between the predicted bounding box and the ground truth:
$$\begin{equation} \text{IoU} = \frac{\text{Area of Overlap}}{\text{Area of Union}} \tag{31} \end{equation}$$
Precision and Recall are defined as:
$$\begin{equation} \text{Precision} = \frac{TP}{TP + FP}, \quad \text{Recall} = \frac{TP}{TP + FN} \tag{32} \end{equation}$$
where TP (true positive), FP (false positive), and FN (false negative) are classification outcomes. Average Precision (AP) is the area under the precision-recall curve.
5.4 Experimental Results
The SSD network was evaluated against Faster-RCNN on the hot spot detection dataset. The detection results are summarized in Table 6.
| Model | Precision | Recall | AP |
|---|---|---|---|
| Faster-RCNN | 0.8723 | 0.8544 | 0.8361 |
| SSD | 0.8965 | 0.8831 | 0.8593 |
SSD exhibits higher precision (0.8965), recall (0.8831), and AP (0.8593) compared to Faster-RCNN, demonstrating superior detection accuracy. Moreover, SSD is computationally more efficient, making it better suited for real-time UAV inspection.
Detection visualizations confirm that SSD accurately localizes both solar panels and hot spots. The network distinguishes genuine hot spots from reflective regions on the panel surface, a crucial capability for reliable inspection. This robust performance is attributed to the multi-scale anchor design and the deep feature extraction capability of the modified VGG backbone.
For evaluating small-target performance, images captured at various UAV altitudes were tested. At moderate altitude, where the image contains multiple panel rows, SSD correctly identifies hot spots despite their small size. At very high altitude, when the image covers the entire station, some challenges arise. The network may merge two adjacent panels into one detection, and small reflective regions may be misclassified as hot spots. These limitations indicate that an optimal flight altitude range exists for reliable detection. When the target size becomes excessively small, adjusting the UAV flight strategy is recommended.
6. Conclusion and Future Prospects
6.1 Summary of Contributions
This dissertation investigates the application of image recognition algorithms for solar panel fault diagnosis in photovoltaic stations. The primary contributions include:
First, an adaptive image segmentation algorithm was developed for dust recognition. By exploiting the distinct characteristics of dust and solar panel backgrounds in the YCbCr color space, and by using row-wise adaptive thresholds, the algorithm effectively segments dust regions, even when dust distribution is uneven. Combined with LBP texture feature extraction and classification, the method achieves 88.56% accuracy with fast processing, providing a lightweight solution for real-time monitoring.
Second, an improved dense convolutional neural network was proposed for dust state recognition. The network integrates parameter-free attention (SimAM) to enhance feature extraction without additional parameters, and central difference convolution to improve illumination robustness. Data augmentation expanded the dataset tenfold, addressing the small-sample challenge. The improved model achieves 95.01% accuracy, outperforming VGG19 and ResNet50, with faster convergence and strong generalization.
Third, the SSD network was applied to hot spot detection in infrared images. SSD multi-scale detection capability enables accurate identification of small hot spot targets. Comparative experiments demonstrate that SSD outperforms Faster-RCNN in precision, recall, and AP, confirming its superiority for small object detection in photovoltaic inspection scenarios.
6.2 Future Prospects and Directions
Several avenues for future research exist. First, the development of large-scale open datasets for solar panel fault detection is crucial. The industrial domain suffers from data scarcity, and publicly available annotated datasets would greatly facilitate algorithm research and evaluation. Collaborative efforts among researchers, equipment manufacturers, and station operators are needed.
Second, improving the generalization ability of deep learning models remains an important goal. Currently, no single network architecture works uniformly well across all engineering applications. Research on domain adaptation and transfer learning methods could enhance performance on unseen data distributions.
Third, although the proposed dual-stage approach achieves effective dust recognition, end-to-end deep learning systems could further streamline the inspection pipeline, potentially combining dust detection and hot spot detection into a unified framework. The integration of multi-modal sensing, including RGB, thermal, and near-infrared imagery, could provide richer information for fault diagnosis.
Fourth, enhancing the robustness of detection models to extreme UAV altitudes and complex outdoor environments is essential. Hybrid approaches that combine traditional image processing with deep learning, and that incorporate physical knowledge about photovoltaic systems, may yield more reliable and interpretable results.
Finally, preventive maintenance should be emphasized. Rather than only detecting faults after occurrence, future work should focus on risk prediction and condition-based monitoring. By integrating real-time sensor data, weather forecasts, and degradation models, proactive maintenance strategies can be implemented to minimize downtime and extend the operational lifetime of solar panels.
In summary, this research provides effective image recognition algorithms for dust and hot spot fault detection on solar panels. The findings contribute to the advancement of intelligent operation and maintenance technologies for photovoltaic stations, supporting the global transition toward clean and sustainable energy sources.