In the context of the global energy transition, photovoltaic power generation has become one of the most important renewable energy sources. The safe and efficient operation of solar panels directly determines the economic benefits of photovoltaic power stations. However, solar panels often suffer from defects such as cracks, dust accumulation, hot spots, and fragmentation during long-term outdoor operation. These defects can significantly reduce the photoelectric conversion efficiency and even cause safety hazards. Traditional manual inspection methods are time-consuming, labor-intensive, and have low accuracy. Therefore, developing an intelligent defect detection method based on deep learning for solar panels is of great practical value. In this thesis, I focus on two core tasks: classification of solar panel defects using a novel random connection network, and edge feature extraction of solar panel cracks using a dual-channel attention mechanism. The entire research is carried out from the perspective of image processing and deep learning.
My research begins with a brief introduction to the development status of solar panels in China, highlighting the importance of defect detection and the limitations of existing methods. Then, I introduce the common defects of solar panels and the construction of the dataset. I explain the basic components of neural networks and design a random connection module for solar panel image classification. To further obtain detailed defect types, I construct a dual-channel network with a multi-scale attention mechanism for edge detection. The main contributions of my work are twofold. First, I propose a random connection network for solar panel defect classification, which combines the idea of Dropout with feature reuse. The network structure and feature repetition rate are determined by different random parameters. I use geometric transformations and generative adversarial networks to enhance the image dataset. Comparative experiments show that the proposed method achieves superior classification accuracy on both CIFAR-10 and electroluminescence (EL) datasets. Second, I propose a dual-channel multi-scale attention mechanism for solar panel crack detection. This method effectively detects low-level edges, boundaries, and object contours by using a semantic branch and a spatial detail branch, with a multi-scale attention mechanism and a hole fusion module. Experimental results demonstrate that the proposed method outperforms traditional edge detection algorithms in both qualitative and quantitative evaluations.
1. Introduction and Problem Statement
Solar panels are widely deployed in distributed and utility-scale photovoltaic systems. During their service life, solar panels are exposed to harsh environmental conditions, including temperature fluctuations, wind load, hail, and dust accumulation. These factors can cause various defects. Among them, cracks are particularly harmful because they can lead to the formation of hot spots, which may cause irreversible damage to the solar cells. In addition, dust accumulation on the surface of solar panels reduces the transmittance of sunlight, thereby decreasing the power output. In severe cases, corrosive substances in the dust can damage the protective layer of solar panels, resulting in internal circuit short circuits and safety hazards. Therefore, regular inspection and maintenance of solar panels are essential to ensure the long-term reliability of photovoltaic systems.
Traditional inspection methods rely on manual visual inspection or simple image processing techniques. However, these methods have several limitations. First, manual inspection is inefficient for large-scale photovoltaic plants. Second, conventional edge detection algorithms such as Sobel, Prewitt, and Canny often produce discontinuous and noisy edges, especially for low-contrast cracks in EL images. Third, the accuracy of traditional methods is highly sensitive to illumination and background interference. With the rapid development of deep learning, convolutional neural networks (CNNs) have shown remarkable performance in image classification, object detection, and semantic segmentation. Therefore, I aim to apply deep learning techniques to solve the defect detection problem of solar panels.
The main challenges in solar panel defect detection are as follows:
- The number of defect samples is limited, which makes training deep networks difficult.
- Defect features are diverse and subtle, such as micro-cracks, black spots, and fragments.
- The background of EL images contains noise and non-uniform illumination, which can interfere with feature extraction.
- Real-time inspection requires models with both high accuracy and low computational cost.
To address these challenges, I first construct a dataset of solar panel EL images and perform data augmentation. Then, I design a random connection network for classification. Finally, I develop an edge detection method based on a dual-channel attention mechanism to localize the defect regions accurately.
2. Dataset Construction and Enhancement
2.1 Dataset Description
In my research, I use electroluminescence (EL) imaging to capture the internal defects of solar panels. EL imaging is a common technique in the photovoltaic industry. It works by applying a forward bias to the solar cells and capturing the near-infrared light emitted by the cells. Defective areas appear as dark regions in the EL image. My dataset contains 300×300 grayscale EL images collected from publicly available sources. The images include three main defect categories: micro-cracks, fragment defects, and black core defects, as well as normal images without any defects.
The defect categories are summarized in the following table:
| Defect Type | Description | Impact on solar panels |
|---|---|---|
| Micro-crack | Thin dark lines in EL image, often caused by mechanical stress or improper handling | Reduces current collection efficiency and may lead to open-circuit failures |
| Fragment defect | Irregular black areas with sharp edges, usually due to severe impact or solder joint failure | Significant power loss; in extreme cases, the whole solar panel becomes unusable |
| Black core defect | Dark circular regions in the center of the cell, caused by material impurities or process issues | Degrades cell performance and reduces the reliability of solar panels |
Due to the limited number of original images, I randomly selected 600 EL images and split them into training, validation, and test sets. However, the imbalance among defect categories may cause the network to be biased toward the majority class. Therefore, I adopt several data augmentation techniques to enlarge the dataset and balance the categories.
2.2 Image Preprocessing
Image preprocessing is essential to improve the quality of EL images and to reduce noise. I apply the following preprocessing steps:
- Grayscale conversion: Since the EL images are already grayscale, I only normalize the pixel values to the range [0,1].
- Contrast enhancement: To make the defect areas more visible, I use histogram equalization and gamma correction. The gamma transformation is defined as:
$$ I_{out}(x,y) = c \cdot I_{in}(x,y)^\gamma $$
where c is a scaling constant and γ is the gamma parameter. - Binarization: For some experiments, I convert the grayscale image to a binary image using a global threshold T, or using Otsu’s method.
- Smoothing: To suppress noise, I apply Gaussian filtering or median filtering. The Gaussian filter output is:
$$ G(x,y) = \frac{1}{2\pi \sigma^2} \exp\left(-\frac{x^2 + y^2}{2\sigma^2}\right) $$
where σ is the standard deviation of the Gaussian kernel.
2.3 Geometric Augmentation
To increase the diversity of the training samples, I use geometric transformations such as rotation, flipping, and random cropping. These operations are label-preserving for classification tasks. The following table lists the geometric augmentation methods I used:
| Operation | Description | Example |
|---|---|---|
| Horizontal flip | Mirror the image along the vertical axis | Original ↔ Flipped |
| Vertical flip | Mirror the image along the horizontal axis | Original ↔ Upside down |
| Rotation | Rotate by angles of 90°, 180°, 270° | Original ↔ Rotated |
| Random cropping | Extract a random sub-region and resize to original size | Various sub-images |
After these operations, the number of solar panel images is significantly increased, which helps the deep neural network to learn more robust feature representations.
2.4 Generative Adversarial Network-Based Augmentation
In addition to geometric augmentation, I employ a Deep Convolutional Generative Adversarial Network (DCGAN) to generate synthetic solar panel defect images. The DCGAN consists of a generator and a discriminator. The generator takes a random noise vector z as input and outputs an image. The discriminator tries to distinguish between real images and generated images. The training objective can be expressed as:
$$ \min_{G} \max_{D} V(D,G) = \mathbb{E}_{x \sim p_{data}(x)}[\log D(x)] + \mathbb{E}_{z \sim p_z(z)}[\log(1 – D(G(z)))] $$
After training, the generator is able to produce images that are difficult for the discriminator to distinguish from real ones. I select generated images with high confidence scores and add them to the training set. This method improves the quality and diversity of the dataset. With geometric and DCGAN-based augmentation, the final dataset is expanded to 1,100 images.
3. Random Connection Network for Solar Panel Defect Classification
3.1 Background of Convolutional Neural Networks
Convolutional neural networks are the foundation of modern deep learning. A typical CNN consists of convolutional layers, pooling layers, fully connected layers, and activation functions. The convolutional layer performs feature extraction using learnable kernels. The pooling layer reduces the spatial dimension and increases the receptive field. The fully connected layer combines the features for classification.
The convolution operation is defined as:
$$ P(i,j) = \sum_{a=0}^{m-1} \sum_{b=0}^{n-1} Q(a,b) \cdot f(i+a, j+b) $$
where P is the output feature map, Q is the convolution kernel of size m×n, and f is the input feature map.
Activation functions introduce nonlinearity into the network. Common activation functions include Sigmoid, Tanh, and ReLU. Their formulas are:
$$ \text{Sigmoid}(x) = \frac{1}{1 + e^{-x}} $$
$$ \text{Tanh}(x) = \frac{e^{x} – e^{-x}}{e^{x} + e^{-x}} $$
$$ \text{ReLU}(x) = \max(0, x) $$
In my research, I use the LeakyReLU activation function, which is defined as:
$$ \text{LeakyReLU}(x) = \begin{cases} x & \text{if } x > 0 \\ \alpha x & \text{otherwise} \end{cases} $$
where α is a small constant (e.g., 0.01). The LeakyReLU function helps to avoid dead neurons and stabilizes the gradient flow.
3.2 Residual and Dense Networks
Residual networks (ResNet) introduce skip connections to solve the degradation problem in deep networks. The residual block is formulated as:
$$ \mathbf{y} = \mathcal{F}(\mathbf{x}, \{W_i\}) + \mathbf{x} $$
where x is the input, F is the residual mapping, and y is the output. Dense networks (DenseNet) concatenate all previous feature maps as input to the current layer. The output of the ℓ-th layer is:
$$ \mathbf{x}_\ell = H_\ell([\mathbf{x}_0, \mathbf{x}_1, \dots, \mathbf{x}_{\ell-1}]) $$
where [·] denotes channel-wise concatenation. Both ResNet and DenseNet have achieved excellent performance in image classification. However, they still have some shortcomings in feature reuse and parameter efficiency.
3.3 Proposed Random Connection Module
Inspired by Dropout and feature reuse, I propose a random connection block that randomly connects the current layer to future layers. The random connection network (RandomNet) is a generalization of DenseNet with stochastic connections. The key idea is to use a random matrix to decide which connections are active during training. This approach increases the feature repetition rate without adding many parameters, thereby improving the generalization ability of the network.
Let y1, y2, …, yn be the outputs of n layers in a random block. The output of layer i is computed as:
$$ \mathbf{y}_i = f\left( \sum_{j=1}^{i-1} R_{j,i} \cdot \mathbf{y}_j + F(\mathbf{y}_i, W_i) \right) $$
where R is a random upper-triangular matrix with binary entries, f is the activation function, and F represents the convolution operation. The random matrix R is generated according to the following rules:
$$ N_{\min} = \left\lceil \frac{n – i}{2} \right\rceil , \quad N_{\max} = n – i $$
where N is the number of jumps and L is the jump length. The random matrix is then used to connect the layers.
The overall architecture of RandomNet50 consists of an input layer, a stem convolutional block, four random blocks, a global average pooling layer, and a fully connected layer. Each random block contains several convolutional layers with batch normalization and LeakyReLU activation. The structure of a random block is illustrated by the following table:
| Block | Layer composition | Output channels |
|---|---|---|
| Block 1 | 1×1 conv, 3×3 conv, 1×1 conv | 64, 64, 256 |
| Block 2 | 1×1 conv, 3×3 conv, 1×1 conv | 128, 128, 512 |
| Block 3 | 1×1 conv, 3×3 conv, 1×1 conv | 256, 256, 1024 |
| Block 4 | 1×1 conv, 3×3 conv, 1×1 conv | 512, 512, 2048 |
In each block, the random connections are applied between the intermediate convolutional layers. I also use Dropout in the fully connected layer to further prevent overfitting.
3.4 Experimental Setup and Evaluation Metrics
I implemented all experiments using Python 3.8 and TensorFlow 2.x. The hardware environment includes an NVIDIA GPU with CUDA support. The training hyperparameters are as follows:
- Batch size: 32
- Initial learning rate: 0.001
- Learning rate decay: multiply by 0.1 when loss plateaus
- Loss function: binary cross-entropy
- Optimizer: Adam
- Number of epochs: 200
The binary cross-entropy loss is defined as:
$$ \mathcal{L} = -\frac{1}{N} \sum_{i=1}^{N} \left[ y_i \log \hat{y}_i + (1 – y_i) \log (1 – \hat{y}_i) \right] $$
where yi is the ground-truth label and ŷi is the predicted probability.
The Adam optimizer updates the parameters as follows:
$$ m_t = \beta_1 m_{t-1} + (1 – \beta_1) g_t $$
$$ v_t = \beta_2 v_{t-1} + (1 – \beta_2) g_t^2 $$
$$ \hat{m}_t = \frac{m_t}{1 – \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 – \beta_2^t} $$
$$ \theta_{t+1} = \theta_t – \frac{\eta}{\sqrt{\hat{v}_t} + \epsilon} \hat{m}_t $$
where η is the learning rate, β1 = 0.9, β2 = 0.999, ε = 10-8.
The evaluation metrics used in my experiments are accuracy, precision, recall, and F1 score. Their definitions are:
$$ \text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN} $$
$$ \text{Precision} = \frac{TP}{TP + FP} $$
$$ \text{Recall} = \frac{TP}{TP + FN} $$
$$ F1 = \frac{2 \times \text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}} $$
where TP, FP, TN, and FN represent the numbers of true positives, false positives, true negatives, and false negatives, respectively.
3.5 Results and Discussion
I first evaluated the proposed RandomNet50 on the CIFAR-10 dataset and compared it with several classical networks, including AlexNet, VGG11, VGG19, ResNet50, and DenseNet50. The accuracy comparison is shown in the following table:
| Model | Accuracy on CIFAR-10 |
|---|---|
| AlexNet | 78.81% |
| VGG11 | 92.61% |
| VGG19 | 93.65% |
| ResNet50 | 94.00% |
| DenseNet50 | 93.89% |
| RandomNet50 (Ours) | 96.15% |
From the table, it can be observed that RandomNet50 achieves the highest accuracy of 96.15%, outperforming DenseNet50 by 2.26% and ResNet50 by 2.15%. This demonstrates that random connections can improve the feature reuse efficiency and enhance the discriminative power of the network.
Next, I evaluated the proposed method on the solar panel EL dataset. The results are shown in the following table:
| Model | Accuracy | Loss | Number of Parameters |
|---|---|---|---|
| RandomNet50 (Ours) | 88.23% | 0.34 | 1.5×106 |
| ResNet50 | 83.15% | 0.41 | 21.3×106 |
| DenseNet50 | 87.46% | 0.35 | 7×106 |
The proposed RandomNet50 achieves an accuracy of 88.23%, which is 5.08% higher than ResNet50 and 1.23% higher than DenseNet50. Moreover, the number of parameters of RandomNet50 is only 1.5 million, which is much smaller than both ResNet50 and DenseNet50. This indicates that the random connection network has high parameter efficiency and is suitable for deployment in resource-constrained environments.
I also investigated the influence of different network structures on the performance of RandomNet. The following table shows the accuracy and loss for three different configurations of the four random blocks:
| Block1 configuration | Block2 configuration | Block3 configuration | Block4 configuration | Accuracy | Loss |
|---|---|---|---|---|---|
| [2,4,4,2] | [2,4,5,2] | [4,3,3,4] | [2,4,4,2] | 88.12% | 0.35 |
| [2,4,5,2] | [2,4,5,2] | [2,4,5,2] | [2,4,5,2] | 88.23% | 0.34 |
| [4,3,3,4] | [4,3,3,4] | [4,3,3,4] | [4,3,3,4] | 88.19% | 0.34 |
The results indicate that the proposed random connection network is robust to different structural configurations. The differences in accuracy are within 0.11%, which shows that the random connection mechanism provides stable performance.
The loss curves during training show that RandomNet50 converges faster than ResNet50 and DenseNet50. In the early stage, the loss of RandomNet50 drops rapidly, which suggests that the random connections help the network to propagate gradients more effectively. The accuracy curves also demonstrate that RandomNet50 maintains a higher accuracy throughout the training process.
4. Dual-Channel Attention Mechanism for Edge Feature Extraction of Solar Panels
4.1 Deep Learning Edge Detection Algorithms
Edge detection of solar panel cracks is a challenging task due to the low contrast and irregular shape of cracks. Traditional edge detection methods, such as Canny and Sobel, fail to produce continuous and accurate edges in EL images. To address this issue, I adopt deep learning-based edge detection algorithms. The two classic algorithms are HED and RCF.
HED (Holistically-Nested Edge Detection) is based on VGG16. It uses multi-scale side outputs and a fusion layer to generate the final edge map. The loss function of HED is computed for each side output as well as the fusion output. HED can directly perform image-to-image training and prediction.
RCF (Rich Convolutional Features) improves HED by using all convolutional layers instead of only the last layer of each stage. RCF accumulates features from all intermediate layers and then applies 1×1 convolution to produce the edge map. Both HED and RCF have achieved good results on natural image edge detection. However, their performance on solar panel cracks is still limited due to the complex background and noise in EL images.
4.2 Proposed Network Architecture
I propose a dual-channel network with a multi-scale attention mechanism (named MCA) for solar panel crack detection. The overall architecture consists of two branches: a spatial detail branch and a semantic branch. The spatial detail branch has three convolutional blocks and captures low-level features such as edges and textures. The semantic branch has five deep convolutional blocks and captures high-level semantic information. The two branches are fused to obtain both fine spatial details and rich semantic context.
The structure of the spatial branch and semantic branch is summarized in the following table:
| Stage | Spatial branch | Semantic branch |
|---|---|---|
| S1 | Conv3×3, 64, stride 2; Conv3×3, 64, stride 1 | Deep Conv3×3, 16, stride 2; Deep Conv3×3, 16, stride 1 |
| S2 | Conv3×3, 128, stride 2; Conv3×3, 128, stride 1 (×2) | Deep Conv3×3, 32, stride 2; Deep Conv3×3, 32, stride 1 |
| S3 | Conv3×3, 256, stride 2; Conv3×3, 256, stride 1 (×2) | Deep Conv3×3, 64, stride 2; Deep Conv3×3, 64, stride 1 |
| S4 | — | Deep Conv3×3, 128, stride 2; Deep Conv3×3, 128, stride 1 (×2) |
| S5 | — | Deep Conv3×3, 256, stride 2; Deep Conv3×3, 256, stride 1 (×2) |
To enhance the feature representation, I embed the multi-scale attention mechanism (MCA) into both branches. The MCA module transforms the input feature map from three dimensions: channel, height, and width. Specifically, for an input feature F with shape C×H×W, I perform dimension transformations to obtain F1 (C×H×W), F2 (H×C×W), and F3 (W×H×C). Then, I apply global average pooling and max pooling to each transformed feature to generate spatial attention maps. The attention maps are fused together. The process can be expressed as:
$$ M_s = \sigma \left( \text{Conv} \left( [ \text{AvgPool}(F_1) ; \text{MaxPool}(F_1) ] \right) \right) + \sigma \left( \text{Conv} \left( [ \text{AvgPool}(F_2) ; \text{MaxPool}(F_2) ] \right) \right) + \sigma \left( \text{Conv} \left( [ \text{AvgPool}(F_3) ; \text{MaxPool}(F_3) ] \right) \right) $$
where σ denotes the sigmoid activation, Conv denotes a 1×1 convolution, and [·;·] represents channel-wise concatenation. The final attention map Ms is used to re-weight the original feature map.
In addition, I propose a multi-scale dilated fusion module to fuse the side outputs from the last two convolutional blocks of the semantic branch. The dilated fusion module uses dilated convolutions with different dilation rates to capture multi-scale contextual information. The structure is shown in the following table:
| Input | Operation | Dilation rate | Output size |
|---|---|---|---|
| Side output 4 | 3×3 conv | 1 | H/16 × W/16 |
| Side output 4 | 3×3 conv | 2 | H/16 × W/16 |
| Side output 5 | 3×3 conv | 1 | H/32 × W/32 |
| Side output 5 | 3×3 conv | 2 | H/32 × W/32 |
| Global pooling | 1×1 conv | — | 1×1 |
All the features are upsampled to the same spatial resolution and concatenated together. Then, a 1×1 convolution is used to produce the final edge map.
4.3 Experimental Results
I conducted comparative experiments on the solar panel EL dataset using HED, FCN, RCF, and the proposed method. The quantitative results are shown in the following table:
| Method | Precision | Recall | F1 measure |
|---|---|---|---|
| HED | 66.51% | 76.96% | 73.35% |
| RCF | 67.80% | 87.73% | 75.81% |
| FCN | 68.22% | 81.68% | 74.64% |
| Proposed method | 69.15% | 84.92% | 76.18% |
The proposed method achieves the highest F1 measure of 76.18%, which is 2.83% higher than HED, 0.37% higher than RCF, and 1.54% higher than FCN. The P-R curves also show that the proposed method has the largest area under the curve, indicating its superior performance.
To analyze the contribution of each component, I performed ablation experiments. The results are shown in the following table:
| Experiment | Multi-scale attention | Dilated fusion | Precision | Recall | F1 measure |
|---|---|---|---|---|---|
| 1 | No | No | 64.43% | 78.05% | 70.81% |
| 2 | Yes | No | 66.56% | 80.14% | 72.54% |
| 3 | Yes | Yes | 69.15% | 84.92% | 76.18% |
From Experiment 1 to Experiment 2, adding the multi-scale attention mechanism improves the F1 measure by 1.73%. From Experiment 2 to Experiment 3, adding the dilated fusion module further improves the F1 measure by 3.64%. This demonstrates that both modules are effective and complementary.
The visual results of the ablation experiments show that the proposed method produces crack edges that are continuous and accurate. In contrast, the baseline dual-channel network produces broken edges and some false detections. The multi-scale attention mechanism helps the network focus on the crack regions, while the dilated fusion module expands the receptive field and captures more contextual information.
5. Conclusion and Future Work
In this thesis, I have presented a deep learning-based approach for solar panel defect detection. My research covers two main tasks: defect classification and crack edge detection. For the classification task, I proposed a random connection network that combines Dropout and feature reuse. The random connections allow the network to flexibly select the connections between layers, resulting in better feature propagation and higher classification accuracy. The experimental results on CIFAR-10 and solar panel EL datasets demonstrate that the proposed RandomNet50 outperforms ResNet50 and DenseNet50 while using significantly fewer parameters.
For the edge detection task, I proposed a dual-channel multi-scale attention mechanism. The network consists of a spatial detail branch and a semantic branch. The multi-scale attention module re-weights the feature maps from three dimension orientations, effectively enhancing the crack features. The dilated fusion module combines multi-scale context from the semantic branch. The proposed method achieves the best F1 measure of 76.18% on the solar panel EL dataset, outperforming HED, RCF, and FCN. The ablation experiments verify the effectiveness of each component.
There are still some limitations in my work. First, the dataset size is relatively small, and the diversity of defect types is limited. In the future, I plan to collect more real-world solar panel images and annotate more defect categories. Second, the proposed methods are mainly tested on offline images. For practical online inspection, the inference speed and memory consumption need to be further optimized. Third, the classification task currently focuses on binary or a few classes. I will extend the model to multi-class defect recognition and localization.
In conclusion, deep learning has great potential in the field of solar panel defect detection. My random connection network and dual-channel attention mechanism provide effective solutions for classifying and localizing defects in solar panels. Future work will focus on improving the generalization ability and deploying the models in real-time photovoltaic inspection systems.