1. Introduction and Background
In recent years, the rapid advancement of global economies and technologies has brought about a dramatic rise in the demand for clean energy. Among various renewable energy sources, solar energy has been recognized as one of the most promising alternatives to fossil fuels. The solar panel, also known as photovoltaic (PV) module, is the core component that converts sunlight into electricity. With the continuous expansion of photovoltaic power stations, the reliability and safety of solar panel systems have become increasingly important. However, solar panels are typically deployed in outdoor environments where they are exposed to harsh weather conditions such as strong winds, heavy rain, high temperatures, and dust accumulation. Over time, these conditions can lead to various types of faults, including hot spots, cracks, shading effects, junction box failures, missing modules, and non-generating strings. These faults can significantly reduce the power generation efficiency of the solar panel system and even cause permanent damage or fire hazards. Therefore, timely and accurate fault detection is essential for maintaining the stable operation of photovoltaic power plants.
Traditional fault detection methods for solar panels heavily rely on manual inspection, which is not only time-consuming and labor-intensive but also subjective and unreliable. To overcome these limitations, researchers have explored various automated approaches. Early methods based on electrical characteristic analysis required installing a large number of sensors, which increased the cost and complexity of the system. More recently, infrared thermography has been widely adopted for fault diagnosis because it can capture temperature distributions on the surface of solar panels. However, processing infrared images with traditional image processing algorithms often suffers from low accuracy and poor generalization, especially when the images contain complex backgrounds or small fault regions.
With the remarkable progress of deep learning, especially convolutional neural networks (CNNs), object detection has achieved outstanding performance in many computer vision tasks. Deep learning-based detectors can automatically learn discriminative features from raw images and have been successfully applied to fault detection in various industrial scenarios. Among these detectors, single-stage algorithms such as YOLO (You Only Look Once) family have gained significant popularity due to their real-time inference speed and competitive accuracy. The latest version, YOLOv7, has demonstrated state-of-the-art performance in real-time object detection. Nevertheless, applying YOLOv7 directly to solar panel fault detection faces several challenges. First, the faults vary significantly in shape, size, and appearance. Second, some faults are tiny and densely distributed in the image, making them difficult to detect. Third, the background of solar panel images often resembles the fault regions, which can confuse the model. As a result, direct application of YOLOv7 leads to missed and false detections.
To address these challenges, this thesis proposes an improved solar panel fault detection model based on YOLOv7. The improvements are threefold. In the backbone network, a parameter-free attention module called SimAM is introduced to highlight fault-related features and suppress the interference from complex backgrounds. In the feature fusion network, a weighted bidirectional feature pyramid network (BiFPN) is adopted to replace the original PANet, which reduces redundant computation and enhances the model’s ability to detect dense and small targets. In the detection head, the SIoU loss function is used to replace the original CIoU loss, which reduces the degrees of freedom in the loss and leads to faster convergence and better accuracy. In addition to accuracy improvement, this work also investigates the lightweight deployment of the fault detection model on edge devices. By replacing the original backbone with lightweight networks such as EfficientNet and EfficientNetV2, the model size is significantly reduced while the inference speed is greatly improved. This meets the real-time inspection requirements of large-scale photovoltaic plants.
The rest of this paper is organized as follows. Section 2 reviews the fundamental technologies related to convolutional neural networks and object detection. Section 3 details the proposed improved YOLOv7 model and presents experimental results. Section 4 discusses the lightweight network design and its experimental evaluation. Finally, Section 5 concludes the paper and outlines future directions.
2. Related Theories and Technologies
2.1 Convolutional Neural Networks
Convolutional neural networks are a class of deep learning models specifically designed for processing grid-structured data such as images. A typical CNN consists of multiple layers, including convolutional layers, pooling layers, batch normalization layers, and activation functions. The convolutional layer is the core building block that extracts local features by performing a dot product between the input and a set of learnable kernels. The operation can be mathematically described as:
$$ Y_{i,j,k} = \sum_{c=0}^{C_{in}-1} \sum_{m=0}^{K-1} \sum_{n=0}^{K-1} X_{i+m, j+n, c} \cdot W_{m,n,c,k} + b_k $$
where \(X\) is the input feature map, \(W\) is the convolution kernel, \(b\) is the bias, and \(Y\) is the output feature map. The pooling layer, such as max pooling or average pooling, reduces the spatial dimension of the feature map, thereby increasing the receptive field and decreasing computational complexity. Batch normalization (BN) stabilizes the distribution of activations by normalizing each mini-batch, which accelerates training and improves generalization. Activation functions introduce non-linearity; common choices include ReLU and SiLU.
2.2 Object Detection Architectures
Modern deep learning-based object detectors are generally composed of three main components: a backbone network, a feature fusion network (neck), and a detection head. The backbone extracts hierarchical features from the input image. Popular backbones include VGGNet, ResNet, DenseNet, and more lightweight ones like SqueezeNet. The feature fusion network aggregates features from different scales to improve detection of objects with varying sizes. The detection head performs classification and bounding box regression. Loss functions, such as IoU-based losses and NMS post-processing, are also crucial components.
The two-stage detectors, e.g., Faster R-CNN, first generate region proposals and then classify and refine them. They are generally more accurate but slower. In contrast, one-stage detectors, e.g., YOLO and SSD, directly predict boxes and class probabilities from feature maps, enabling real-time inference. YOLOv7 is a state-of-the-art one-stage detector that introduces several advanced modules such as ELAN, MP-1, and the SPPCSPC architecture.
3. Improved YOLOv7 Model for Solar Panel Fault Detection
3.1 Baseline YOLOv7 Architecture
YOLOv7 was proposed in 2022 and achieved excellent performance in real-time object detection. Its architecture is illustrated in the following description. The input image is first resized to 640×640 pixels. The backbone consists of convolutional blocks (CBS), ELAN (Efficient Layer Aggregation Network), and MP-1 modules. The CBS block is composed of a conventional convolution, batch normalization, and SiLU activation. The SiLU function is defined as:
$$ \text{SiLU}(x) = x \cdot \sigma(x) $$
where \(\sigma(\cdot)\) is the sigmoid function. ELAN is designed to capture richer features by controlling the shortest and longest gradient paths. It splits the input into two branches, applies a series of 1×1 and 3×3 convolutions, and concats the results. The MP-1 module performs downsampling by combining max pooling and a strided convolution branch, as shown in Figure 1. The feature fusion network of YOLOv7 uses PANet (Path Aggregation Network) to combine low-level and high-level features. Finally, the detection head predicts boxes at three scales.
3.2 Challenges in Solar Panel Fault Detection
Solar panel fault images present several unique difficulties. The faults can be as large as a whole panel or as tiny as a few pixels. For instance, hot spots are usually small and appear as bright spots in infrared images, while missing components may occupy a large area. The background often contains similar textures and edges, making it hard for the model to differentiate faults from normal panel structures. Moreover, in a single image, there may be dozens or even hundreds of small faults, which are densely packed. This leads to severe occlusions and overlapping bounding boxes, causing the baseline YOLOv7 to miss many targets. Therefore, the model must be enhanced to become more focused on informative regions and more robust to scale variations.
3.3 Attention Mechanism: SimAM
Attention mechanisms have been widely used in computer vision to enable neural networks to concentrate on the most relevant parts of the input. Traditional attention modules such as channel attention and spatial attention exploit only one dimension. However, neuroscience suggests that human visual processing simultaneously considers both spatial and channel information. SimAM (Simple, parameter-free Attention Module) is a recent attention module that computes a three-dimensional attention weight for each neuron without introducing extra parameters. It directly estimates the importance of each neuron based on its energy function.
The energy value for a neuron \(t_i\) in a single channel is computed as:
$$ E(t_i) = \frac{4\left(\hat{\sigma}^2 + \lambda\right)}{(t_i – \hat{\mu})^2 + 2\hat{\sigma}^2 + 2\lambda} $$
where \(\hat{\mu}\) and \(\hat{\sigma}^2\) are the mean and variance of all neurons in that channel, and \(\lambda\) is a hyperparameter (set to 0.0001). A lower energy value indicates that the neuron is more informative and should be weighted higher. The weighted feature map is obtained by:
$$ \widetilde{X} = X \cdot \text{sigmoid}\left(\frac{1}{E}\right) $$
where \(E\) contains the energy values for all neurons. This operation is parameter-free and computationally efficient. In the proposed model, SimAM is inserted into the backbone network after the ELAN modules. This allows the model to emphasize fault features and suppress irrelevant background features, thereby improving detection accuracy.
3.4 Feature Fusion Network with BiFPN
PANet, the original feature fusion structure in YOLOv7, propagates features in a bottom-up and top-down manner. However, some nodes in PANet have only one input edge, contributing little to feature fusion while increasing computation. The Weighted Bidirectional Feature Pyramid Network (BiFPN) was introduced in EfficientDet to address these issues. BiFPN deletes nodes that have only one input edge, adds extra cross-scale connections between the top-down and bottom-up paths, and assigns learnable weights to each input feature. The weighted fusion operation is defined as:
$$ O = \sum_i \frac{w_i}{\epsilon + \sum_j w_j} \cdot I_i $$
where \(w_i\) is a learnable weight, and \(I_i\) is the input feature map at a given level. This mechanism allows the network to determine the relative importance of features at different scales, improving the detection of small and dense objects. In this work, PANet is replaced with BiFPN. The modified model reduces redundant computations and enhances multi-scale feature fusion, making it more suitable for solar panel fault detection tasks.
3.5 SIoU Loss Function
The original YOLOv7 uses CIoU (Complete IoU) as the localization loss. CIoU considers the overlap area, the distance between center points, and the aspect ratio consistency. However, CIoU has some limitations when the predicted box and ground truth box share the same aspect ratio but differ in scale, which makes the aspect ratio penalty ineffective. Moreover, CIoU cannot independently optimize the width and height due to its coupling. To overcome these issues, SIoU (Smooth IoU) loss is introduced. SIoU incorporates the angle cost, distance cost, and shape cost, which reduces the degrees of freedom and accelerates convergence.
The angle cost is defined as:
$$ \Lambda = 1 – 2 \sin^2\left( \arcsin\left( \frac{h_{gt} – h_c}{2\sigma} \right) – \frac{\pi}{4} \right) $$
where \(\sigma\) is the distance between the centers of the predicted and ground truth boxes. The distance cost is:
$$ \Delta = \sum_{t \in \{x, y\}} (1 – e^{-\gamma \rho_t}) $$
where \(\gamma = 2 – \Lambda\) and \(\rho_t = \left( \frac{b_{c}^{gt} – b_c}{w_c} \right)^2\) for the x-axis and similarly for y-axis. The shape cost is:
$$ \Omega = \sum_{t \in \{w, h\}} (1 – e^{-\theta \omega_t}) $$
with \(\theta\) controlling the degree of attention to shape. Finally, the SIoU loss is given by:
$$ L_{SIoU} = 1 – IoU + \frac{\Delta + \Omega}{2} $$
By using SIoU, the model can achieve more stable and accurate bounding box regression, which is critical for localizing solar panel faults precisely.
3.6 Overall Improved Model Structure
The proposed solar panel fault detection model is based on YOLOv7 with three major modifications: (1) SimAM attention modules are inserted into the backbone; (2) BiFPN replaces PANet; (3) SIoU replaces CIoU. The overall structure is shown in Figure 2. The input image goes through the backbone with SimAM, then through BiFPN, and finally to the detection head with SIoU loss.
3.7 Dataset and Evaluation Metrics
To evaluate the proposed model, a solar panel fault dataset was constructed from aerial images captured by unmanned aerial vehicles (UAVs). The dataset contains 6205 images with six categories of faults: hot spot (A1), crack (A2), shaded hot spot (A3), junction box issue (B), missing component (C), and non-generating string (D). Table 1 presents the distribution of fault types and the number of labeled bounding boxes.
| Fault Type | Number of Images | Number of Annotations |
|---|---|---|
| A1 | 2022 | 25720 |
| A2 | 936 | 1643 |
| A3 | 1060 | 11004 |
| B | 1411 | 2088 |
| C | 131 | 240 |
| D | 645 | 1379 |
| Total | 6205 | 42074 |
The annotations were created using the LabelImg tool, generating XML files in PASCAL VOC format. Each XML contains the image path, image size, and bounding box coordinates for each fault. For model evaluation, three metrics are used: Precision, Recall, and mean Average Precision (mAP). Precision and Recall are defined as:
$$ \text{Precision} = \frac{TP}{TP + FP}, \qquad \text{Recall} = \frac{TP}{TP + FN} $$
where TP (true positives) are correctly detected faults, FP (false positives) are incorrectly detected faults, and FN (false negatives) are missed faults. The average precision (AP) is the area under the Precision-Recall curve for each class. The mean average precision (mAP@.5) is the average AP over all classes at an IoU threshold of 0.5.
3.8 Experimental Results and Analysis
All experiments were conducted on a Linux server with Ubuntu 18.04, Python 3.9, and PyTorch 1.11. The GPU was an NVIDIA Tesla V100 with 128 GB of RAM. The baseline and improved models were trained with the same hyperparameters, including an input size of 640×640 and 300 epochs.
To verify the contribution of each improvement module, an ablation study was designed. Table 2 reports the results of different configurations.
| Experiment | SimAM | BiFPN | SIoU | Precision (%) | Recall (%) | mAP@.5 (%) |
|---|---|---|---|---|---|---|
| 1 (Baseline) | 89.1 | 90.6 | 89.6 | |||
| 2 | ✓ | 93.8 | 93.9 | 95.6 | ||
| 3 | ✓ | ✓ | 95.3 | 95.1 | 95.7 | |
| 4 (Ours) | ✓ | ✓ | ✓ | 95.2 | 95.4 | 96.1 |
From Table 2, it is clear that each modification contributes positively to the overall accuracy. The complete improved model achieves a mAP@.5 of 96.1%, which is 6.5 percentage points higher than the baseline YOLOv7 (89.6%). Precision improved from 89.1% to 95.2%, indicating a significant reduction in false positives.
Table 3 compares the proposed model with other popular object detectors on the same solar panel fault dataset.
| Model | Input Size | Precision (%) | mAP@.5 (%) |
|---|---|---|---|
| Faster R-CNN | 640 | 83.6 | 80.1 |
| Mask R-CNN | 640 | 84.5 | 85.3 |
| SSD | 640 | 82.3 | 82.1 |
| YOLOv5 | 640 | 84.8 | 85.3 |
| YOLOv7 | 640 | 89.1 | 89.6 |
| Ours | 640 | 95.2 | 96.1 |
The proposed model outperforms all compared methods. The Precision-Recall curves for the proposed model show that most categories have AP values above 97%, except for the challenging shaded hot spot category. The visual inspection results also demonstrate that the improved model can accurately localize small and dense faults, confirming its effectiveness.
4. Lightweight Network Design for Real-Time Deployment
4.1 Motivation
In practical applications, real-time inspection of solar panels is usually performed by UAVs carrying edge devices. However, edge devices have limited computational power, memory, and energy consumption. Although the improved YOLOv7 model achieves high accuracy, its computational cost is still too high for deployment on such devices. Therefore, model lightweighting is necessary. The goal is to reduce the model size and computation while maintaining acceptable accuracy. Common techniques include pruning, knowledge distillation, and designing lightweight module replacements.
4.2 Depthwise Separable Convolution
Depthwise separable convolution (DSConv) is a key component in many lightweight networks. It decomposes a standard convolution into two steps: a depthwise convolution and a pointwise convolution. The depthwise convolution applies a single filter to each input channel, and the pointwise convolution uses 1×1 convolutions to combine the channels. The computational cost of DSConv is:
$$ Q_{dsc} = (D_K^2 \cdot M + M \cdot N) \cdot D_W \cdot D_H $$
where \(D_K\) is the kernel size, \(M\) is the number of input channels, \(N\) is the number of output channels, and \(D_W \times D_H\) is the spatial size. Compared with the standard convolution cost of:
$$ Q_{std} = D_K^2 \cdot M \cdot N \cdot D_W \cdot D_H $$
DSConv reduces the computation by a factor of approximately \(1/N + 1/D_K^2\). This replacement is used in the YOLOv7 backbone to reduce the model size.
4.3 Lightweight Backbone: EfficientNet and EfficientNetV2
EfficientNet is a family of models that uses a compound scaling method to balance depth, width, and resolution. Its basic building block is the MBConv (Mobile Inverted Residual Bottleneck Convolution) module, which includes depthwise convolutions, squeeze-and-excitation (SE) attention, and residual connections. The MBConv block can effectively reduce parameters while maintaining high expressiveness. EfficientNetV2 improves upon EfficientNet by introducing Fused MBConv, which merges the depthwise convolution and the pointwise expansion into a single regular convolution. This reduces memory access and improves training/inference speed. In EfficientNetV2, Fused MBConv is used in early layers, while MBConv is used in deeper layers, achieving the optimal trade-off between accuracy and efficiency.
The SE module in MBConv computes channel-wise attention by:
$$ s = \sigma(W_2 \cdot \text{ReLU}(W_1 \cdot \text{GlobalAvgPool}(x))) $$
where \(W_1\) and \(W_2\) are fully connected layers. The module suppresses less useful channels and amplifies important ones.
4.4 Proposed Lightweight Fault Detection Models
To adapt the YOLOv7 model for edge devices, two lightweight variants are designed. In the first variant, EfficientNet is used as the backbone to replace the original CSPDarknet-style backbone. The feature maps extracted from different stages of EfficientNet are then fed into the BiFPN and detection head. In the second variant, EfficientNetV2 is used as the backbone, which further reduces the model size and improves the inference speed. Both variants retain the aforementioned improvements (SimAM, BiFPN, SIoU) to preserve detection accuracy.
4.5 Experimental Evaluation of Lightweight Models
Table 4 compares the performance of the baseline YOLOv7 and the lightweight variants on the same dataset.
| Experiment | Precision (%) | Model Size (MB) | FPS |
|---|---|---|---|
| 1: Baseline YOLOv7 | 89.3 | 68.17 | 50.74 |
| 2: DSConv-YOLOv7 | 88.5 | 66.84 | 54.23 |
| 3: EfficientNet-YOLOv7 | 90.2 | 56.75 | 68.24 |
| 4: EfficientNetV2-YOLOv7 | 91.6 | 51.32 | 82.65 |
Among the lightweight designs, EfficientNetV2-YOLOv7 achieves the smallest model size (51.32 MB) and the highest FPS (82.65), while maintaining a precision of 91.6%. In contrast, the simple DSConv substitution slightly reduces precision but still increases speed. The results confirm that selecting a more advanced lightweight backbone is beneficial for both size and speed.
Finally, we combined the best lightweight backbone (EfficientNetV2) with the accuracy improvement modules (SimAM, BiFPN, SIoU). The combined model is compared with the baseline and the previous improved model in Table 5.
| Model | mAP@.5 (%) | Model Size (MB) | FPS |
|---|---|---|---|
| Baseline YOLOv7 | 89.6 | 68.17 | 50.74 |
| Improved YOLOv7 (Chapter 3) | 96.1 | 67.86 | 51.36 |
| EfficientNetV2-YOLOv7 | 91.8 | 51.32 | 82.65 |
| Ours (Combined) | 96.7 | 49.47 | 84.31 |
The combined model achieves the highest mAP@.5 of 96.7% and the fastest inference speed of 84.31 FPS, while the model size is only 49.47 MB, which is about 72% of the original YOLOv7 size. This indicates that the combination of accuracy improvement and lightweight backbone can satisfy the real-time inspection requirements of large-scale photovoltaic plants using edge devices.
5. Conclusion and Future Work
This paper presents a comprehensive study on solar panel fault detection based on deep learning. The proposed improved YOLOv7 model introduces a parameter-free attention module (SimAM) to the backbone, a weighted bidirectional feature pyramid network (BiFPN) for feature fusion, and the SIoU loss function for optimized bounding box regression. The experimental results on a real-world solar panel fault dataset demonstrate that these improvements increase the mean average precision by 6.5 percentage points compared with the baseline YOLOv7, achieving 96.1% mAP@.5. To enable real-time inspection on edge devices, several lightweight backbone designs are explored. Replacing the original backbone with EfficientNetV2 reduces the model size to 51.32 MB and boosts the inference speed to 82.65 FPS while maintaining high precision. Combining the lightweight backbone with the accuracy enhancement modules yields a final model with 96.7% mAP@.5, 49.47 MB model size, and 84.31 FPS, which is well-suited for drone-based inspection of solar panels.
For future work, several directions can be pursued. First, more diverse fault datasets covering various weather conditions and panel types should be collected to further validate the robustness of the model. Second, model compression techniques such as pruning and quantization can be applied to reduce the model size even further. Third, more advanced attention mechanisms and feature fusion strategies could be investigated to push the accuracy boundary. Finally, the deployment on real UAV platforms with hardware acceleration (e.g., tensor cores or edge AI chips) should be tested to ensure practical viability.