As non-renewable energy resources continue to be consumed, the development and utilization of new energy sources have become an important direction of global energy strategy. Solar energy, as a clean and renewable energy source, has received extensive attention. Solar panels are the core components of photovoltaic power stations, and their operating condition directly affects the efficiency and safety of the entire power generation system. However, due to long-term exposure to harsh outdoor environments, solar panels are prone to various surface defects such as cracks, hot spots, and PID effects. These defects lead to a significant reduction in photoelectric conversion efficiency and may even cause safety accidents. In infrared thermal images, such defects usually appear as abnormal high-temperature regions, which provides a reliable basis for defect detection. In recent years, the rapid development of unmanned aerial vehicle (UAV) inspection technology and infrared thermal imaging technology has made intelligent defect detection for solar panels a research hotspot. Traditional manual inspection methods cannot meet the operational requirements of large-scale photovoltaic power stations, so automated and intelligent detection methods are urgently needed.
In this work, I propose an improved solar panel defect detection method based on the RT-DETR-R50 architecture. The method is specifically designed to handle the challenges of complex backgrounds in aerial infrared images of solar panels. The proposed approach integrates three main contributions. First, I introduce an EMA-PConv module that combines the efficient multi-head attention (EMA) mechanism with partial convolution (PConv). This module reduces the number of parameters and computational cost of the backbone network while maintaining a strong feature extraction capability. Second, I propose a multi-path downsampling enhancement module (MDEM) to replace the original max-pooling downsampling operation. MDEM captures both global and local contextual information through three parallel paths, thereby enhancing the representation of defect features and suppressing background interference. Third, I construct a multiscale feature adaptive pyramid network (MFAPN) to improve the fusion of features from different scales. MFAPN dynamically combines high-level semantic information with low-level texture details, which is crucial for detecting small and ambiguous defects. Extensive experiments on a real aerial infrared dataset demonstrate that the proposed method improves the mean average precision (mAP50) by 3.3 percentage points and mAP50@95 by 1.2 percentage points compared with the original RT-DETR-R50 model, while reducing the parameter count by 16.8%. The results show that the proposed method effectively solves the problem of missed detections in complex backgrounds.
1. Related Work
Defect detection in solar panels can be roughly divided into two categories: traditional image processing methods and deep learning based methods. Traditional methods often rely on thresholding, edge detection, and morphological operations. For example, an improved selfish herd optimization algorithm was used to enhance edge detection for hot spot recognition in infrared images. Another approach combined Otsu threshold segmentation with multispectral image fusion to reduce false alarms. However, these traditional methods suffer from low computational efficiency and poor adaptability to varying environmental conditions. They are also vulnerable to noise and uneven thermal distribution. As a result, their performance in real-world applications is often limited.
Deep learning methods have become the dominant approach for solar panel defect detection due to their powerful feature extraction and pattern recognition capabilities. Convolutional neural networks (CNNs) and transformer-based detectors have been widely studied. For example, SNCF-Net constructed a multi-scale feature correlation model to improve the detection of tiny defects in complex backgrounds. Another method used U2-Net to generate saliency maps and fused them with infrared images before feeding them into an attention-based detection network. The RCAG module employed residual channel-wise attention to suppress irrelevant background information and enhance defect features. Some methods improved image quality before detection, such as using an improved MSRCR algorithm to enhance the dark regions of infrared images. Homomorphic filtering based on an improved transfer function has also been applied to increase detection accuracy. Real-time models such as CEMP-YOLOv10n and FAA-YOLO have been proposed to balance lightweight design and detection speed. Nevertheless, these methods still face difficulties in extracting robust features under highly complex backgrounds, leading to missed or false detections. The RT-DETR family has shown strong performance in real-time object detection, but its backbone may lose important contextual information during downsampling. In this paper, I aim to address these limitations by redesigning the downsampling and feature fusion stages of RT-DETR-R50.
2. Proposed Method
The overall architecture of the proposed model, referred to as MPMA-DETR, is built upon the RT-DETR-R50 framework. It consists of three main components: a backbone network, a hybrid encoder, and a transformer decoder. The backbone extracts hierarchical features from the input infrared image. The hybrid encoder enhances these features through scale-internal feature interaction and cross-scale fusion. The decoder then uses IoU-aware query selection and iterative refinement to produce final detection results. In the proposed model, the original BasicBlock module in the backbone is replaced with the EMA-PConv module. The max-pooling downsampling operation is replaced by the MDEM block. In addition, the original PAN-FPN in the encoder is replaced with the proposed MFAPN module. These modifications improve the accuracy and efficiency of the model for solar panel defect detection.
2.1 EMA-PConv Module
To reduce parameter redundancy and computational complexity, I introduce a lightweight residual module named EMA-PConv. The core idea is to apply partial convolution (PConv) only to a subset of the input channels instead of all channels. PConv has been shown to effectively reduce floating-point operations while preserving informative features. Specifically, only one quarter of the input channels are convolved with a 3×3 kernel, while the remaining channels remain unchanged. This design is motivated by the observation that image features often exhibit high redundancy along the channel dimension. By convolving only a part of the channels, the model can reduce redundant calculations while retaining key features. The unchanged channels preserve the original semantic structure, which helps the subsequent attention mechanism focus on important target regions.
To further enhance the representation capability, I combine PConv with the efficient multi-head attention (EMA) mechanism. EMA divides the input feature map into groups along the channel dimension and computes attention within each group. It consists of two parallel branches: a spatial branch and a local branch. The spatial branch performs global average pooling along the height and width directions, followed by a 1×1 convolution to generate spatial attention weights. The local branch uses a 3×3 convolution to extract fine-grained local information. The outputs of these two branches are then combined through matrix multiplication and a sigmoid activation to produce the final attention weights. This mechanism helps the network focus on defect regions while suppressing background noise. The entire EMA-PConv module is formulated as follows:
$$ Z = \text{EMA}\left( \text{ReLU}\left( \text{LayerNorm}\left( \text{PConv}_{3\times3}\left(Z_{C/4}\right) + Z_{3C/4} \right) \right) \right) $$
where \(Z_{C/4}\) denotes the first quarter of the channels and \(Z_{3C/4}\) denotes the remaining three quarters. The EMA computation can be written as:
$$ Y = X_{\text{Group}} \times \sigma\left\{ \text{MatMul}\left(\text{Softmax}\left(\text{AvgPool}(X_1)\right), X_2\right) + \text{MatMul}\left(\text{Softmax}\left(\text{AvgPool}(X_2)\right), X_1\right) \right\} $$
Here, \(X_{\text{Group}}\) is the reshaped feature map with dimensions \(B \times G \times C/G \times H \times W\). \(X_1\) and \(X_2\) are the outputs of the spatial and local branches, respectively. The notation \(\text{MatMul}\) denotes matrix multiplication, and \(\sigma\) is the sigmoid activation. The module also uses the DropPath regularization strategy in the residual path to improve training stability and prevent overfitting. The EMA-PConv module is used to replace the original BasicBlock in the backbone, resulting in a significant reduction in the parameter count and GFLOPs.
2.2 Multi-Path Downsampling Enhancement Module (MDEM)
The original RT-DETR model uses standard max-pooling for downsampling, which may ignore fine-grained subject features and lose important global context information. In infrared images of solar panels, defect areas often have low contrast against the background, and traditional downsampling can easily blur or even lose these subtle patterns. To address this issue, I design a multi-path downsampling enhancement module (MDEM). MDEM performs feature extraction and downsampling in a dual-stage manner, integrating multiple parallel paths with different receptive fields and functions.
In the first stage, a 7×7 convolution is applied to the input feature map to extract initial features, reducing the channel dimension to C/4 while keeping the spatial resolution unchanged. This initial feature is denoted as \(X_{\text{init}}\). Then, two parallel paths are used. The stride path uses a stride-2 convolution to downsample the feature map by half and increase the channel dimension to C/2. The depth path uses depthwise separable convolution to extract local detailed features and also expands the channels to C/2. The outputs of these two paths are concatenated along the channel dimension and then fused through a 1×1 convolution followed by batch normalization. In this stage, max-pooling is not used because the shallow feature maps still contain a lot of noise, and max-pooling would preserve that noise and hinder subsequent feature learning. The operations are formulated as follows:
$$ X_{\text{init}} = \text{Conv}_{7\times7}\left( X \right) \in \mathbb{R}^{C/4 \times H \times W} $$
$$ F = \text{Conv}_{1\times1}\left( \text{Concat}\left( \text{Conv}_{3\times3}\left(X_{\text{init}}\right), \text{DWConv}_{3\times3}\left(X_{\text{init}}\right) \right) \right) $$
where \(\text{DWConv}_{3\times3}\) represents a depthwise separable convolution.
In the second stage, three parallel modalities are used. The depth path continues to use depthwise separable convolution to expand the channel dimension to C while further reducing the spatial size. The stride path applies a stride convolution to downsample the feature map while preserving the original information. The newly added pooling path uses max-pooling to extract high-response region features. Finally, the outputs of the three paths are concatenated and fused through a 1×1 convolution. This fusion operation performs implicit channel-wise weighting through learnable parameters, enabling the model to adaptively balance the importance of different paths. The entire process can be expressed as:
$$ V = \text{BN}\left( \text{Conv}_{1\times1}\left( \text{Concat}\left( \text{Conv}_{3\times3}\left(F\right), \text{DWConv}_{3\times3}\left(F\right), \text{MaxPool}\left(F\right) \right) \right) \right) $$
By integrating these three branches, MDEM not only preserves detailed spatial information but also captures global contextual knowledge, leading to a stronger discriminative ability in complex backgrounds. The proposed MDEM replaces the original max-pooling downsampling layer in RT-DETR-R50 and significantly improves the quality of the feature representations.
2.3 Multiscale Feature Adaptive Pyramid Network (MFAPN)
Infrared solar panel images often contain defects of varying sizes, and the difference between the target and the background is sometimes very subtle. It is challenging for a single-scale feature to capture both fine details and global semantics. To address this, I propose a multiscale feature adaptive pyramid network (MFAPN) to enhance multi-scale feature fusion. The input to MFAPN consists of three feature maps with different resolutions, denoted as \(X_L\) (large receptive field), \(X_M\) (medium receptive field), and \(X_S\) (small receptive field). These feature maps have different spatial sizes.
First, the three feature maps are adjusted to the same spatial resolution. For the large feature map \(X_L\), I apply both adaptive max pooling and adaptive average pooling to reduce its size to match \(X_M\), and then sum the two pooled feature maps. This operation combines the advantages of max pooling (preserving strong responses) and average pooling (preserving global context). The resulting feature map is expressed as:
$$ X’_L = \text{MaxPool}\left(X_L\right) + \text{AdaPool}\left(X_L\right) $$
For the small feature map \(X_S\), I use nearest-neighbor interpolation to upsample it to the same spatial size as \(X_M\):
$$ X’_S = \text{Upsample}\left(X_S\right) $$
After adjusting all feature maps to the same resolution, they are concatenated along the channel dimension:
$$ Y = \text{Concat}\left( X’_L, X_M, X’_S \right) $$
The concatenation order is carefully chosen as large, medium, and small. This order follows a semantic-strength-priority strategy, where the high-level features with larger receptive fields are placed first to provide a stable semantic foundation, followed by medium and then low-level details. This helps mitigate the semantic inconsistency among different scales and improves the discriminative power of the fused features.
To further exploit the correlations among different scales, I reshape the concatenated feature vector into a 5D tensor by stacking the features along the scale dimension:
$$ F = \text{Reshape}(Y) \in \mathbb{R}^{B \times C \times 3 \times H \times W} $$
Here, the third dimension represents the scale sequence. A 3D convolution with a 1×1×1 kernel is then applied to perform feature interaction across scales. This operation simulates the process of capturing temporal dependencies in sequence modeling, but in the scale dimension. After batch normalization and ReLU activation, a 3D max pooling with kernel size 3×1×1 is used to compress the scale dimension and produce the final output feature vector:
$$ F_{\text{out}} = \text{MaxPool}_{3\times1\times1}\left( \text{ReLU}\left( \text{BN}\left( \text{Conv}_{1\times1\times1}\left(F\right) \right) \right) \right) $$
This design enables the network to dynamically integrate multi-scale information and enrich the semantic representation of defect features. MFAPN is used in the hybrid encoder to replace the original cross-scale feature fusion module, leading to better detection performance for solar panel defects.
3. Experimental Setup
All experiments are conducted on an NVIDIA 4080 GPU with Python 3.10 and PyTorch 1.12.1. To ensure a fair comparison, none of the models use pretrained weights. The batch size is set to 8, and the number of training epochs is 200. The initial learning rate is 0.0001 with an early stopping patience of 50 epochs. Other training hyperparameters keep their default values from the original RT-DETR implementation. I use the standard stochastic gradient descent optimizer with a warm-up schedule. All images are resized to 640×640 before being fed into the models.
3.1 Dataset
The experimental dataset is collected from a photovoltaic power plant using a UAV equipped with an infrared camera. The dataset contains 3,694 infrared images with a spatial resolution of 640×640. There are four types of solar panel defects: diode short circuit (hotspot), PID effect (golden-spot), slight cracks (light-golden-spot), and occlusion (shadow). The images are annotated using the Labelme tool. The dataset is split into a training set, a validation set, and a test set. Specifically, 2,586 images are used for training, 382 for validation, and 726 for testing, corresponding to a ratio of approximately 7:1:2. This dataset covers a wide range of complex backgrounds, including varying illumination, different shooting angles, and diverse terrain, making it a challenging benchmark for evaluating the robustness of the proposed method.
3.2 Evaluation Metrics
I use precision (P), recall (R), mean average precision (mAP), and mAP50@95 as the primary accuracy metrics. Precision indicates the proportion of correctly predicted positive samples among all predicted positives, and recall indicates the proportion of correctly predicted positive samples among all true positives. They are defined as:
$$ P = \frac{TP}{TP + FP} $$
$$ R = \frac{TP}{TP + FN} $$
The average precision (AP) for a single class is the area under the precision-recall curve. The mean average precision (mAP) is the average of AP over all classes. The mAP50 metric uses an IoU threshold of 0.5, while mAP50@95 averages the AP over IoU thresholds from 0.5 to 0.95 with a step of 0.05. In addition to accuracy, I also report the number of parameters, GFLOPs, model weight size, and inference speed in FPS to evaluate the computational efficiency of the models.
4. Ablation Studies
To validate the effectiveness of each component in the proposed method, I conducted a series of ablation experiments on the test set. The results are presented in Table 1. The baseline model is RT-DETR-R50. I incrementally add the EMA-PConv module, the MFAPN module, and the MDEM module to observe their contributions.
| ID | EMA-PConv | MFAPN | MDEM | P (%) | R (%) | mAP50 (%) | Parameters (107) | GFLOPs | FPS |
|---|---|---|---|---|---|---|---|---|---|
| I | — | — | — | 73.7 | 75.9 | 75.0 | 4.30 | 129.6 | 100 |
| II | √ | — | — | 72.3 | 68.7 | 73.3 | 1.69 | 51.5 | 151 |
| III | — | √ | — | 73.3 | 80.4 | 76.9 | 4.34 | 142.2 | 88 |
| IV | — | — | √ | 74.5 | 83.4 | 76.4 | 4.31 | 50.2 | 154 |
| V | √ | √ | — | 75.1 | 75.3 | 76.6 | 3.57 | 131.2 | 72 |
| VI | √ | — | √ | 75.1 | 73.7 | 75.5 | 3.47 | 47.5 | 70 |
| VII | — | √ | √ | 75.7 | 76.4 | 75.8 | 4.32 | 53.4 | 149 |
| VIII | √ | √ | √ | 76.3 | 79.6 | 78.3 | 3.58 | 50.6 | 135 |
From Table 1, several observations can be made. The EMA-PConv module alone reduces the parameter count from 4.30×107 to 1.69×107, a reduction of 60.7%, but it also causes a decrease in mAP50 from 75.0% to 73.3%. This is because the lightweight module has a weaker capacity for capturing long-range dependencies and may lose some semantic information in the feature extraction process. However, when MDEM is added to compensate for this loss, the mAP50 increases to 75.5%. The combination of EMA-PConv and MFAPN yields 76.6%, and the full model with all three modules achieves the highest mAP50 of 78.3% while keeping the parameter count at 3.58×107, which is 16.8% lower than the baseline. GFLOPs are also reduced from 129.6 to 50.6, showing a significant efficiency improvement. These results confirm that the proposed modules are complementary and collectively improve the detection performance.
4.1 Impact of Channel Convolution Ratio in EMA-PConv
In the EMA-PConv module, the ratio of channels that actually participate in the convolution operation is a hyperparameter. I tested three ratios: 1/2, 1/4, and 1/8. The results are shown in Table 2. As the ratio decreases, the model becomes more compact, with lower GFLOPs and higher FPS. However, the mAP50 is maximized at the ratio of 1/4, reaching 73.3%. A ratio of 1/2 introduces more redundancy and slightly reduces the mAP to 73.1%. A ratio of 1/8 provides too few channels for convolution, resulting in insufficient feature expressiveness and a drop to 72.9%. Therefore, the ratio of 1/4 is chosen as the default setting for all experiments, as it achieves the best trade-off between accuracy and efficiency.
| Ratio | P (%) | R (%) | mAP50 (%) | Parameters (107) | GFLOPs | FPS |
|---|---|---|---|---|---|---|
| 1/2 | 72.1 | 67.6 | 73.1 | 1.81 | 54.3 | 149 |
| 1/4 | 72.3 | 68.7 | 73.3 | 1.69 | 51.5 | 151 |
| 1/8 | 69.2 | 67.4 | 72.9 | 1.66 | 50.7 | 154 |
4.2 Impact of Feature Concatenation Order in MFAPN
The MFAPN module concatenates feature maps from different scales. The order of concatenation can affect the fusion quality. I compared six possible orders: L-M-S, L-S-M, M-L-S, M-S-L, S-L-M, and S-M-L. The results are listed in Table 3. The L-M-S order achieves the highest mAP50 of 76.9%, while the S-L-M order yields the lowest mAP50 of 73.6%. This indicates that placing the large receptive field feature first helps build a global-to-local semantic hierarchy, which improves the model’s ability to identify objects in complex backgrounds. All orders have similar parameter counts, GFLOPs, and FPS, so the differences are solely attributed to the concatenation strategy. Therefore, L-M-S is used as the default order in MFAPN.
| Order | P (%) | R (%) | mAP50 (%) | Parameters (107) | GFLOPs | FPS |
|---|---|---|---|---|---|---|
| L, M, S | 73.3 | 80.4 | 76.9 | 4.34 | 142.2 | 88 |
| L, S, M | 73.1 | 75.1 | 76.6 | 4.34 | 142.2 | 88 |
| M, L, S | 72.8 | 76.0 | 74.3 | 4.31 | 142.2 | 87 |
| M, S, L | 72.9 | 73.6 | 75.2 | 4.31 | 142.2 | 87 |
| S, L, M | 71.5 | 78.5 | 73.6 | 4.31 | 142.2 | 88 |
| S, M, L | 68.8 | 78.0 | 74.1 | 4.31 | 142.2 | 88 |
5. Comparison with State-of-the-Art Methods
To prove the superiority of the proposed method, I compare it with several popular object detection algorithms, including YOLOv10x, YOLOv11, YOLOv12, Faster-RCNN, RT-DETR-R18, RT-DETR-R101, and DE-DETR. All models are trained under the same experimental conditions. The results are presented in Table 4. For a fair comparison, I report precision (P), recall (R), mAP50, mAP50@95, parameters, GFLOPs, weight size, and FPS.
| Model | P (%) | R (%) | mAP50 (%) | mAP50@95 (%) | Parameters (106) | GFLOPs | Weight (MB) | FPS |
|---|---|---|---|---|---|---|---|---|
| RT-DETR-R50 | 73.7 | 75.9 | 75.0 | 44.3 | 43.0 | 129.6 | 86.1 | 100 |
| RT-DETR-R18 | 72.4 | 71.9 | 71.5 | 41.6 | 20.0 | 57.0 | 40.5 | 192 |
| RT-DETR-R101 | 73.4 | 82.8 | 74.4 | 44.4 | 74.0 | 247.1 | 153.8 | 71 |
| Faster-RCNN | 68.3 | 71.3 | 71.9 | 40.3 | 41.3 | 133.9 | 161.9 | 42 |
| YOLOv10x | 75.3 | 68.4 | 75.3 | 44.8 | 31.6 | 169.8 | 64.1 | 161 |
| YOLOv11 | 75.2 | 79.4 | 76.4 | 45.1 | 2.6 | 6.3 | 5.5 | 435 |
| YOLOv12 | 73.6 | 74.7 | 73.4 | 42.0 | 2.7 | 6.4 | 5.6 | 357 |
| DE-DETR | 71.2 | 74.3 | 72.2 | 41.9 | 40.3 | 86.0 | 175.0 | 167 |
| Ours | 76.3 | 79.6 | 78.3 | 45.5 | 35.8 | 50.6 | 76.1 | 135 |
From Table 4, the proposed method achieves the highest precision of 76.3% and the highest mAP50 of 78.3%. Compared with the original RT-DETR-R50, the proposed method improves mAP50 by 3.3 percentage points and mAP50@95 by 1.2 percentage points. It also outperforms RT-DETR-R101, which has nearly twice the parameter count, by 3.9 percentage points in mAP50. This suggests that the proposed feature extraction and fusion strategies are more effective for detecting solar panel defects in complex backgrounds. The proposed method also provides a lower computational cost than many competitors, with only 50.6 GFLOPs, which is less than half of RT-DETR-R50’s 129.6 GFLOPs and only one fifth of RT-DETR-R101’s 247.1 GFLOPs. The model size is 76.1 MB, smaller than RT-DETR-R50’s 86.1 MB. The inference speed reaches 135 FPS, which is sufficient for real-time UAV inspection scenarios. YOLOv11 and YOLOv12 are much lighter and faster, but their detection accuracy is lower than the proposed method, especially in terms of P and mAP50. Overall, the proposed model achieves a strong balance between accuracy, model complexity, and inference speed, making it highly suitable for practical solar panel inspection tasks.
6. Visualization Analysis
To better understand the behavior of the proposed model, I use Grad-CAM to generate class activation heatmaps for the last multi-head attention module in the transformer decoder. The heatmaps reveal which regions of the input image the model focuses on when making predictions. I compare the heatmaps of the original RT-DETR-R50, the model with only MDEM (RT-DETR-MDEM), and the full proposed model.
In the heatmap results, the original RT-DETR-R50 often spreads its attention over large non-target areas, indicating that it is disturbed by background noise. For example, in one image, the original model completely fails to focus on the defect region. The model with MDEM, on the other hand, shows more concentrated attention on the actual defect area, while suppressing the background response. The full proposed model further improves the focus, producing sharp and well-localized heatmaps that align accurately with the defect boundaries. In the bounding-box visualizations, the original model sometimes produces false detections caused by background heat, whereas the proposed model reliably detects all defects with higher confidence. These qualitative results support the quantitative improvements and demonstrate that the proposed modules help the network learn more discriminative features for solar panel defects.
I also observe a limitation of the MDEM-only model: in some scenes with highly cluttered backgrounds, MDEM does not fully suppress the noise, because the features extracted by the parallel paths may not be perfectly aligned in the semantic space. However, when combined with MFAPN, the model can leverage multi-scale interactions to refine the feature representations and achieve stronger noise suppression. This further justifies the complementary nature of the proposed modules.
7. Conclusion
In this paper, I proposed an improved solar panel defect detection method based on RT-DETR-R50, which combines multiple technical contributions to address the challenges of complex backgrounds and small defect areas in infrared images. The EMA-PConv module reduces the model parameters and computational cost while preserving essential features through partial convolution and efficient multi-head attention. The multi-path downsampling enhancement module (MDEM) effectively captures both global and local information and suppresses background interference by fusing multiple downsampling paths. The multiscale feature adaptive pyramid network (MFAPN) improves the interaction among features of different scales, enriching the semantic representation and boosting detection accuracy. Extensive ablation studies and comparisons with state-of-the-art methods demonstrate that the proposed method achieves superior mAP50 and mAP50@95 values while maintaining lower model complexity and competitive inference speed. The visualization analysis further confirms that the model focuses more precisely on real defect regions and is less distracted by complex backgrounds. The proposed approach provides a promising solution for automatic and efficient solar panel inspection in large-scale photovoltaic power plants.