In this work, we present a comprehensive study on a novel defect detection method for solar panels based on an enhanced RT-DETR architecture. Our approach targets the challenges of low accuracy, large model size, and missed or false detections under complex backgrounds, which are common in traditional detection pipelines for solar panels. We propose three key improvements: a novel backbone named FIREBlock for efficient feature extraction, a multi-scale feature fusion structure called CRDFP, and a deformable attention mechanism (DAttention) to focus on relevant regions. Extensive experiments demonstrate that our method achieves superior detection performance while significantly reducing computational cost. The results show that the enhanced model reaches 79.2% mean average precision (mAP), which is 3.6 percentage points higher than the baseline RT-DETR, with 22.6% fewer parameters and 25.9% lower floating-point operations (FLOPs). Our work contributes to the deployment of efficient and accurate defect detection systems for solar panels in real-world photovoltaic power plants.
1. Introduction
The global demand for renewable energy has accelerated the adoption of photovoltaic (PV) systems. Solar panels, as the core component, are susceptible to various defects such as dust accumulation, bird droppings, cracks, hot spots, and other anomalies. These defects not only reduce energy conversion efficiency but also pose potential safety hazards. Timely and efficient solar panel defect detection is critical for maintaining the reliability and longevity of PV systems.
Traditional inspection methods, including manual inspection and electrical characteristic monitoring, suffer from inefficiency, high cost, and lack of flexibility. Computer vision-based methods have become dominant due to their low cost and real-time capability. Convolutional neural networks (CNNs), such as YOLO series and Faster R-CNN, have been widely used for solar panel defect detection. However, these methods typically require non-maximum suppression (NMS) post-processing, which deviates from true end-to-end detection. The emergence of Transformer-based detectors, such as DETR and its variants, eliminates NMS by treating object detection as a set prediction problem. Among them, RT-DETR (Real-Time Detection Transformer) achieves a good balance between speed and accuracy. Nevertheless, for solar panel defects, the original RT-DETR still faces challenges in detecting small objects, handling complex backgrounds, and maintaining efficiency in resource-constrained environments.
In this paper, we propose an improved RT-DETR framework tailored for solar panel defect detection. Our contributions are threefold:
- We design a novel backbone, FIREBlock, which integrates reparameterization (RepConv) and an efficient multi-scale attention module (EMA) to enhance feature extraction while reducing model complexity.
- We develop a multi-scale feature fusion structure named CRDFP (Context Reconstruction and Dynamic Fusion Pyramid), which leverages rectangular self-calibration modules (RCM) and dynamic interpolation fusion (DIF) to improve context-aware representation.
- We introduce a deformable attention mechanism (DAttention) in the encoder module to focus on informative regions, thereby improving detection accuracy for small and complex defects on solar panels.
The experimental results on a collected dataset of 4,271 images (containing three defect types: hot spot, dirt, and damage) and on the public PVEL-AD dataset demonstrate that our method outperforms state-of-the-art approaches in both accuracy and efficiency. The rest of the paper is organized as follows: Section 2 details the proposed FCD-DETR network architecture. Section 3 presents the experimental setup, ablation studies, and comparison results. Section 4 concludes the paper.
2. Methodology
2.1 Overall Architecture
The overall structure of our proposed FCD-DETR is shown in Figure 1 of the original paper. It consists of three main components: (1) a feature extraction backbone using FIREBlock, (2) a hybrid encoder composed of a modified AIFI (Attention-based Intrascale Feature Interaction) module with deformable attention and a CRDFP-based cross-scale feature fusion module, and (3) a Transformer decoder with auxiliary prediction heads. In the following subsections, we elaborate on each improvement.
2.2 FIREBlock: Efficient Feature Extraction Backbone
The traditional BasicBlock used in ResNet is suboptimal for capturing fine-grained features of solar panel defects. To address this, we propose FIREBlock, which combines reparameterized partial convolution (RPCConv) with an efficient multi-scale attention mechanism (EMA). The design is inspired by FasterNet, which uses partial convolution (PConv) to reduce redundant computation. However, PConv alone may lose important features in complex natural scenes. Therefore, we introduce reparameterization to merge the multi-branch structure during inference, improving both performance and speed.
2.2.1 RPCConv Module
PConv applies standard convolution on only a subset of input channels, while the rest remain unchanged. The FLOPs for a PConv layer are:
$$ \text{FLOPs} = h \times w \times k^2 \times c_p^2, $$
where \(h\) and \(w\) are spatial dimensions, \(k\) is the kernel size, and \(c_p\) is the number of processed channels. Typically, \(c_p = c/4\), leading to a 16× reduction in FLOPs compared to standard convolution. To further enhance representational power, we adopt structural reparameterization (RepConv) that fuses the 1×1 and 3×3 convolutions with batch normalization (BN) during training, then merges them into a single convolution at inference. The BN fusion formula is:
$$ \hat{x}_i = \gamma \cdot \frac{x_i – \mu}{\sqrt{\sigma^2 + \epsilon}} + \beta = \frac{\gamma}{\sqrt{\sigma^2 + \epsilon}} \cdot x_i + \left( \beta – \frac{\gamma \cdot \mu}{\sqrt{\sigma^2 + \epsilon}} \right). $$
After fusion, the combined weight and bias become:
$$ w = w_{\text{BN}} \cdot w_{\text{conv}}, \quad b = w_{\text{BN}} \cdot b_{\text{conv}} + b_{\text{BN}}, $$
where \(w_{\text{BN}} = 1/\sqrt{\sigma^2+\epsilon}\) and \(b_{\text{BN}} = \beta – \gamma\mu/\sqrt{\sigma^2+\epsilon}\). This reduces the computational overhead caused by additional branches.
2.2.2 EMA Attention Module
To better capture multi-scale contextual information, we insert an efficient multi-scale attention (EMA) module within FIREBlock. EMA splits the input into G groups, applies 1D horizontal and vertical global pooling, and generates spatial attention maps through matrix dot product operations. The complexity is reduced from \(O(C^2 HW)\) to \(O((C^2/G) HW)\). The module also uses Sigmoid activation to highlight object regions and suppress background noise. Figure 2 in the original paper illustrates the structure of FIREBlock and its subcomponents.
The complete FIREBlock is formulated as:
$$ \mathbf{x}_{\text{out}} = \text{EMA}(\text{RPCConv}(\mathbf{x}_{\text{in}})) + \mathbf{x}_{\text{in}}, $$
where the residual connection ensures gradient flow and feature reuse.
2.3 CRDFP: Multi-Scale Feature Fusion
In the original CCFFM (Cross-Scale Feature Fusion Module), simple top-down and bottom-up pathways may cause conflicts between shallow detail features and deep semantic features, especially under complex backgrounds. To overcome this, we propose CRDFP, which integrates a pyramid context extraction (PCE) module with dynamic interpolation fusion (DIF) and multi-fusion blocks (MFB).
2.3.1 RCM and PCE Modules
The rectangular self-calibration module (RCM) consists of rectangular self-calibration attention (RCA), batch normalization (BN), and MLP. RCA captures global context through horizontal and vertical pooling, generating two axial vectors that are added via broadcast to model rectangular regions of interest. This process enhances the model’s sensitivity to foreground objects. The PCE module stacks multiple RCMs with parallel patch-aware attention (PPA) to aggregate multi-scale context. The output of PCE is a feature map enriched with contextual information.
2.3.2 DIF and MFB for Feature Fusion
The Dynamic Interpolation Fusion (DIF) module handles cross-scale feature alignment. Given two feature maps \(\mathbf{X}_1\) and \(\mathbf{X}_2\) from different scales, DIF first resizes \(\mathbf{X}_2\) to match \(\mathbf{X}_1\) using bilinear interpolation, then applies a convolution layer to produce the fused feature:
$$ \mathbf{X}_{\text{fuse}} = \text{Conv}\big( \text{Upsample}(\mathbf{X}_2) + \mathbf{X}_1 \big). $$
The Multi-Fusion Block (MFB) further combines low-frequency (global) and high-frequency (detail) features. It processes each branch with separate convolutions, adjusts the high-frequency branch via bilinear upsampling, and fuses them with an activation function:
$$ \mathbf{X}_{\text{MFB}} = \text{ReLU}\big( \text{Conv}_{\text{low}}(\mathbf{X}_{\text{low}}) + \text{Upsample}(\text{Conv}_{\text{high}}(\mathbf{X}_{\text{high}})) \big). $$
The overall CRDFP pyramid structure integrates PCE, DIF, and MFB in a hierarchical manner, allowing the model to adaptively reweight features from different receptive fields. The architecture is illustrated in Figure 6 and Figure 7 of the original paper.
2.4 Deformable Attention in AIFI
The original AIFI module in RT-DETR uses standard multi-head self-attention (MHSA), which processes all pixels uniformly, leading to high computational cost and limited ability to handle cluttered backgrounds. We replace MHSA with deformable attention (DAttention), which dynamically selects a sparse set of key sampling points for each query. This focuses the model on the most informative regions, reducing FLOPs while improving accuracy.
Given an input feature map \(\mathbf{x}\), we compute query embeddings \(\mathbf{q} = \mathbf{x} \mathbf{W}_q\). A lightweight subnetwork \(\theta_{\text{offset}}\) predicts 2D offsets \(\Delta \mathbf{p}\) for each reference point:
$$ \Delta \mathbf{p} = \theta_{\text{offset}}(\mathbf{q}). $$
The deformed key and value embeddings are sampled via bilinear interpolation:
$$ \tilde{\mathbf{x}} = \phi(\mathbf{x}; \mathbf{p} + \Delta \mathbf{p}), \quad \tilde{\mathbf{k}} = \tilde{\mathbf{x}} \mathbf{W}_k, \quad \tilde{\mathbf{v}} = \tilde{\mathbf{x}} \mathbf{W}_v. $$
Then, the attention head output is:
$$ \mathbf{z}^{(m)} = \sigma\left( \frac{\mathbf{q}^{(m)} \tilde{\mathbf{k}}^{(m)\text{T}}}{\sqrt{d}} + \phi(\hat{\mathbf{B}}; \mathbf{R}) \right) \tilde{\mathbf{v}}^{(m)}, $$
where \(\sigma\) is Softmax, \(d\) is the head dimension, and \(\phi(\hat{\mathbf{B}}; \mathbf{R})\) is a relative position bias term computed from the deformed points. The outputs from all heads are concatenated and linearly projected to obtain the final feature.
We incorporate this deformable attention into the single-scale encoder (AIFI) to suppress background interference and enhance local detail capture for solar panel defects.
3. Experiments and Results
3.1 Dataset and Implementation Details
We constructed a dataset for solar panel defect detection by collecting and filtering images from multiple public sources, including the Flying Paddle dataset, Roboflow, and PV-HSD-2025. The final dataset comprises 4,271 images containing three defect categories: hot spot, dirt, and damage. We split the dataset into training, validation, and test sets in an 8:1:1 ratio. All images were resized to 640×640 pixels. The training was conducted for 200 epochs with a batch size of 8 and 4 worker threads on an NVIDIA RTX 4070 Super GPU. The experimental environment is summarized in Table 1.
| Parameter | Configuration |
|---|---|
| Operating System | Windows 10 |
| GPU | NVIDIA GeForce RTX 4070 Super |
| CPU | Intel Core i5-13400F |
| Memory | 12 GiB |
| Python Version | 3.9 |
| Framework | PyTorch |
3.2 Evaluation Metrics
We adopt the following metrics to evaluate our method: precision (\(P\)), recall (\(R\)), mean average precision at IoU=0.5 (mAP@0.5), number of parameters (Params), and floating-point operations (FLOPs). The formulas are:
$$ P = \frac{TP}{TP + FP}, \quad R = \frac{TP}{TP + FN}, \quad \text{mAP} = \frac{1}{N}\sum_{c=1}^{N} AP_c, $$
where \(AP_c\) is the area under the precision-recall curve for class \(c\).
3.3 Ablation Study
We conduct ablation experiments to validate the contribution of each proposed component. The baseline is the original RT-DETR. Results are shown in Table 2.
| Exp | Baseline | FIREBlock | CRDFP | DAttention | P (%) | R (%) | FLOPs (×10^9) | Params (×10^6) | mAP (%) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | √ | 76.6 | 68.3 | 58.3 | 20.8 | 75.6 | |||
| 2 | √ | 78.1 | 70.6 | 52.0 | 17.1 | 76.7 | |||
| 3 | √ | 78.4 | 71.2 | 48.6 | 19.3 | 77.3 | |||
| 4 | √ | 79.8 | 71.9 | 58.5 | 20.0 | 77.5 | |||
| 5 | √ | √ | 81.6 | 73.4 | 43.3 | 16.4 | 78.5 | ||
| 6 | √ | √ | 81.5 | 73.7 | 49.1 | 18.6 | 78.7 | ||
| 7 | √ | √ | √ | 82.3 | 74.2 | 43.2 | 16.1 | 79.2 |
From Table 2, we observe: (1) Using FIREBlock alone (Exp 2) increases mAP by 1.1% while reducing parameters by 17.8% and FLOPs by 10.8%. (2) Adding CRDFP (Exp 3) or DAttention (Exp 4) individually improves mAP by 1.7% and 1.9%, respectively, with slight changes in model size. (3) Combining FIREBlock with either CRDFP or DAttention yields further gains (mAP 78.5% and 78.7%). (4) The full model (Exp 7) achieves the best mAP of 79.2%, which is 3.6% higher than the baseline, while reducing FLOPs by 25.9% and parameters by 22.6%. This confirms that each component synergistically contributes to both accuracy and efficiency.
3.4 Comparison with State-of-the-Art Methods
We compare our FCD-DETR with several advanced detectors, including Faster R-CNN, YOLOv5m, YOLOv5l, YOLOv8m, YOLOv8l, YOLOv10m, YOLOv11m, YOLOv12m, Deformable DETR, and RT-DETR. The results are listed in Table 3.
| Model | P (%) | R (%) | Params (×10^6) | FLOPs (×10^9) | mAP@0.5 (%) |
|---|---|---|---|---|---|
| Faster R-CNN | 63.9 | 58.4 | 137.1 | 303.1 | 63.6 |
| YOLOv5m | 71.9 | 63.1 | 21.3 | 64.1 | 71.2 |
| YOLOv5l | 72.8 | 64.5 | 46.2 | 78.7 | 72.0 |
| YOLOv8m | 74.5 | 67.2 | 25.8 | 59.0 | 74.3 |
| YOLOv8l | 75.2 | 68.9 | 43.4 | 91.6 | 75.2 |
| YOLOv10m | 75.4 | 69.3 | 17.2 | 63.6 | 75.3 |
| YOLOv11m | 75.7 | 69.6 | 20.6 | 67.7 | 75.5 |
| YOLOv12m | 75.9 | 69.6 | 20.1 | 68.2 | 74.9 |
| Deformable DETR | 74.6 | 68.4 | 39.8 | 196.1 | 75.1 |
| RT-DETR | 75.9 | 69.8 | 20.8 | 58.2 | 75.6 |
| FCD-DETR (Ours) | 82.3 | 74.2 | 16.1 | 43.2 | 79.2 |
Our FCD-DETR achieves the highest mAP (79.2%), precision (82.3%), and recall (74.2%) among all methods. Notably, it surpasses the baseline RT-DETR by 3.6% in mAP while using 22.6% fewer parameters and 25.8% less FLOPs. Compared to YOLOv8l and YOLOv10m, our model obtains better accuracy with significantly lower computational cost. The Deformable DETR, which also uses a transformer backbone, performs worse than our model in both accuracy and efficiency. This demonstrates the effectiveness of our combined improvements for solar panel defect detection.
3.5 Generalization Experiment on PVEL-AD Dataset
To evaluate the generalization ability of our method, we test on the public PVEL-AD dataset, which contains 3,812 infrared images of solar panels with seven defect categories (e.g., black core, crack, finger, short circuit, star crack, thick line, horizontal dislocation). The results are shown in Table 4.
| Model | P (%) | R (%) | mAP@0.5 (%) | Params (×10^6) |
|---|---|---|---|---|
| RT-DETR | 66.6 | 69.7 | 74.4 | 20.8 |
| FCD-DETR (Ours) | 76.7 | 72.9 | 80.8 | 16.1 |
Our FCD-DETR again outperforms RT-DETR by a large margin: mAP improves from 74.4% to 80.8%, with a 6.4% increase, while parameter count is reduced by 22.6%. This demonstrates that our improvements generalize well across different datasets and defect types of solar panels. Visual comparisons (shown in Figure 9 and Figure 10 of the original paper) indicate that our model produces fewer false positives and fewer missed detections, especially for small and low-contrast defects.
4. Conclusion
In this paper, we have presented an improved RT-DETR framework called FCD-DETR for accurate and efficient defect detection of solar panels. We introduced three key innovations: (1) a lightweight yet powerful FIREBlock backbone that combines reparameterized convolutions with efficient multi-scale attention; (2) a CRDFP feature fusion module that incorporates rectangular self-calibration and dynamic interpolation to capture multi-scale context; and (3) a deformable attention mechanism in the encoder to focus on critical image regions. Extensive experiments on both our collected dataset and the public PVEL-AD dataset demonstrate that FCD-DETR achieves state-of-the-art performance, with a 3.6% improvement in mAP over the baseline RT-DETR while reducing parameters by 22.6% and FLOPs by 25.9%. The model shows strong generalization ability across different background conditions and defect types. Future work will explore further lightweight model compression to enable deployment on edge devices for real-time monitoring of solar panels in large-scale photovoltaic plants.