In this study, we present an enhanced RT-DETR model designed specifically for the detection of defects in solar panels, such as cracks, hot spots, dirt, and bird droppings. These defects significantly reduce the energy conversion efficiency of solar panels and pose safety risks. Traditional manual inspection is inefficient, while electrical characteristic testing is costly and inflexible. Deep learning-based computer vision methods offer a promising alternative, but existing models like YOLO and Faster R-CNN still suffer from redundant proposals and require non-maximum suppression (NMS), deviating from true end-to-end detection. Our improved model directly addresses these limitations by proposing three key innovations: a FREDlock backbone for efficient feature extraction, a CRDFP multi-scale feature fusion module, and a deformable attention mechanism for focusing on critical regions. Experimental results demonstrate that our method achieves a mean average precision (mAP) of 79.2%, which is 3.6 percentage points higher than the baseline RT-DETR, while simultaneously reducing the model parameter count by 22.6% and the computational load (FLOPs) by 25.9%. This makes our approach highly suitable for real-time solar panel inspection applications.
Methodology
The proposed FCD-DETR network architecture is built upon the RT-DETR framework, which consists of a backbone, a hybrid encoder (including AIFI and CCFM), and a decoder with auxiliary prediction heads. To improve detection performance for solar panel defects, we introduce three major modifications: the FREDlock backbone, the CRDFP feature pyramid, and the deformable attention-based AIFI module (D-AIFI). Each component is detailed below.
FREDlock Backbone
The original RT-DETR uses ResNet18 as its backbone. However, for fine-grained defect detection in solar panels, ResNet18 struggles to capture subtle local features such as thin cracks or small dirt spots. To address this, we propose the FREDlock module, which is built upon the FasterNet architecture. FasterNet uses partial convolution (PConv) to reduce computational redundancy and memory access. The FLOPs of a PConv layer are given by:
$$ mathit{FLOPs}_{PConv} = h times w times k^2 times c_p^2 $$
where \(h\) and \(w\) are the spatial dimensions, \(k\) is the kernel size, and \(c_p\) is the number of channels processed. For a typical setting, this is approximately 1/16 of the FLOPs of a standard convolution.
We further enhance PConv by applying structural reparameterization (RepConv). During training, we use a multi-branch convolutional layer, which is then fused into a single 3×3 convolution during inference to reduce computation. The batch normalization (BN) fusion process is described by:
$$ widehat{x}_i = gamma cdot frac{x_i – mu}{sqrt{sigma^2 + epsilon}} + beta $$
$$ widehat{x}_i = (w_{BN} cdot w_{conv}) cdot x_i + (w_{BN} cdot b_{conv} + b_{BN}) $$
Additionally, we integrate the Efficient Multi-scale Attention (EMA) module into the processing flow. EMA performs 1D horizontal and vertical global pooling, concatenates the features, and applies matrix dot products to generate spatial attention maps. By splitting the input into G groups, the computational complexity is reduced from \(O(C^2 times H times W)\) to \(O((C^2/G) times H times W)\). This allows the model to focus on important defect regions while suppressing background noise. The combination of RepConv and EMA forms the FREConv block, which improves feature extraction capability without significantly increasing parameters.
CRDFP Multi-Scale Feature Fusion
The original CCFM module in RT-DETR fuses three layers of semantic information from the backbone. However, for solar panel defects under complex backgrounds, simple feature fusion often leads to misalignment between shallow position details and deep semantic information. To overcome this, we design the CRDFP (Contextual Reconstruction and Dynamic Fusion Pyramid) structure. This module consists of two main components: the Pyramid Context Extraction (PCE) block and the Dynamic Interpolation Fusion (DIF) and Multi-Fusion Block (MFB) modules.
The PCE block utilizes the Rectangular Self-Calibration Module (RCM), which captures global context through horizontal and vertical pooling. RCM includes a Rectangular Self-Calibration Attention (RCA) mechanism, batch normalization, and a multi-layer perceptron (MLP). The RCA applies large-kernel striped convolutions in two directions to model rectangular regions of interest, enhancing the model’s ability to adapt to different defect shapes.
For feature fusion, we introduce DIF and MFB. DIF dynamically adjusts the channel numbers and spatial sizes of features from different scales using bilinear interpolation and convolution, achieving efficient cross-scale fusion. MFB processes low-frequency and high-frequency features separately through convolutional layers and activation functions, then merges them to enhance both global semantics and local details. The overall CRDFP structure significantly improves the model’s contextual awareness and representation capability for multi-scale solar panel defects.
Deformable Attention for AIFI
The original AIFI module in RT-DETR applies standard multi-head self-attention (MHSA) to process low-level features. However, MHSA processes all pixels, leading to high computational cost and inefficiency for complex backgrounds. We replace MHSA with Deformable Attention (DAT). DAT dynamically selects key sampling points instead of processing the entire feature map. Given a set of reference points, a lightweight sub-network generates offsets from the query features:
$$ q = xW_q, quad widetilde{k} = widetilde{x}W_k, quad widetilde{v} = widetilde{x}W_v $$
$$ Delta p = theta_{offset}(q), quad widetilde{x} = phi(x; p + Delta p) $$
The sampling function \(\phi\) is implemented as bilinear interpolation, making it differentiable. Relative position bias is computed based on the deformed key points. The attention for each head is then:
$$ z^{(m)} = sigma left( frac{q^{(m)} widetilde{k}^{(m)T}}{sqrt{d}} + phi(widehat{B}; R) right) widetilde{v}^{(m)} $$
By focusing only on important regions, the D-AIFI module reduces computational overhead while maintaining high detection accuracy, especially for small and obscured defects on solar panels.
Experiments
Dataset and Experimental Setup
Since there is a lack of publicly available, specialized datasets for solar panel defect detection, we compiled a dataset from multiple sources, including the PaddlePaddle dataset, Roboflow, and PV-HSD-2025. After filtering, we obtained 4,271 images containing three classes of defects: hot spots, dirt, and cracks. The dataset was split into training, validation, and test sets with an 8:1:1 ratio. All images were resized to 640×640 pixels. The training configuration is summarized in the table below.
| Parameter | Configuration |
|---|---|
| Operating System | Windows 10 |
| GPU | NVIDIA GeForce RTX 4070 Super |
| CPU | Intel Core i5-13400F |
| Memory | 12 GiB |
| Python Version | 3.9 |
| Framework | PyTorch |
Training was performed for 200 epochs with a batch size of 8 and 4 worker threads. Other hyperparameters were set to default values.
Evaluation Metrics
We evaluate the model using precision (P), recall (R), FLOPs, parameter count (Params), and mean average precision (mAP). Precision and recall are defined as:
$$ P = frac{N_{TP}}{N_{TP} + N_{FP}} $$
$$ R = frac{N_{TP}}{N_{TP} + N_{FN}} $$
The average precision (AP) for each class is the area under the precision-recall curve, and the mAP is the mean over all classes:
$$ AP = int_0^1 P(R) dR $$
$$ mAP = frac{1}{n} sum_{j=1}^n AP_j $$
Ablation Study
We performed an ablation study to verify the contribution of each proposed component. The baseline is the original RT-DETR. Results are shown in the table below.
| No. | Baseline | FREBlock | CRDFP | DAIFI | P (%) | R (%) | FLOPs (10^9) | Params (10^6) | mAP (%) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | ✓ | 76.6 | 68.3 | 58.3 | 20.8 | 75.6 | |||
| 2 | ✓ | ✓ | 78.1 | 70.6 | 52.0 | 17.1 | 76.7 | ||
| 3 | ✓ | ✓ | 78.4 | 71.2 | 48.6 | 19.3 | 77.3 | ||
| 4 | ✓ | ✓ | 79.8 | 71.9 | 58.5 | 20.0 | 77.5 | ||
| 5 | ✓ | ✓ | ✓ | 81.6 | 73.4 | 43.3 | 16.4 | 78.5 | |
| 6 | ✓ | ✓ | ✓ | 81.5 | 73.7 | 49.1 | 18.6 | 78.7 | |
| 7 | ✓ | ✓ | ✓ | ✓ | 82.3 | 74.2 | 43.2 | 16.1 | 79.2 |
Experiment No. 1 shows the baseline achieving 75.6% mAP. Adding FREBlock (No. 2) increases mAP by 1.1 percentage points while reducing parameters by 17.8% and FLOPs by 10.8%. This demonstrates that the improved backbone enhances feature extraction efficiency for solar panel defects. Introducing CRDFP (No. 3) and DAIFI (No. 4) individually improves mAP by 1.7 and 1.9 percentage points, respectively, with moderate parameter increases. Combining all components (No. 7) yields the best performance: mAP of 79.2% (a 3.6-percentage-point improvement over baseline), with a 22.6% reduction in parameters and a 25.9% reduction in FLOPs. This confirms the synergistic effect of our proposed modules.
Comparison with State-of-the-Art Methods
We compared our method with several advanced detection algorithms, including Faster R-CNN, YOLO variants, Deformable DETR, and the baseline RT-DETR. The results are presented in the table below.
| Model | P (%) | R (%) | Params (10^6) | FLOPs (10^9) | mAP@0.5 (%) |
|---|---|---|---|---|---|
| Faster R-CNN | 63.9 | 58.4 | 137.1 | 303.1 | 63.6 |
| YOLOv5m | 71.9 | 63.1 | 21.3 | 64.1 | 71.2 |
| YOLOv5l | 72.8 | 64.5 | 46.2 | 78.7 | 72.0 |
| YOLOv8m | 74.5 | 67.2 | 25.8 | 59.0 | 74.3 |
| YOLOv8l | 75.2 | 68.9 | 43.4 | 91.6 | 75.2 |
| YOLOv10m | 75.4 | 69.3 | 17.2 | 63.6 | 75.3 |
| YOLOv11m | 75.7 | 69.6 | 20.6 | 67.7 | 75.5 |
| YOLOv12m | 75.9 | 69.6 | 20.1 | 68.2 | 74.9 |
| Deformable DETR | 74.6 | 68.4 | 39.8 | 196.1 | 75.1 |
| RT-DETR | 75.9 | 69.8 | 20.8 | 58.2 | 75.6 |
| FCD-DETR (Ours) | 82.3 | 74.2 | 16.1 | 43.2 | 79.2 |
Our FCD-DETR achieves the highest mAP (79.2%), precision (82.3%), and recall (74.2%) among all compared methods. Notably, it does so with the fewest parameters (16.1M) and the lowest FLOPs (43.2G), outperforming even the latest YOLO series and other transformer-based detectors. This demonstrates that our model strikes an excellent balance between accuracy and efficiency for solar panel defect detection.
Generalization Experiment
To validate the generalization ability of our method, we conducted additional experiments on the public PVEL-AD dataset, which contains 3,812 near-infrared images with seven defect types (e.g., black core, crack, horizontal dislocation, finger, short circuit, star crack, thick line). The comparison between our model and the baseline RT-DETR is shown in the table below.
| Model | P (%) | R (%) | mAP@0.5 (%) | Params (10^6) |
|---|---|---|---|---|
| RT-DETR | 66.6 | 69.7 | 74.4 | 20.8 |
| FCD-DETR (Ours) | 76.7 | 72.9 | 80.8 | 16.1 |
Our model achieves a significant improvement in mAP (80.8% vs. 74.4%) while reducing parameters by 22.6%. This result strongly supports the robustness and generalization capability of the proposed method for detecting diverse solar panel defects under different imaging conditions.
Conclusion
In this work, we presented an improved RT-DETR-based model for efficient and accurate solar panel defect detection. By introducing the FREDlock backbone, the CRDFP multi-scale fusion structure, and the deformable attention mechanism, we significantly enhanced feature extraction, context fusion, and focus on critical regions. Extensive experiments showed that our model achieves a 3.6-percentage-point improvement in mAP over the baseline, along with substantial reductions in parameters (22.6%) and computational load (25.9%). The generalization test on the PVEL-AD dataset further confirmed the model’s robustness. These results highlight the potential of our approach for real-time, high-quality inspection of solar panels in practical engineering applications. Future work will focus on further optimizing the model for deployment on resource-constrained devices.