In this work, we present an enhanced Detection Transformer (DETR) framework tailored for detecting defects in solar panels captured by unmanned aerial vehicles (UAVs). The proposed method addresses critical challenges such as low detection accuracy for small-scale defects and slow inference speed. By integrating relative position encoding (RPE), a dynamic sparse attention (DSA) mechanism, and a focal loss function, we achieve a mean average precision (mAP) of 94.7% on our dataset, significantly outperforming the baseline DETR and other mainstream detectors. Our study provides a practical solution for automated inspection of solar panels, contributing to the reliability and efficiency of photovoltaic power stations.
Introduction
Solar energy has become a cornerstone of renewable power generation worldwide. However, solar panels operating outdoors are susceptible to various defects such as cracks, hot spots, diode failures, and junction box anomalies. These defects not only reduce energy conversion efficiency but also pose safety risks. Traditional manual inspection is time‑consuming and labor‑intensive. Therefore, automated defect detection using deep learning has emerged as a promising alternative.
Recent advances in object detection, particularly transformer‑based architectures, offer end‑to‑end solutions without hand‑crafted post‑processing. The Detection Transformer (DETR) simplifies the detection pipeline by treating object detection as a set prediction problem. However, DETR struggles with small objects and suffers from high computational cost due to full self‑attention. In this paper, we propose an improved DETR tailored for solar panel defect detection from UAV imagery. Our contributions include:
- Introducing relative position encoding (RPE) to enhance sensitivity to small flaw positions.
- Employing a dynamic sparse attention (DSA) module to reduce attention complexity and speed up inference.
- Replacing the standard classification loss with a focal loss to better handle hard‑to‑classify samples.
Extensive experiments on our solar panel defect dataset demonstrate the effectiveness of each component and the overall superiority of our method.
Related Work
Defect Detection for Solar Panels
Numerous deep learning approaches have been applied to solar panel defect detection. Faster R‑CNN with multi‑scale feature fusion was used to identify defects in infrared images (Li et al., 2023). Improved SSD with attention mechanisms and transfer learning achieved high accuracy on UAV‑captured images (Zhong et al., 2022). Despite these efforts, challenges remain in detecting small defects and achieving real‑time performance.
DETR and Its Variants
DETR (Carion et al., 2020) eliminates the need for anchor generation and non‑maximum suppression by leveraging a transformer encoder‑decoder architecture. However, its quadratic self‑attention complexity limits speed, and its absolute position encoding is suboptimal for small objects. Several works have attempted to improve DETR by introducing deformable attention (Zhu et al., 2021), conditional queries (Meng et al., 2021), or efficient attention mechanisms. Our work focuses on lightweight attention and position encoding enhancements specific to solar panel defect inspection.
Proposed Method
We build upon the original DETR architecture, which consists of a CNN backbone, a transformer encoder with self‑attention, a decoder with cross‑attention, and prediction heads. Our modifications target three key aspects: position encoding, attention computation, and loss function.
Relative Position Encoding (RPE)
In the vanilla transformer, absolute position encodings are added to the input embeddings. For small defects in solar panels, the absolute position does not capture the pairwise spatial relationships effectively. We replace the absolute encodings with relative position encodings, as proposed in the Transformer‑XL (Dai et al., 2019). The attention score between query i and key j is decomposed as:
$$
A^{\text{abs}}_{i,j} = E_{x_i}^T W_q^T W_k E_{x_j} + E_{x_i}^T W_q^T W_k U_j + U_i^T W_q^T W_k E_{x_j} + U_i^T W_q^T W_k U_j
$$
We transform it to relative form:
$$
A^{\text{rel}}_{i,j} = E_{x_i}^T W_q^T W_{k,E} E_{x_j} + E_{x_i}^T W_q^T W_{k,R} R_{i-j} + u^T W_{k,E} E_{x_j} + v^T W_{k,R} R_{i-j}
$$
Here, $W_{k,E}$ and $W_{k,R}$ are weight matrices for the key associated with the embedding and the relative position, respectively. $R_{i-j}$ is a sinusoidal encoding of the relative distance, and $u$, $v$ are learnable vectors. This formulation allows the model to focus on the relative offset between query and key, which is crucial for discriminating small defects such as diode hot spots that occupy only a few pixels in the feature map. Our experiments show that RPE improves the detection precision for small targets by 3.9%.
Dynamic Sparse Attention (DSA)
Full self‑attention in DETR has a complexity of $O(n^2)$ with respect to the sequence length $n$ (the total number of pixels in the feature map). To accelerate detection speed for real‑time UAV inspection, we introduce a dynamic sparse attention (DSA) module (Ma et al., 2022). DSA adds a lightweight prediction path that learns a sparse attention pattern for each input. The attention computation is then reformulated as a sparse‑dense matrix multiplication (SODMM). Specifically, for a given query $Q$ and key $K$, we predict a binary mask $M \in \{0,1\}^{n \times n}$ top‑k indices based on a similarity proxy (e.g., linear projection). The attention output is computed as:
$$
\text{Attention}(Q,K,V) = \text{Softmax}\left( \frac{QK^T \odot M}{\sqrt{d_k}} \right) V
$$
where $\odot$ denotes element‑wise multiplication. The mask $M$ retains only the most relevant key‑value pairs, reducing the effective number of multiplications. During training, the mask is learned via a gating mechanism. This approach reduces the computational cost by approximately 40% while maintaining competitive accuracy. In our experiments, DSA led to a 0.8% drop in mAP but significantly improved inference speed.
Focal Loss for Hard Samples
Solar panel defect datasets often contain a high proportion of easy‑to‑classify background regions, while defects like hairline cracks or faint hot spots are hard to distinguish. The original DETR uses a cross‑entropy loss for classification, which treats all samples equally. We adopt the focal loss (Lin et al., 2017) to down‑weight easy samples and focus on hard ones:
$$
\text{FL}(p_t) = -(1-p_t)^\gamma \log(p_t)
$$
where $p_t$ is the model’s predicted probability for the ground‑truth class, and $\gamma$ is a focusing parameter (set to 2 in our experiments). For multi‑class scenarios, the focal loss is summed over all classes. This modification increases the loss contribution of misclassified instances, improving the detection of ambiguous defects. With focal loss, our mAP improved by 2.6% compared to using cross‑entropy alone.
Overall Architecture
The complete improved DETR pipeline is as follows. Input infrared images of solar panels (640×512 pixels) are fed into a ResNet‑50 backbone to extract feature maps. The feature maps are flattened into a sequence of patches and processed by a transformer encoder with six layers, each employing DSA and RPE. The decoder has six layers with cross‑attention and self‑attention. Prediction heads output bounding boxes and class logits. The total loss $\mathcal{L}$ combines smooth L1 regression loss $\mathcal{L}_{\text{reg}}$ and focal classification loss $\mathcal{L}_{\text{cls}}$:
$$
\mathcal{L} = \lambda_{\text{reg}} \mathcal{L}_{\text{reg}} + \lambda_{\text{cls}} \mathcal{L}_{\text{cls}}
$$
We set $\lambda_{\text{reg}}=5$ and $\lambda_{\text{cls}}=1$.
Experiments
Dataset Construction
We collected 235 infrared images of solar panels from the rooftop of Shanghai Dianji University using a DJI Mavic 2 drone. The images were annotated with three defect types: diode failure, junction box failure, and surface cracks. To increase diversity, data augmentation techniques (rotation, flipping, shearing, blurring) were applied to expand the dataset to 1,200 images. The dataset was split into training and validation sets in a 4:1 ratio, yielding 960 training and 240 validation images. The validation set contains 284 defect instances, with some images containing multiple defects.
Implementation Details
Experiments were conducted on an Intel i9-10900 CPU with 32 GB RAM and an NVIDIA GTX 3080 (16 GB) GPU. The code was implemented in PyTorch. Training parameters are summarized in Table 1.
| Parameter | Value |
|---|---|
| Input image size | 640×512 pixels |
| Initial learning rate | 1e-4 |
| Weight decay | 1e-5 |
| Batch size | 2 |
| Number of epochs | 300 |
| Optimizer | AdamW |
| Focal loss $\gamma$ | 2 |
Evaluation Metrics
We used precision ($P$) and mean average precision (mAP) as metrics:
$$
P = \frac{TP}{TP + FP}, \quad \text{mAP} = \frac{1}{C} \sum_{c=1}^{C} AP_c
$$
where $TP$ and $FP$ are true and false positives, $C=3$ is the number of defect classes, and $AP_c$ is the average precision for class $c$.
Training Behavior
Figure 5 of the original paper (not reproduced here) shows the training loss curves. The improved DETR converges faster and achieves a lower loss plateau compared to the original DETR. Stable convergence is reached after 200 epochs.
Ablation Study and Comparisons
We conducted ablation experiments by incrementally adding the proposed modules to the baseline DETR. We also compared with YOLOv5, a popular one‑stage detector. Results are presented in Table 2.
| Method | P0 (%) | P1 (%) | P2 (%) | mAP (%) |
|---|---|---|---|---|
| YOLOv5 | 90.3 | 90.4 | 89.9 | 90.2 |
| DETR (baseline) | 89.8 | 89.8 | 89.2 | 89.6 |
| DETR + RPE | 92.5 | 93.1 | 93.1 | 92.9 |
| DETR + RPE + DSA | 92.0 | 92.4 | 91.9 | 92.1 |
| DETR + RPE + DSA + FL | 94.8 | 95.0 | 94.3 | 94.7 |
From Table 2, we observe that:
- Relative Position Encoding (RPE) boosts mAP by 3.3%, with the most significant gain (3.9%) on the small diode failure class (P2). This confirms that RPE enhances the model’s ability to localize tiny defects on solar panels.
- Dynamic Sparse Attention (DSA) reduces mAP by 0.8% compared to the full‑attention version (92.9% → 92.1%) but substantially speeds up inference. The trade‑off is acceptable for real‑time applications.
- Focal Loss (FL) further improves mAP by 2.6%, bringing the final mAP to 94.7%. The focal loss effectively focuses on hard‑to‑classify samples such as ambiguous hot spots or partially occluded defects.
- The improved DETR outperforms YOLOv5 (90.2%) by 4.5%, demonstrating the advantage of the transformer‑based approach for this task.
Conclusion
In this paper, we presented an improved DETR algorithm for defect detection in solar panels from UAV‑acquired infrared images. Our three modifications—relative position encoding, dynamic sparse attention, and focal loss—address the key issues of small‑target detection, slow inference, and hard‑sample classification, respectively. Experiments on a real‑world dataset show that the proposed method achieves a mAP of 94.7%, a 5.1% improvement over the baseline DETR and a 4.5% improvement over YOLOv5. The enhanced model provides a reliable and efficient solution for automated inspection of solar panels, facilitating the maintenance of photovoltaic power stations.
Future work may explore further acceleration techniques and adaptation to other renewable energy infrastructure, such as wind turbine blades. We also plan to deploy the model on edge devices for on‑site UAV inspections.