Solar photovoltaic power generation is a cornerstone of China’s energy transition and the achievement of the dual-carbon goals. The efficient and safe operation of solar panel systems is of paramount importance, as faults in solar panels directly lead to significant power drops, increased safety risks, and higher maintenance costs. Leveraging existing security surveillance equipment in solar farms, this work proposes a cross-scale feature fusion method for diagnosing six operational states of solar panels: normal, dust accumulation, bird droppings, electrical loss, physical damage, and snow coverage. The method integrates Transformer and Convolutional Neural Network (CNN) branches to capture both global context and local details, enhancing diagnostic accuracy without additional hardware costs. A dense connection mechanism within the CNN branch improves feature propagation and reuse. A dedicated visible-light dataset covering diverse lighting and weather conditions is developed. Extensive experiments demonstrate that the proposed method achieves a Top-1 accuracy of 93.33% and a Top-3 accuracy of 100%, with a moderate parameter count of 125.59M and 32.19 GFLOPs, making it suitable for real-world deployment in solar farms.
Introduction
The rapid expansion of solar photovoltaic capacity demands reliable and cost-effective maintenance solutions. Solar panels, as the core components for energy conversion, are exposed to various environmental stressors, leading to faults such as surface contamination (dust, bird droppings, snow) and structural damage (cracks, electrical degradation). These faults can reduce power generation efficiency by 10% to 50% or even higher. Traditional fault detection relies on specialized sensors or periodic manual inspections, which are economically unfeasible for large-scale solar farms. Therefore, utilizing existing security cameras combined with deep learning offers a promising pathway for automated, real-time solar panel fault diagnosis.
Recent advances in deep learning have shown significant potential in solar panel fault detection. Convolutional neural networks (CNNs) excel at extracting local features from images, making them suitable for detecting small-scale defects like bird droppings or micro-cracks. However, CNNs often struggle with global contextual information, such as the spatial distribution of dust or snow across the entire panel. On the other hand, Vision Transformers (ViTs) leverage self-attention mechanisms to capture long-range dependencies, but they may lack fine-grained local detail extraction. To overcome these limitations, hybrid architectures that combine CNNs and Transformers have emerged. Yet, many existing fusion strategies simply concatenate features without effective guidance from global semantics to local feature extraction.
In this work, we propose a cross-scale feature fusion framework tailored for solar panel fault diagnosis. The key contributions are:
- A dual-branch architecture where the Transformer branch captures global context and the CNN branch extracts local details, with global information dynamically guiding local feature learning.
- Integration of dense connections in the CNN branch to enhance feature propagation and reuse across layers.
- Construction of a visible-light dataset covering six solar panel states under diverse conditions, along with extensive experiments to validate the method’s effectiveness and deployment feasibility.
Related Work
CNN-Based Solar Panel Fault Detection
Early approaches used classic CNN architectures like LeNet and AlexNet for solar panel image classification. Deeper networks such as VGG and ResNet improved feature extraction through increased depth and skip connections. For instance, Feng et al. developed a YOLOX-based system for foreign object detection on solar panels in desert regions. Lv et al. incorporated dual-channel attention into EfficientNet to boost spatial feature extraction. However, these methods rely on fixed receptive fields and cannot effectively model long-range dependencies, limiting performance on faults with large spatial extents like electrical loss or snow coverage.
Transformer-Based Methods
Transformers, with their self-attention mechanism, have been applied to solar panel fault detection. Ramadan et al. used a Transformer for infrared thermal image fault detection. Khan et al. proposed a multi-label Transformer model for overlapping faults. While Transformers excel at global reasoning, they are prone to missing fine-grained local details, especially for small anomalies like bird droppings or minor cracks.
Hybrid CNN-Transformer Models
To combine the strengths of both, researchers have proposed hybrid models. Guo et al. built TransPV for semantic segmentation of solar panels using U-Net and ViT. Mahboob et al. applied SegFormer for defect segmentation in solar panel manufacturing. Wang et al. developed PDFormer with attention-based feature reweighting for small sample scenarios. Nevertheless, many hybrid approaches adopt simple concatenation or late fusion, failing to exploit cross-scale synergies. Our method introduces a mechanism where global features from the Transformer branch actively guide the CNN branch’s local feature extraction at multiple scales, achieving deeper fusion.
Proposed Method
Overall Framework
The overall framework of our cross-scale feature fusion network for solar panel fault diagnosis is illustrated conceptually. The architecture consists of two parallel branches: a Transformer branch (upper) for global context modeling and a CNN branch (lower) for local detail extraction. Cross-scale fusion modules are inserted at multiple stages to allow global information to refine local features. The final fused feature vector is passed through a global average pooling layer and a multi-layer perceptron (MLP) classifier to output probabilities for the six solar panel states.
Figure: Schematic of the cross-scale feature fusion framework (conceptual representation – actual architecture described in text).
Transformer Branch
Given an input solar panel image $$I \in \mathbb{R}^{3 \times 224 \times 224}$$, we first divide it into non-overlapping patches of size $$P=16$$. The number of patches is $$N = (224/16)^2 = 196$$. Each patch is flattened and linearly projected to a $$D=768$$ dimensional embedding, resulting in $$X_p \in \mathbb{R}^{N \times D}$$. A learnable class token and positional embeddings are added. The sequence then passes through a stack of $$L$$ Transformer Encoder layers (we set $$L=3$$ after ablation). Each Encoder comprises layer normalization, multi-head self-attention (MHA), and a feed-forward MLP with residual connections. The MHA is formulated as:
$$
\begin{aligned}
\text{MHA}(Q,K,V) &= \text{Concat}(\text{head}_1, \dots, \text{head}_h) W^O, \\
\text{head}_i &= \text{Attention}(Q W_i^Q, K W_i^K, V W_i^V), \\
\text{Attention}(Q,K,V) &= \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) V.
\end{aligned}
$$
Here, $$Q, K, V$$ are query, key, and value matrices; $$W_i^Q, W_i^K, W_i^V$$ are projection matrices; $$d_k$$ is the dimension of keys; and $$h=12$$ is the number of heads. The output of the Transformer branch is a global feature tensor $$X_{tr} \in \mathbb{R}^{N \times D}$$, which is later reshaped and upsampled for fusion.
CNN Branch
The CNN branch begins with a downsampling convolution using a $$7 \times 7$$ kernel (stride 4, padding 3), compressing the input from $$3 \times 224 \times 224$$ to $$64 \times 56 \times 56$$:
$$
X_{cd} = \text{Conv}(I; W_d, b_d),
$$
where $$W_d \in \mathbb{R}^{64 \times 3 \times 7 \times 7}$$ and $$b_d \in \mathbb{R}^{64}$$.
Subsequently, we design a dense network module called RdnNet composed of multiple stages. Each stage contains a Dense Block and a Transition layer. In a Dense Block, the input to the $$k$$-th layer is the concatenation of feature maps from all preceding layers:
$$
x_k = H_k([x_0, x_1, \dots, x_{k-1}]),
$$
where $$H_k$$ includes batch normalization, ReLU, and a $$3 \times 3$$ convolution; $$[\cdot]$$ denotes channel-wise concatenation. The Transition layer performs $$1 \times 1$$ convolution followed by $$2 \times 2$$ average pooling for downsampling and channel compression:
$$
X_{Trans} = \text{AvgPool}(\text{Conv}(X_{dense}; W_{1\times1}, b_{1\times1})).
$$
This design enhances feature reuse and multi-scale representation within the local branch.
Cross-Scale Feature Fusion
To fuse global and local features, we align the spatial dimensions of the Transformer and CNN branches at multiple stages. At a fusion point (e.g., after the first RdnNet stage), the Transformer feature $$X_{tr}$$ is reshaped to $$X_{tr\_s} \in \mathbb{R}^{768 \times 14 \times 14}$$, then upsampled via transposed convolution to match the CNN feature map size (e.g., $$256 \times 56 \times 56$$). The fusion is performed as element-wise addition with a learnable weight ratio:
$$
X_{fused} = X_{rdn} + \alpha \cdot X_{tr\_up},
$$
where $$\alpha$$ controls the contribution of global features. In our best configuration, we use four fusion nodes with increasing ratios (1/8, 1/4, 1/2, 1) from early to late stages, emphasizing global guidance for deeper local features while preserving low-level details.
After all fusion stages, the final combined feature map $$X_{final} \in \mathbb{R}^{1040 \times 7 \times 7}$$ is globally average pooled:
$$
X_{gap} = \frac{1}{7 \times 7} \sum_{i=1}^{7} \sum_{j=1}^{7} X_{final}[i, j], \quad X_{gap} \in \mathbb{R}^{1040}.
$$
The vector is then fed into an MLP classifier with one hidden layer (512 units) and a softmax output layer for six categories.
Experiments
Dataset
We constructed a visible-light dataset of solar panel images collected from security cameras in solar farms located in Northeast and Northwest China, supplemented by online sources. The dataset contains 885 images across six classes: normal (clean), dust accumulation, bird droppings, electrical loss, physical damage, and snow coverage. Table 1 summarizes the sample distribution.
| Category | Training | Validation | Total | Percentage (%) |
|---|---|---|---|---|
| Normal (Clean) | 183 | 10 | 193 | 21.8 |
| Bird Droppings | 197 | 10 | 207 | 23.4 |
| Dust Accumulation | 180 | 10 | 190 | 21.5 |
| Electrical Loss | 93 | 10 | 103 | 11.6 |
| Physical Damage | 59 | 10 | 69 | 7.8 |
| Snow Coverage | 113 | 10 | 123 | 13.9 |
| Total | 825 | 60 | 885 | 100 |
Data Augmentation
To improve generalization, we applied online data augmentation on the training set only:
- Geometric transforms: Horizontal flip, rotation within ±10°, random crop from 224×224 to 200×200 then resize back.
- Photometric transforms: Brightness/contrast adjustment (0.8 to 1.2 of original), additive Gaussian noise with $$\sigma \in [0, 0.02]$$.
These transforms simulate various capture conditions in solar farms (different panel orientations, lighting, and sensor noise).
Evaluation Metrics
We employ four key metrics: number of parameters (Param.), computational complexity (GFLOPs), Top-1 accuracy, and Top-3 accuracy.
- Param. = total trainable parameters.
- GFLOPs = floating-point operations for one forward pass.
- Top-1 accuracy = $$\frac{N_{\text{correct,1}}}{N_{\text{total}}} \times 100\%$$, where $$N_{\text{correct,1}}$$ is the number of samples with the highest confidence prediction correct.
- Top-3 accuracy = $$\frac{N_{\text{correct,3}}}{N_{\text{total}}} \times 100\%$$, where the true label is within the top three predicted classes.
Implementation Details
All experiments were conducted on an NVIDIA RTX 4090 GPU with Intel Core i7-13700K CPU and 64 GB RAM. The framework is PyTorch-based. Training uses stochastic gradient descent (SGD) with an initial learning rate of 0.001, momentum 0.9, weight decay 1e-4, batch size 32, and 100 epochs. The loss function is cross-entropy:
$$
\mathcal{L} = -\frac{1}{N} \sum_{n=1}^{N} \sum_{c=1}^{C} y_{n,c} \log(p_{n,c}),
$$
where $$C=6$$ is the number of classes, $$y_{n,c}$$ is the one-hot label, and $$p_{n,c}$$ is the softmax output.
Results and Analysis
Loss Curves
We compared the training and validation loss trends of our proposed model with four baseline families: ResNet (18/34/50/101/152), VGG (11/13/16/19), ViT (base_patch16, large_patch16, base_patch32, large_patch32), and Swin-Transformer (tiny, small, base, large). Our model (CTFNet-Base) shows rapid convergence: training loss drops from 2.33 at epoch 1 to 0.028 at epoch 99, while validation loss stabilizes around 0.43. The loss curves demonstrate effective learning without severe overfitting.
Performance Comparison
Table 2 presents the quantitative comparison of all models on our solar panel dataset.
| Method | Param. (M) | GFLOPs | Top-1 Acc (%) | Top-3 Acc (%) |
|---|---|---|---|---|
| ResNet18 | 11.18 | 1.82 | 80.00 | 86.67 |
| ResNet34 | 21.29 | 3.68 | 80.00 | 93.33 |
| ResNet50 | 23.52 | 4.13 | 78.33 | 95.00 |
| ResNet101 | 42.51 | 7.86 | 76.67 | 93.33 |
| ResNet152 | 58.16 | 11.60 | 70.00 | 90.00 |
| VGG11 | 128.79 | 7.61 | 81.67 | 95.00 |
| VGG13 | 128.98 | 11.30 | 78.33 | 91.67 |
| VGG16 | 134.29 | 15.45 | 76.67 | 90.00 |
| VGG19 | 139.59 | 19.63 | 73.33 | 86.67 |
| vit_base_patch16 | 85.65 | 16.86 | 61.67 | 90.00 |
| vit_large_patch16 | 303.11 | 59.69 | 61.67 | 80.00 |
| vit_base_patch32 | 87.42 | 4.37 | 63.33 | 88.33 |
| vit_large_patch32 | 305.46 | 15.26 | 61.67 | 85.00 |
| swin_tiny | 27.50 | 4.37 | 68.33 | 91.67 |
| swin_small | 48.79 | 8.54 | 70.00 | 93.33 |
| swin_base | 86.69 | 15.17 | 65.00 | 95.00 |
| swin_large | 194.91 | 34.08 | 68.33 | 86.67 |
| Ours (CTFNet-Base) | 125.59 | 32.19 | 93.33 | 100.00 |
Our model achieves the highest Top-1 accuracy of 93.33%, surpassing the second-best (VGG11, 81.67%) by 11.66 percentage points. The Top-3 accuracy of 100% indicates that the true fault class is always among the top three predictions, significantly reducing the risk of missed detection. In terms of parameter efficiency, our model (125.59M) is much smaller than ViT-large (303M+) and VGG19 (139.6M), while maintaining competitive GFLOPs (32.19).
Ablation Studies
Effect of Transformer Encoder Depth
We varied the number of Transformer Encoder layers $$L$$ from 1 to 5. As shown in Table 3, the model performance improves with $$L$$ up to 3, after which it saturates. Increasing $$L$$ beyond 3 adds unnecessary computational cost without accuracy gain. Hence we set $$L=3$$.
| L | Param. (M) | GFLOPs | Top-1 Acc (%) |
|---|---|---|---|
| 1 | 82.11 | 18.45 | 85.00 |
| 2 | 103.85 | 25.32 | 90.00 |
| 3 | 125.59 | 32.19 | 93.33 |
| 4 | 147.33 | 39.06 | 93.33 |
| 5 | 169.07 | 45.93 | 91.67 |
Effect of Fusion Ratio Strategy
We compared five fusion ratio schemes (Table 4). The best performance is achieved when early fusion uses a small global weight (1/8) and later stages use larger weights (up to 1). This allows the low-level features to preserve local details, while deeper features benefit from stronger global guidance.
| Scenario | Fusion1 | Fusion2 | Fusion3 | Fusion4 | Top-1 (%) | Top-3 (%) |
|---|---|---|---|---|---|---|
| 1 | 1/8 | 1/8 | 1/8 | 1/8 | 91.67 | 96.67 |
| 2 | 1/3 | 1/3 | 1/3 | 1/3 | 93.33 | 100.0 |
| 3 (Ours) | 1/8 | 1/4 | 1/2 | 1 | 96.67 | 100.0 |
| 4 | 1 | 1/2 | 1/4 | 1/8 | 78.33 | 93.33 |
| 5 | 1 | 1 | 1 | 1 | 70.00 | 88.33 |
Attention Visualization
Using Grad-CAM, we visualized attention maps for several test images. Our model consistently focuses on the actual fault regions (e.g., bird droppings, snow, cracks) with clear boundaries, whereas other models like ResNet34 or VGG11 show diffused attention or mis-focus on background. This qualitative evidence confirms that cross-scale fusion guides the network to recognize critical fault patterns while suppressing irrelevant areas.
Deployment Suitability Analysis
To evaluate practical deployment in solar farms, we propose a three-dimensional metric: Model Accuracy, Efficiency Score (ES), and Scale Score (SS). The ES and SS are defined as:
$$
\begin{aligned}
ES &= 100 \times \frac{\text{GFLOPs}_{\max} – \text{GFLOPs}}{\text{GFLOPs}_{\max} – \text{GFLOPs}_{\min}}, \\
SS &= 100 \times \frac{\text{Param.}_{\max} – \text{Param.}}{\text{Param.}_{\max} – \text{Param.}_{\min}}.
\end{aligned}
$$
A model must have Top-1 accuracy ≥90% to be feasible. Our model satisfies this requirement, while all other models fall below 90%. Although our ES and SS scores (32.19 GFLOPs, 125.59M) are not the highest, the overall balance makes it the only viable candidate among the compared methods for real-world solar panel fault monitoring.
Generalization and Robustness
Unknown Fault Categories
We augmented the classifier with an “Other” class (leaf occlusion, water stains, frame deformation). Using a probability threshold of 0.3, the model correctly identified 8 out of 10 “Other” samples, with only two misclassified due to small object size. The average accuracy over all seven classes reached 94.29% (Table 5).
| Class | Samples | Correct | Accuracy (%) |
|---|---|---|---|
| Normal | 10 | 10 | 100 |
| Bird Droppings | 10 | 9 | 90 |
| Dust | 10 | 10 | 100 |
| Electrical Loss | 10 | 9 | 90 |
| Physical Damage | 10 | 10 | 100 |
| Snow | 10 | 10 | 100 |
| Other | 10 | 8 | 80 |
| Average | 94.29 |
Robustness to Image Variations
We tested the model under different image sizes (320×320, 640×640, 1024×1024), lighting conditions (strong glare, low-light), and weather conditions (fog, rain). The average accuracy remained above 91%, with the largest drop of -5.71% only when shrinking to 320×320 (Table 6). These results demonstrate strong robustness for practical solar farm environments.
| Variation Type | Condition | Avg Accuracy (%) | Deviation (pp) |
|---|---|---|---|
| Image Size | 640×640 (baseline) | 95.00 | — |
| 320×320 | 89.29 | -5.71 | |
| 1024×1024 | 95.00 | 0.00 | |
| Lighting | Normal | 95.00 | — |
| Strong glare | 92.14 | -2.86 | |
| Low-light | 93.57 | -1.43 | |
| Weather | Clear | 95.00 | — |
| Fog | 92.14 | -2.86 | |
| Rain | 91.43 | -3.57 |
Cross-Domain Generalization
We further tested the model on the SDNET2018 public dataset (crack detection in bridges, roads, and walls). As shown in Table 7, our method achieves 91.86% Top-1 accuracy, outperforming all compared models, including ResNet50 (89.73%) and swin_base (82.21%). This validates that our cross-scale fusion strategy is not limited to solar panel fault diagnosis but can generalize to other industrial visual inspection tasks.
| Method | Param. (M) | GFLOPs | Top-1 Acc (%) |
|---|---|---|---|
| ResNet34 | 21.29 | 3.68 | 88.35 |
| ResNet50 | 23.52 | 4.13 | 89.73 |
| VGG11 | 128.79 | 7.61 | 86.56 |
| VGG13 | 128.98 | 11.30 | 87.92 |
| vit_base_patch16 | 85.65 | 16.86 | 78.36 |
| vit_large_patch32 | 305.46 | 15.26 | 76.53 |
| swin_small | 48.79 | 8.54 | 83.37 |
| swin_base | 86.69 | 15.17 | 82.21 |
| Ours (CTFNet-Base) | 125.59 | 32.19 | 91.86 |
Conclusion
This paper presents a cross-scale feature fusion network for solar panel fault diagnosis. By integrating a Transformer branch for global context and a CNN branch with dense connections for local details, the proposed method effectively diagnoses six common solar panel states: normal, dust, bird droppings, electrical loss, physical damage, and snow coverage. The cross-scale fusion mechanism allows global semantic information to guide local feature extraction, leading to superior performance. Our model achieves a Top-1 accuracy of 93.33% and Top-3 accuracy of 100%, outperforming existing CNN, Transformer, and hybrid models. With 125.59M parameters and 32.19 GFLOPs, it strikes a balance between accuracy and computational cost, making it suitable for deployment in solar farms using existing security cameras. Additionally, the method demonstrates strong robustness to variations in image size, lighting, and weather, as well as good generalization to other industrial inspection datasets like SDNET2018. Future work will focus on constructing larger multi-region, multi-climate solar panel datasets and developing lighter network architectures via knowledge distillation for edge devices with limited computational resources.