Photovoltaic power generation is a cornerstone of China’s energy transition and the achievement of the “dual carbon” goals. The efficient and safe operation of solar panels is critical. However, faults in solar panels directly lead to a sharp drop in power generation, increased safety risks, and higher operation and maintenance costs, posing a key bottleneck to the high-quality development of the photovoltaic industry. To address this, we propose a fault diagnosis method for solar panels based on cross-scale feature fusion. This method leverages existing security surveillance equipment in photovoltaic stations without additional hardware costs. It achieves accurate detection of six operating states of solar panels: normal, dust accumulation, bird droppings, electrical loss, physical damage, and snow coverage. Specifically, we first construct a cross-scale feature fusion framework combining Transformer and Convolutional Neural Network (CNN). The Transformer captures global contextual features of solar panel images through its self-attention mechanism, providing global guidance for the CNN to extract local detailed features. Second, we design a dense connection mechanism in the CNN branch to enhance feature propagation and reuse through cross-layer connections of feature maps. Meanwhile, we develop a targeted visible-light dataset of six operating states of solar panels, covering different lighting conditions, weather conditions, and fault types. Compared with other models, our method achieves superior comprehensive performance with a Top-1 accuracy of 93.33% and a Top-3 accuracy of 100%. The model is also relatively lightweight with low hardware computing requirements. Furthermore, we establish a three-dimensional comprehensive evaluation index—”model accuracy, usage efficiency, and application scale”—to evaluate the engineering application value, further verifying the feasibility of deploying the method in actual photovoltaic fields.
Introduction
Solar panels are the core component of photovoltaic systems. Their operating state directly affects the overall power generation efficiency. In long-term operation, solar panels are subject to various environmental and external influences, leading to faults such as surface contamination (dust, bird droppings, snow) and functional damage (electrical degradation, physical cracks). These faults can reduce system efficiency by 10% to 50% or more. Traditional fault detection often relies on specialized sensors, but cost constraints limit their adoption in many photovoltaic stations. Therefore, utilizing existing surveillance cameras combined with deep learning provides a practical and economical solution.
Recent advances in deep learning have shown promise for fault diagnosis of solar panels. Convolutional neural networks (CNNs) excel at extracting local features, while Transformers capture global dependencies. However, single models often struggle with multi-scale fault features. For instance, small bird droppings require fine local details, while large-area dust or snow coverage requires global context. To overcome this limitation, we propose a cross-scale feature fusion framework that integrates Transformer and CNN in a dual-branch architecture, guided by global information to enhance local feature extraction. Additionally, we incorporate a dense connection mechanism to improve feature propagation and reuse.
Proposed Method
Overall Framework
Figure 1 illustrates the overall framework of our cross-scale feature fusion method for fault diagnosis of solar panels. The framework consists of two parallel branches: a Transformer branch (upper) for global context modeling and a CNN branch (lower) for local fine-grained feature extraction. Cross-scale fusion is applied to combine features from both branches, followed by global average pooling and a multi-layer perceptron (MLP) classifier to output predictions for six states of solar panels. The input image is first partitioned into non-overlapping patches of size 16×16 pixels, which are embedded into a 768-dimensional space. The Transformer branch uses three stacked Transformer encoders to encode global features. The CNN branch applies a 7×7 convolution with stride 4 to downsample the input to 64×56×56 dimensions, followed by a dense connection-based RdnNet module to progressively enhance feature representation. At the fusion stage, Transformer features are upsampled to match the spatial dimensions of CNN features, and residual connections are used to dynamically guide local features with global information.
Transformer Branch
For faults such as electrical loss and large-area dust or snow, which exhibit long-range spatial correlations, the self-attention mechanism of the Transformer is effective. Given an input image I ∈ R3×224×224, we divide it into patches of size P = 16, yielding N = (224/16) × (224/16) = 196 patches. Each patch is flattened and linearly projected to D = 768 dimensions, resulting in an embedding feature Xp ∈ RN×D.
The embedded features are fed into Transformer Encoder modules. Each encoder consists of layer normalization, multi-head self-attention (MHA), and a multi-layer perceptron (MLP), with residual connections. Multi-head attention computes multiple attention heads in parallel to capture long-range dependencies:
$$ \text{MHA}(Q,K,V) = \text{Concat}(\text{head}_1, \dots, \text{head}_h) \cdot W^O $$
$$ \text{head}_i = \text{Attention}(Q \cdot W_i^Q, K \cdot W_i^K, V \cdot W_i^V) $$
$$ \text{Attention}(Q,K,V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) V $$
where Q, K, V are query, key, value matrices; WiQ, WiK, WiV are projection matrices; dk is the key dimension; h is the number of heads; WO is the output projection.
CNN Branch
For local faults like bird droppings and minor physical damage, CNNs with local receptive fields are suitable. The input image I ∈ R3×224×224 first passes through a downsampling convolution layer with kernel size 7×7, stride 4, padding 3, expanding channels from 3 to 64 and reducing spatial dimensions to 56×56:
$$ X_{cd} = \text{Conv}(I; W_d, b_d) $$
where Wd ∈ R64×3×7×7 and bd ∈ R64 are weights and bias.
Then, an RdnNet module based on dense connections enhances multi-scale local feature extraction. In a Dense Block, the k-th layer receives concatenated outputs of all previous layers:
$$ x_k = H_k([x_0, x_1, \dots, x_{k-1}]) $$
where Hk consists of batch normalization, ReLU, and convolution. Transition layers perform spatial downsampling and channel compression using 1×1 convolution and 2×2 average pooling:
$$ X_{Trans} = \text{AvgPool}(\text{Conv}(X_{dense}; W_{1\times1}, b_{1\times1})) $$
This preserves key local features while reducing resolution for subsequent fusion.
Cross-Scale Feature Fusion
Faults in solar panels exhibit multi-scale characteristics: small bird droppings and fine cracks require local details, while large-area dust, snow, and electrical loss require global context. Our fusion mechanism uses Transformer global information to guide CNN local feature extraction. First, the Transformer encoder output Xtr ∈ R196×768 is reshaped to spatial feature Xtr_s ∈ R768×14×14, then upsampled via convolution to match the 56×56 dimension of CNN features, with channel adjustment to 256. The upsampled Transformer feature Xtr_up ∈ R256×56×56 is element-wise added to the CNN branch output Xrdn ∈ R256×56×56:
$$ X_{fused} = X_{rdn} + X_{tr_up} $$
This operation allows global information to modulate local feature extraction. After fusion, the combined feature Xfinal ∈ R1040×7×7 passes through global average pooling to compress spatial dimensions:
$$ X_{gap} = \frac{1}{7 \times 7} \sum_{i=1}^{7} \sum_{j=1}^{7} X_{final}[i,j], \quad X_{gap} \in \mathbb{R}^{1040} $$
Finally, a multi-layer perceptron (MLP) classifier outputs prediction scores for the six states of solar panels.
Dataset and Data Augmentation
We constructed a visible-light dataset of solar panels covering six operating states. Samples were collected from surveillance cameras in photovoltaic stations in Northeast and Northwest China, as well as public internet sources. All images are captured under natural conditions without intervention, preserving complex environmental features such as different lighting conditions (bright sunny noon, overcast, sunrise/sunset), weather conditions, and panel types (monocrystalline, polycrystalline). The dataset is open-sourced. Table 1 shows the number and proportion of samples per class.
| Class | Training Set | Validation Set | Total | Proportion (%) |
|---|---|---|---|---|
| Normal (clean) | 183 | 10 | 193 | 21.8 |
| Bird droppings | 197 | 10 | 207 | 23.4 |
| Dust accumulation | 180 | 10 | 190 | 21.5 |
| Electrical loss | 93 | 10 | 103 | 11.6 |
| Physical damage | 59 | 10 | 69 | 7.8 |
| Snow coverage | 113 | 10 | 123 | 13.9 |
| Total | 825 | 60 | 885 | 100.0 |
To enhance model robustness, we applied online data augmentation during training only, including random geometric transformations and illumination/noise adjustments. Random horizontal flip simulates different installation orientations. Random rotation within [-10°, 10°] accounts for camera angle shifts. Random cropping from 224×224 to 200×200 then resizing back improves extraction of small faults. Brightness and contrast adjustments range from 0.8 to 1.2 times the original. Gaussian noise with zero mean and standard deviation σ ∈ [0, 0.02] simulates sensor noise under low light.
Experimental Setup
All experiments were conducted on an Intel Core i7-13700K CPU, NVIDIA RTX 4090 GPU, 64 GB RAM, with PyTorch, Python 3.8. We used stochastic gradient descent optimizer with initial learning rate 1×10-3, momentum 0.9, weight decay 1×10-4, batch size 32, and 100 epochs. The loss function is cross-entropy:
$$ \mathcal{L} = -\frac{1}{N} \sum_{n=1}^{N} \sum_{c=1}^{C} y_{n,c} \log(p_{n,c}) $$
where N is batch size, C=6 classes (states of solar panels), yn,c is one-hot label, pn,c is softmax probability. Evaluation metrics include parameter count (Param.), computational complexity (GFLOPs), Top-1 accuracy, and Top-3 accuracy.
Results and Analysis
Loss Curve Comparison
We compared our proposed method with ResNet, VGG, ViT, and Swin-Transformer families. Figure 2 shows training and validation loss curves. Our model achieves rapid loss decrease and converges to low values. Training loss drops from 2.3290 initially to 0.0279 at epoch 99, while validation loss decreases from 1.6885 to 0.4306, indicating good fitting and generalization.
Performance Comparison
Table 2 summarizes the performance of various models on the solar panel fault diagnosis task. Our model (ctfnet_base) achieves Top-1 accuracy of 93.33% and Top-3 accuracy of 100%, with 125.59 M parameters and 32.19 GFLOPs. In contrast, the next best model (VGG11) has 81.67% Top-1 accuracy, while models like vit_base_patch32 achieve only 63.33%. Our method demonstrates superior accuracy with moderate computational cost.
| Method | Structure | Param. (M) | GFLOPs | Top-1 AC (%) | Top-3 AC (%) |
|---|---|---|---|---|---|
| ResNet | ResNet18 | 11.18 | 1.82 | 80.00 | 86.67 |
| ResNet34 | 21.29 | 3.68 | 80.00 | 93.33 | |
| ResNet50 | 23.52 | 4.13 | 78.33 | 95.00 | |
| ResNet101 | 42.51 | 7.86 | 76.67 | 93.33 | |
| ResNet152 | 58.16 | 11.60 | 70.00 | 90.00 | |
| VGG | VGG11 | 128.79 | 7.61 | 81.67 | 95.00 |
| VGG13 | 128.98 | 11.30 | 78.33 | 91.67 | |
| VGG16 | 134.29 | 15.45 | 76.67 | 90.00 | |
| VGG19 | 139.59 | 19.63 | 73.33 | 86.67 | |
| ViT | vit_base_patch16 | 85.65 | 16.86 | 61.67 | 90.00 |
| vit_large_patch16 | 303.11 | 59.69 | 61.67 | 80.00 | |
| vit_base_patch32 | 87.42 | 4.37 | 63.33 | 88.33 | |
| vit_large_patch32 | 305.46 | 15.26 | 61.67 | 85.00 | |
| Swin-T | swin_tiny | 27.50 | 4.37 | 68.33 | 91.67 |
| swin_small | 48.79 | 8.54 | 70.00 | 93.33 | |
| swin_base | 86.69 | 15.17 | 65.00 | 95.00 | |
| swin_large | 194.91 | 34.08 | 68.33 | 86.67 | |
| Ours | ctfnet_base | 125.59 | 32.19 | 93.33 | 100.00 |
Hyperparameter Analysis
We studied the effect of the number of Transformer encoder layers L and fusion ratios. Table 3 shows the impact of fusion ratios at four fusion positions. Scene 3 (our default) uses increasing ratios (1/8, 1/4, 1/2, 1) from early to late stages, achieving 96.67% Top-1 accuracy. Scene 4 (decreasing ratios) yields only 78.33%, showing that early layers should be less influenced by global features to preserve low-level information.
| Scene | Fusion Pos 1 | Fusion Pos 2 | Fusion Pos 3 | Fusion Pos 4 | Top-1 AC (%) | Top-3 AC (%) |
|---|---|---|---|---|---|---|
| 1 | 1/8 | 1/8 | 1/8 | 1/8 | 91.67 | 96.67 |
| 2 | 1/3 | 1/3 | 1/3 | 1/3 | 93.33 | 100.00 |
| 3 (Ours) | 1/8 | 1/4 | 1/2 | 1 | 96.67 | 100.00 |
| 4 | 1 | 1/2 | 1/4 | 1/8 | 78.33 | 93.33 |
| 5 | 1 | 1 | 1 | 1 | 70.00 | 88.33 |
For encoder layers L, we observed that accuracy increases with L up to 3, then plateaus. L=3 provides the best trade-off between performance and computational cost.
Attention Visualization
We used Grad-CAM to visualize attention heatmaps for typical faults. Our model precisely focuses on fault regions (e.g., bird droppings, cracks, snow areas) with clear boundaries, while baseline models like ResNet34 and VGG11 show diffused attention, and ViT/Swin models may miss multi-scale faults. This demonstrates the effectiveness of cross-scale fusion.
Field Deployment Adaptability
We defined a three-dimensional “accuracy-efficiency-scale” metric. Efficiency score ES and scale score SS are computed as:
$$ ES = 100 \times \frac{GFLOPs_{max} – GFLOPs}{GFLOPs_{max} – GFLOPs_{min}} $$
$$ SS = 100 \times \frac{Param._{max} – Param.}{Param._{max} – Param._{min}} $$
Our model achieves the highest Top-1 accuracy among all candidates, meeting the 90% threshold for field deployment. Although its efficiency and scale scores are lower than some lightweight models, it is the only model that satisfies accuracy requirements, proving its practical viability.
Generalization Analysis
We evaluated robustness to unknown classes by adding an “other” category. Table 4 shows accuracy on 70 new samples (10 per class including “other”). Our model correctly identifies most “other” samples (80% accuracy) without degrading performance on original classes, maintaining 94.29% average accuracy.
| Class | Samples | Correct | Accuracy (%) |
|---|---|---|---|
| Normal (clean) | 10 | 10 | 100.00 |
| Bird droppings | 10 | 9 | 90.00 |
| Dust accumulation | 10 | 10 | 100.00 |
| Electrical loss | 10 | 9 | 90.00 |
| Physical damage | 10 | 10 | 100.00 |
| Snow coverage | 10 | 10 | 100.00 |
| Other | 10 | 8 | 80.00 |
| Average | 94.29 |
We further tested robustness under image size variation, lighting changes, and weather interference. Results (Table 5) show that our model maintains average accuracy above 91.4% under challenging conditions such as rain and fog, with maximum deviation of -5.71% only under severe size reduction.
| Interference Type | Condition | Average Accuracy (%) | Deviation (%) |
|---|---|---|---|
| Image size | 640×640 (baseline) | 95.00 | — |
| 320×320 (reduced) | 89.29 | -5.71 | |
| 1024×1024 (enlarged) | 95.00 | 0.00 | |
| Lighting | Normal light (baseline) | 95.00 | — |
| Strong light (glare) | 92.14 | -2.86 | |
| Weak light (dusk) | 93.57 | -1.43 | |
| Weather | Normal (baseline) | 95.00 | — |
| Fog (low visibility) | 92.14 | -2.86 | |
| Rain (wet reflection) | 91.43 | -3.57 |
To test cross-domain generalization, we applied our model to the SDNET2018 dataset (crack detection on bridge, road, wall). As shown in Table 6, our model achieves 91.86% Top-1 accuracy, outperforming all baselines, indicating strong transferability to industrial visual inspection tasks.
| Method | Param. (M) | GFLOPs | Top-1 AC (%) |
|---|---|---|---|
| ResNet34 | 21.29 | 3.68 | 88.35 |
| ResNet50 | 23.52 | 4.13 | 89.73 |
| VGG11 | 128.79 | 7.61 | 86.56 |
| VGG13 | 128.98 | 11.30 | 87.92 |
| vit_base_patch16 | 85.65 | 16.86 | 78.36 |
| vit_large_patch32 | 305.46 | 15.26 | 76.53 |
| swin_small | 48.79 | 8.54 | 83.37 |
| swin_base | 86.69 | 15.17 | 82.21 |
| Ours | 125.59 | 32.19 | 91.86 |
Conclusion
We proposed a cross-scale feature fusion network for fault diagnosis of solar panels. By integrating local features from CNN with global context from Transformer, and introducing dense connections, our method achieves high accuracy (Top-1: 93.33%, Top-3: 100%) with moderate computational cost (125.59 M parameters, 32.19 GFLOPs). The method utilizes existing surveillance equipment, reducing hardware expenses. It demonstrates strong robustness, generalization to unknown states, and adaptability to field deployment in photovoltaic stations. Future work will focus on constructing larger multi-region datasets and applying knowledge distillation for lighter architectures to suit edge devices with limited computing power.