Abstract
This study proposes a deep learning-based methodology for the classification and precise localization of defects in solar panels. Traditional single-image defect detection methods often suffer from perspective limitations and low resolution. To address these challenges, we introduce a comprehensive framework that integrates image registration, panoramic stitching, and advanced convolutional neural networks (CNNs). By combining infrared and visible-light images, our approach achieves high-resolution panoramic mapping of solar panels, enabling robust defect identification. Experimental results demonstrate classification accuracy, precision, recall, and F1 scores of 93.71%, 93.13%, 93.20%, and 93.11%, respectively. The proposed method offers non-contact, high-efficiency, and rapid inspection capabilities, making it suitable for large-scale solar panel defect detection.
Introduction
The growing demand for renewable energy has accelerated the deployment of solar panels worldwide. However, defects such as hotspots, cracks, and diode failures significantly reduce energy conversion efficiency. Traditional manual inspection methods are time-consuming and prone to human error. Computer vision techniques, particularly those leveraging unmanned aerial vehicles (UAVs), have emerged as promising alternatives. While existing approaches analyze localized infrared or visible-light images, they often fail to provide holistic defect mapping due to limited resolution and environmental noise.
This work addresses these limitations by integrating image stitching algorithms with deep learning models. Our contributions include:
- Panoramic Image Generation: Using the AKAZE feature detection algorithm to stitch localized UAV-captured images into high-resolution panoramas.
- Solar Panel Segmentation: Combining HSV color thresholding and morphological operations to isolate individual solar panel components.
- Defect Classification: Employing EfficientNet_B0, a lightweight yet powerful CNN, to classify defects with minimal computational overhead.
Methodology
1. Panoramic Image Generation
Capturing entire solar farms using UAVs requires stitching multiple overlapping images into a unified panorama. The AKAZE algorithm outperforms traditional feature detectors (e.g., SIFT, ORB) by leveraging nonlinear diffusion filtering for scale-space construction. This preserves edge details and enhances robustness to blur and noise.
Key Equations:
- Nonlinear Diffusion:
∂L∂t=div(c(x,y,t)⋅∇L)∂t∂L=div(c(x,y,t)⋅∇L)
where LL is the image luminance matrix, c(x,y,t)c(x,y,t) is the conduction function, and ∇∇ denotes the gradient operator.
- Hessian Matrix for Feature Detection:
LHessian=σ2(LxxLyy−Lxy2)LHessian=σ2(LxxLyy−Lxy2)
Here, σσ is the scale coefficient, and Lxx,Lyy,LxyLxx,Lyy,Lxy are second-order derivatives.
- Feature Matching:
dmdn<Tdndm<T
where dmdm and dndn are the nearest and second-nearest neighbor distances, and TT is a threshold for rejecting ambiguous matches.
Process Workflow:
- Feature Detection: AKAZE identifies keypoints in overlapping images.
- Feature Matching: kk-nearest neighbors and RANSAC refine matches.
- Perspective Transformation: Aligns images using a homography matrix:
[x′y′1]=[a11a12a13a21a22a23a31a32a33][uv1]x′y′1=a11a21a31a12a22a32a13a23a33uv1
where (u,v)(u,v) and (x′,y′)(x′,y′) are coordinates in the original and transformed images.
2. Solar Panel Segmentation
Accurate segmentation of solar panels from background clutter is critical for defect localization. Our pipeline includes:
- HSV Color Thresholding: Converts RGB images to HSV space to isolate solar panels based on color and brightness.
- Morphological Operations: Erosion and dilation refine the mask to remove noise.
- Contour Detection: Identifies individual panels using minimum bounding rectangles.
3. Defect Classification with EfficientNet_B0
EfficientNet_B0 balances model depth, width, and resolution for optimal performance. Its MBConv blocks integrate depthwise separable convolutions, Swish activation, and squeeze-and-excitation (SE) attention mechanisms.
MBConv Block:
- Depthwise Separable Convolution: Reduces parameters while preserving feature extraction capabilities.
- SE Attention: Enhances channel-wise feature importance:
zc=1H×W∑i=1H∑j=1Wxc(i,j)zc=H×W1i=1∑Hj=1∑Wxc(i,j)s=σ(W2⋅δ(W1⋅z))s=σ(W2⋅δ(W1⋅z))
where zczc is the global average pooling output, δδ is ReLU, and σσ is sigmoid.
Loss Function: Cross-entropy loss optimizes classification:L=−1N∑i=1N∑c=1Myi,clog(pi,c)L=−N1i=1∑Nc=1∑Myi,clog(pi,c)
where yi,cyi,c is the ground-truth label, and pi,cpi,c is the predicted probability.
Experiments
1. Dataset and Training
- Dataset: 7,000 infrared images (24×40 pixels) of solar panels, categorized into four classes:
- Normal
- Hotspot
- Diode Fault
- Crack
Data Split:
| Dataset | Normal | Hotspot | Diode Fault | Crack |
|---|---|---|---|---|
| Training | 1,571 | 1,469 | 1,172 | 688 |
| Validation | 228 | 217 | 171 | 84 |
| Test | 461 | 440 | 331 | 168 |
- Augmentation: Horizontal/vertical flips, random rotations.
- Optimizer: Adam (lr=0.001lr=0.001, batch size=16batch size=16).
2. Performance Evaluation
Metrics:
- Accuracy: TP+TNTP+TN+FP+FNTP+TN+FP+FNTP+TN
- Precision: TPTP+FPTP+FPTP
- Recall: TPTP+FNTP+FNTP
- F1 Score: 2×Precision×RecallPrecision+Recall2×Precision+RecallPrecision×Recall
Model Comparison:
| Model | Accuracy | Precision | Recall | F1 Score |
|---|---|---|---|---|
| ResNet101 | 89.57% | 89.51% | 89.25% | 89.36% |
| MobileNet_V3 | 92.14% | 91.69% | 91.34% | 91.48% |
| RegNet | 90.85% | 90.29% | 90.34% | 90.28% |
| EfficientNet_B0 | 93.71% | 93.13% | 93.20% | 93.11% |
EfficientNet_B0 outperforms competitors due to its parameter efficiency and SE-enhanced feature learning.
3. Defect Localization Results
Defective solar panels are highlighted using bounding boxes:
- Green: Normal
- Red: Crack
- Blue: Hotspot
- Cyan: Diode Fault
Conclusion
This work presents a robust deep learning framework for detecting and localizing defects in solar panels. By combining AKAZE-based panoramic stitching, HSV segmentation, and EfficientNet_B0, we achieve state-of-the-art classification accuracy (93.71%) while minimizing computational costs. Future work will focus on real-time UAV deployment and multi-modal fusion for enhanced defect characterization.