Abstract: This article focuses on the research of solar panel defect classification and localization algorithm based on deep learning. It first analyzes the problems and challenges in the traditional solar panel defect detection methods, and then proposes an improved algorithm. The algorithm uses image registration and stitching techniques to generate high-resolution panoramic images, and utilizes deep learning models for defect classification. Through a series of experiments and evaluations, the effectiveness and superiority of the proposed algorithm are verified. The article also includes discussions on future research directions and potential improvements.
1. Introduction
1.1 Background and Significance
With the increasing global demand for clean energy, solar energy has become one of the most important renewable energy sources. Solar panels, as the key components of solar power generation systems, their performance and reliability directly affect the power generation efficiency and economic benefits. However, during the manufacturing, installation, and operation of solar panels, various defects may occur, such as cracks, hot spots, and diode open circuits. These defects not only reduce the power generation efficiency of solar panels but may also cause safety hazards. Therefore, it is of great significance to develop an efficient and accurate solar panel defect detection method.
1.2 Traditional Detection Methods and Their Limitations
Traditional solar panel defect detection methods mainly include visual inspection, infrared imaging, and electroluminescence imaging. Visual inspection is a simple and intuitive method, but it is highly dependent on the experience and skills of inspectors and is prone to misjudgment and omission. Infrared imaging can detect the temperature distribution of solar panels and identify hot spot defects, but it has difficulty in accurately classifying and locating other types of defects. Electroluminescence imaging can effectively detect cracks and other defects, but the equipment is expensive and requires professional operation. In addition, these traditional methods are time-consuming, labor-intensive, and difficult to meet the needs of large-scale solar panel defect detection.
1.3 Research Objectives and Contributions
The objective of this research is to develop a solar panel defect classification and localization algorithm based on deep learning to overcome the limitations of traditional methods. The main contributions of this research are as follows:
- Propose an image stitching algorithm based on AKAZE operator to generate high-resolution panoramic images of solar panels, which can provide a more comprehensive view for defect detection.
- Use EfficientNet_B0 network for solar panel defect classification, which can improve the accuracy and reliability of defect classification.
- Conduct a series of experiments and evaluations to verify the effectiveness and superiority of the proposed algorithm. The results show that the proposed algorithm has high accuracy, precision, recall, and F1 score in solar panel defect detection.
2. Related Work
2.1 Computer Vision in Solar Panel Defect Detection
Computer vision technology has been widely used in solar panel defect detection. By analyzing the images of solar panels collected by drones or other devices, it can quickly and accurately detect defects. For example, some researchers use neural networks to analyze the infrared images of solar panels to identify hot spot defects. However, the application of computer vision technology also faces some challenges, such as the limited resolution of aerial images and the need for further optimization of neural network models.
2.2 Image Processing Algorithms for Defect Detection
Image processing algorithms play an important role in solar panel defect detection. These algorithms can enhance the image quality, extract features, and segment the regions of interest. For example, image thresholding, edge detection, and morphological operations are commonly used in defect detection. In addition, some advanced algorithms, such as SIFT, SURF, and AKAZE, are used for feature extraction and matching to improve the accuracy of defect detection.
2.3 Deep Learning Models for Defect Classification
Deep learning models have shown excellent performance in image classification and recognition tasks and have also been applied to solar panel defect classification. Convolutional neural networks (CNNs) are the most commonly used deep learning models in this field. By training on a large number of defect images, CNNs can learn the features of different types of defects and accurately classify them. Some advanced CNN architectures, such as ResNet, MobileNet, and EfficientNet, have been proposed to improve the performance and efficiency of defect classification.
3. Methodology
3.1 Image Splicing Algorithm
3.1.1 AKAZE Algorithm for Feature Detection
The AKAZE algorithm is used for feature detection in image splicing. It is an improved version of the KAZE algorithm and has better performance in nonlinear scale space feature detection. The AKAZE algorithm constructs a scale space using nonlinear diffusion filtering, which can retain more edge features. The feature points are detected by comparing the Hessian matrices of pixels at different scales. The Hessian matrix represents the second-order derivative information of the image local structure and can be used to describe the curvature and shape of the image region.
3.1.2 Feature Matching and RANSAC Algorithm
After feature detection, the k-nearest neighbor matching algorithm is used to match the feature points. The similarity between feature points is evaluated by calculating the distance ratio, and the closest points are selected as matching points. Then, the RANSAC algorithm is used to further refine the matching points to remove incorrect matches. The RANSAC algorithm is a commonly used parameter estimation and model fitting method that can estimate the model parameters that conform to the data characteristics and effectively exclude outliers.
3.1.3 Perspective Transformation for Image Stitching
Finally, the perspective transformation technique is used to stitch the local images into a panorama. Perspective transformation is an image processing technique that can stitch the images to be spliced together according to a certain geometric relationship to form a seamless panorama. The single homography matrix is used to describe the mapping relationship between planes, and the coordinates of the images to be spliced in another image can be obtained. Then, the bilinear interpolation method is used to fuse the images to be spliced.
3.2 Image Segmentation Algorithm
3.2.1 Color Threshold Segmentation in HSV Space
The image segmentation algorithm first converts the aerial image to the HSV color space because the HSV color space separates the color and brightness information of the image, which is more conducive to extracting the solar panel region. Then, color threshold segmentation is performed on the converted image to generate a mask image, which extracts the solar panel region.
3.2.2 Morphological Operations and Masking
The mask image is further processed by a series of morphological operations, including gray-scale transformation, image binarization, and morphological processing, to optimize the quality of the mask image. These operations can make the solar panel region more clearly displayed and accurately segment each solar panel component. To perform defect recognition, the mask image is masked with the infrared image to highlight the solar panel component region and filter out other irrelevant information.
3.2.3 Contour Detection and Perspective Correction
Finally, contour detection is performed to obtain the minimum bounding rectangle image of each solar panel component, and perspective transformation is used to correct the shape of the solar panel component, which is convenient for inputting each solar panel component image into the neural network for defect classification.
3.3 Defect Classification Model
3.3.1 EfficientNet_B0 Network Architecture
The EfficientNet_B0 network is used for solar panel defect classification. The EfficientNet series networks improve the model performance by balancing the network width, depth, and resolution. The width parameter controls the number of channels in the convolutional layer, the depth parameter represents the number of network layers, and the resolution parameter determines the size of the input image. The MBConv module in the EfficientNet_B0 network uses depthwise separable convolution to reduce the amount of computation and parameters, and introduces residual structures and Swish activation functions to effectively alleviate the gradient disappearance problem. In addition, the SE attention mechanism is added to enhance the channel features.
3.3.2 Cross-Entropy Loss Function
The cross-entropy loss function is used to measure the difference between the model prediction results and the true labels. It calculates the negative logarithm of the predicted probability of the correct category for each sample and averages them over all samples. The cross-entropy loss function can effectively guide the model to learn and improve the classification accuracy.
3.3.3 Evaluation Metrics (Accuracy, Precision, Recall, F1 Score)
The accuracy, precision, recall, and F1 score are used as evaluation metrics for the defect classification model. The accuracy represents the proportion of correctly classified samples in all predicted samples. The precision represents the proportion of truly positive samples among the samples predicted as positive by the model. The recall represents the proportion of samples predicted as positive by the model among the actually positive samples. The F1 score is the weighted harmonic mean of the precision and recall, which comprehensively considers the accuracy and completeness of the classifier.
4. Experimental Results and Analysis
4.1 Experimental Setup
4.1.1 Hardware and Software Environment
The experimental environment includes a server (Intel®Core™ i7 – 7700 CPU @ 3.60 GHz) and GPU NVIDIA GeForce GTX 1080 Ti, as well as CUDA toolkit v10.1, PyCharm software, and PyTorch framework.
4.1.2 Dataset Description
The infrared image dataset used in this experiment contains 7000 solar cell infrared images with a resolution of 24 pixels × 40 pixels. These images are divided into four categories, including normal images, hot spot defect images, diode open circuit images, and crack images. The dataset is divided into training set, test set, and validation set in a ratio of 7:2:1.
4.2 Image Splicing Results
4.2.1 Feature Detection Results
The results of feature point detection of solar panel local images using the AKAZE algorithm, including visible light images and infrared images. The detected feature points are evenly distributed on the solar panel surface, which can effectively represent the characteristics of the image.
4.2.2 Feature Matching Results
The feature matching results obtained by the k-nearest neighbor matching algorithm for visible light and infrared local images. The matching points are accurately located, and the matching accuracy is high.
4.2.3 Image Splicing Results
The splicing results of visible light and infrared images after homography transformation. The spliced panorama has a seamless effect, and the solar panel region is clearly visible.
4.3 Image Segmentation Results
4.3.1 HSV Color Space Conversion and Segmentation
The results of HSV color space conversion, color threshold segmentation, and morphological operations on the solar panel panorama image. The solar panel region is accurately segmented, and the noise and interference are effectively removed.
4.3.2 Infrared Image Masking and Correction
The infrared image of the solar panel component obtained by masking the infrared image with the mask image and the corrected image after perspective transformation. The solar panel component region is highlighted, and the shape is corrected, which is convenient for defect classification.
4.4 Defect Classification Results
4.4.1 Training and Validation
During the network training, data augmentation operations such as horizontal and vertical flipping and random rotation are used to increase the diversity of the dataset. The Adam optimizer is used to train the model, and the initial learning rate is set to 0.001, and the batch_size is set to 16. In addition, gradient clipping is set during training to avoid gradient explosion. After 200 epochs of training, the training is stopped to avoid overfitting and reduce the computational burden.
4.4.2 Comparison with Other Models
Table 2 shows the performance comparison of different models in defect classification. The EfficientNet_B0 model has the highest accuracy, precision, recall, and F1 score, which indicates that it has better performance in solar panel defect classification.
4.4.3 Defect Classification and Localization Visualization
The defect classification and localization results of solar panel components. The normal components are marked with green frames, and the defective components are marked with different colors according to the defect type, which is intuitive and clear.
5. Conclusion and Future Work
5.1 Summary of Research Findings
This research proposes a solar panel defect classification and localization algorithm based on deep learning. The algorithm uses image splicing technology to generate panoramic images, and uses image processing algorithms and deep learning models to segment and classify solar panel defects. The experimental results show that the proposed algorithm has high accuracy and reliability in solar panel defect detection, and can effectively classify and locate different types of defects.
5.2 Limitations and Future Research Directions
Although the proposed algorithm has achieved good results, there are still some limitations. For example, the algorithm may be affected by the quality of aerial images and the complexity of the environment. In future research, we can explore more advanced image processing algorithms and deep learning models to improve the performance and robustness of the algorithm. In addition, we can also study the application of the algorithm in other fields of solar energy, such as solar power plant operation and maintenance.
5.3 Potential Impact on Solar Panel Inspection and Maintenance
The proposed algorithm has the potential to improve the efficiency and accuracy of solar panel inspection and maintenance. By using drones or other devices to collect images of solar panels and using the proposed algorithm for defect detection, it can quickly and accurately identify defective solar panels and take timely maintenance measures, which can reduce the loss of power generation and improve the economic benefits of solar power plants.
In conclusion, this research provides a new method and 思路 for solar panel defect detection, which has important theoretical and practical significance. We believe that with the continuous development of technology, the proposed algorithm will be further improved and applied in more fields.
| Table 1: Data Distribution in Datasets | ||
|---|---|---|
| Dataset | Category | Number |
| Training Set | Normal | 1571 |
| Hot Spot Defect | 1469 | |
| Bypass Capacitor Short Circuit | 1172 | |
| Crack Defect | 688 | |
| Test Set | Normal | 461 |
| Hot Spot Defect | 440 | |
| Bypass Capacitor Short Circuit | 331 | |
| Crack Defect | 168 | |
| Validation Set | Normal | 228 |
| Hot Spot Defect | 217 | |
| Bypass Capacitor Short Circuit | 171 | |
| Crack Defect | 84 |
| Table 2: Model Evaluation Indicators | ||||
|---|---|---|---|---|
| Network Model | Accuracy | Precision | Recall | F1 |
| ResNet101 | 0.8957 | 0.8951 | 0.8925 | 0.8936 |
| MobileNet_V3 | 0.9214 | 0.9169 | 0.9134 | 0.9148 |
| RegNet | 0.9085 | 0.9029 | 0.9034 | 0.9028 |
| EfficientNet_B0 | 0.9371 | 0.9313 | 0.932 | 0.9311 |