I have developed an intelligent system that leverages image processing, machine learning, and unmanned aerial vehicle (UAV) hardware to automatically inspect solar panels and identify faults for modeling purposes. This system integrates a thermal imager, wireless video transmission module, and advanced algorithms to achieve rapid detection, short inspection cycles, high accuracy, and intuitive fault localization. It effectively addresses the inefficiencies of manual inspection that often delay fault repair. By applying this system, we can prevent and resolve power generation losses caused by solar panel failures, thereby conserving energy and promoting the development of renewable energy sources. Moreover, it enhances the economic benefits of photovoltaic power generation and provides robust technical support for large-scale construction and maintenance of solar power plants.
Background and Significance
With the rapid advancement of renewable energy technologies, an increasing number of photovoltaic power plants are being established. In 2018, China’s newly installed photovoltaic capacity reached 44.1 GW, with a cumulative installed capacity of 174.63 GW, covering approximately 620 million square meters of solar panels. By 2020, this area is expected to reach 941 million square meters. Given such a massive market scale, ensuring the normal operation of solar panels is crucial for the stable and efficient growth of photovoltaic power stations. Therefore, the maintenance and inspection of solar panels have become vitally important. Defects such as black spots, fragments, and hidden cracks can severely reduce power generation efficiency, leading to energy waste. Traditionally, solar panel management relies on manual inspection, which is time-consuming, labor-intensive, and hazardous, especially for large-scale and remote-location power stations. Applying intelligent detection systems to photovoltaic power plant inspections not only improves accuracy and speed but also significantly reduces labor costs. My proposed solution is simple, cost-effective, and highly accurate in detecting faulty solar panels. It can be widely applied to various photovoltaic power stations and holds promising application prospects.
System Composition and Technical Details
The system comprises two main components: the UAV inspection subsystem and the fault detection and processing subsystem. The UAV, equipped with a thermal imager and wireless video transmission module, performs scheduled inspections along predefined flight paths over the photovoltaic array. It captures infrared videos of the solar panels and transmits them in real time to a ground computer. The computer then uses machine learning algorithms to quickly identify fault locations and mark them, providing timely feedback to maintenance personnel.
UAV Inspection System
I utilize the Zenmuse XT2 thermal imager mounted on a UAV, with its angle pre-adjusted relative to the ground solar panels. During flight, the thermal imager’s video is transmitted via the UAV’s wireless video transmission system to a ground receiver. A video capture card then feeds the video to a computer for real-time fault detection. The wireless transmission uses an HDMI output module and a video capture card to ensure low-latency, high-quality image streaming.
Fault Detection and Processing System
The system reads frames from the incoming video and performs target recognition. I apply the OTSU algorithm and morphological processing (specifically a “hot spot” operation) combined with mathematical calculations to obtain clear and complete infrared images of the solar panels. Subsequently, I extract SIFT features from the obtained images. Solar panels lacking SIFT features are initially classified as “normal” or “faulty.” Using the Bag of Words (BOW) model, I describe the features of all image patches in the “faulty” class and extract vectors. Then, a Support Vector Machine (SVM) performs a secondary classification to ultimately produce “normal” and “faulty” categories. The fault information is then passed to a modeling algorithm. The modeling process uses remote control to define the inspection path and capture video, from which a panoramic view of the route is generated. Each inspection result is displayed on this panorama for easy maintenance. For example, if a specific solar panel (e.g., panel No. 1) is detected as faulty, the system returns its label and area, marking it on the panorama. The system also stores the positions, power generation data, operating time, and fault types of both healthy and faulty solar panels, uploading the data to a cloud database. This enables product improvement and lays the groundwork for future big data analysis and next-generation rapid fault detection systems.
Technical Analysis
The infrared imaging of solar panel hot spots arises from different materials having distinct radiation characteristics. The emitted thermal radiation is captured by the thermal imager and converted into visible images that represent the magnitude, intensity, and distribution of thermal energy. This allows us to visualize otherwise invisible temperature data. Thermal imaging offers strong anti-interference capability, long-range operation, and high precision. Initially used in military and specialized industries, it has gradually become widespread in agriculture, industry, and even civilian vehicles. Everyday objects emit infrared radiation of varying intensities, which are proportional to their temperature distribution. By converting these radiation intensities into electrical signals and displaying them as images or videos on a monitor, we can intuitively observe temperature data invisible to the naked eye.
The target recognition algorithm follows these steps:
1. Convert the extracted video frame into a grayscale image I(x, y).
2. Apply the OTSU algorithm to segment the target and binarize I(x, y) into BW(x, y).
3. Use morphological processing (the “hot spot” operation) to remove thin connecting “tubes” between targets.
4. Obtain the boundaries of connected regions in the processed image and expand them into rectangular areas.
5. Calculate the average length, width, and total area of these rectangles. Rectangles with an aspect ratio of 1:2 and an area exceeding the average are identified as target regions, i.e., solar panels.
After completing these steps, I extract and save the solar panel regions as images for later recognition. To avoid incomplete panels, I use a video frame reading method: I take one frame every 6 seconds, extracting a total of 15 frames for segmentation. By using both aspect ratio and area mean as constraints, I separate individual solar panels accurately, even when the video contains incomplete panels that happen to match the aspect ratio. This improves segmentation accuracy and adaptability.
Fault Detection Technology
SIFT Algorithm
During UAV inspection, the captured images inevitably contain noise and distortion due to variations in angle, sun position, ambient light, etc. Although preprocessing reduces noise and corrects distortions, some residual effects remain and affect feature points. These feature points contain critical information such as color, texture, and shape. Therefore, feature extraction is essential. I employ the SIFT (Scale-Invariant Feature Transform) algorithm, known for its robustness, distinctiveness, abundance, and scalability. SIFT has strong anti-noise capability and adaptability to spatial similarity transformations.
The SIFT implementation involves four steps:
(1) Detect scale-space extrema.
(2) Locate keypoints.
(3) Determine keypoint orientations.
(4) Generate feature descriptors.
By combining SIFT feature extraction with the BOW model, I classify solar panel images into normal and faulty categories. Although the presence or absence of SIFT features can give a preliminary classification, it is not always accurate due to reflections and edge artifacts. Therefore, the BOW model is used for further classification, evaluating the likelihood of normal and faulty panels.
Bag of Words (BOW) Model
Originally from NLP and IR, the BOW model represents a document as an unordered set of words, ignoring grammar and word order. In computer vision, it is widely applied. I use the BOW model to further distinguish between fault-induced bright spots and reflection interference, achieving high accuracy.
The procedure is as follows:
– First, extract all SIFT features from training samples and cluster them into 8 classes using K-means, forming a dictionary of 8 visual words.
– Then, for each test image, construct a feature vector based on the dictionary.
For example, a solar panel with no detected SIFT features is directly considered normal. For other panels, the feature vectors are computed. Based on the scores, panels are classified as normal or faulty. After confirming a fault, the system proceeds to modeling and outputting the image.
To map the physical location of each panel, I define the inspection path as an “S” shape starting from the upper-left corner of Area A. Using time and flight speed, I determine the absolute position of each panel in the video. To avoid “ghost images” caused by heading changes, the system detects landmarks (e.g., black strips in the area) to segment the video and prevent missed detections.
Single frames from the UAV video are extracted, and the infrared image of each individual solar panel is isolated by comparing grayscale values. The OTSU algorithm is used for accurate grayscale segmentation, and morphological processing removes interference.
Fault Detection and Analysis
After image extraction, fault detection is performed. By extracting SIFT features, I initially classify the panels. Then, the BOW model is applied to the second category to remove interference and obtain true “normal” and “faulty” items.
In performance testing, I used 60 prior samples with known faults. The detection accuracy ranged from 85% to 92%, with a recall of 90% and precision between 80% and 90%. False positives were within a reasonable range. The Receiver Operating Characteristic (ROC) curve for this technique achieved an Area Under Curve (AUC) of 0.90, indicating high reliability. Additionally, I introduced slack variables to prevent overfitting in the SVM, making the algorithm more robust.
| Metric | Value |
|---|---|
| Detection Accuracy | 85% – 92% |
| Recall | 90% |
| Precision | 80% – 90% |
| AUC (ROC) | 0.90 |
| Processing Speed | 0.06 s per panel |
The fault detection system processes images through preprocessing, stitching, segmentation, and localization. The hot spot detection module processes the infrared radiation images captured by the UAV to identify the location of hot spots. I tested the algorithm on an AMD processor (2.6 GHz) multi-threaded environment with 120 samples. The algorithm took approximately 4 seconds to process the entire set, corresponding to a detection speed of 0.06 seconds per solar panel image. Image preprocessing involves noise reduction and enhancement to improve clarity for stitching. I use a scale-invariant feature point image stitching algorithm to seamlessly merge overlapping images. Segmentation is performed using the OTSU thresholding method to distinguish areas with the maximum variance, thereby pixel-wise locating the hot spots on the solar panels.
Advantages of UAV Intelligent Inspection
By combining visual neural image processing, pattern recognition, and UAV technology, my system takes full advantage of the “photovoltaic + UAV + intelligence” development model. Compared to traditional manual inspection, UAV-based intelligent inspection offers significantly higher efficiency – detection speed is thousands of times faster than manual inspection – along with high accuracy and intuitive visualization. I believe this approach will become the dominant method for detecting surface and near-surface defects in solar panels.
Application to Quality Control in Solar Panel Production
During manufacturing, complex processes can introduce subtle defects. If defective solar panels are deployed, they severely compromise power generation efficiency. Applying this technology enables batch inspection of large quantities of solar panels, allowing timely detection and repair of faulty units, thus ensuring product quality and reducing energy waste caused by manufacturing defects.
Application to Large, Medium, and Small Photovoltaic Power Plants
With the rapid growth of the photovoltaic industry, integrating UAV technology and image processing for hot spot detection in solar panels overcomes the limitations of traditional manual inspection. This system can quickly, accurately, safely, and stably complete tasks such as acquisition, stitching, segmentation, calibration, and classification of infrared radiation images of solar panels. By analyzing the development of the PV industry and the characteristics of hot spot phenomena, and studying hot spot detection and UAV inspection technologies, I have demonstrated the urgency and feasibility of developing an efficient automated hot spot inspection system for PV plants. The UAV-mounted thermal imaging hot spot detection system, as compared to existing technologies, defines the overall design framework and communication methods, and studies related image processing techniques. Based on existing algorithms, it achieves hot spot extraction and pixel coordinate localization on infrared images of solar panels, making it well-suited for application in large, medium, and small photovoltaic power stations.
Conclusion
Given the high failure rates and complex interconnections in traditional photovoltaic arrays, system power generation is directly impacted. My system uses thermal imaging to quickly and accurately identify hot spots on solar panels, enabling timely screening and repair. Combined with cloud platform data analysis, statistical methods, and clustering algorithms, it provides targeted rapid detection of fault areas and types. By monitoring data anomalies across the entire network, comprehensive and efficient circuit fault diagnosis is achieved. In future system upgrades, a Raspberry Pi and thermal imager could be mounted for real-time onboard detection: the Raspberry Pi periodically reads images from the thermal imager, and the rapid fault detection system performs segmentation and detection. If a fault is found, the Raspberry Pi transmits the GPS coordinates marked by the thermal imager back to the ground, further accelerating processing.
The core mathematical formulation for the OTSU thresholding is:
$$ \sigma^2_{\text{between}}(t) = \omega_0(t) \omega_1(t) [\mu_0(t) – \mu_1(t)]^2 $$
where \(t\) is the threshold, \(\omega_0\) and \(\omega_1\) are the class probabilities, and \(\mu_0\), \(\mu_1\) are the class means.
The SIFT keypoint descriptor is a 128-dimensional vector, and the clustering objective for the BOW dictionary is:
$$ \arg\min_{\{c_k\}} \sum_{i=1}^N \min_{k} \| x_i – c_k \|^2 $$
where \(x_i\) are the SIFT descriptors and \(c_k\) are cluster centers.
The SVM classifier solves:
$$ \min_{\mathbf{w}, b, \xi} \frac{1}{2} \|\mathbf{w}\|^2 + C \sum_{i=1}^n \xi_i $$
$$ \text{subject to } y_i (\mathbf{w}^T \phi(x_i) + b) \geq 1 – \xi_i, \quad \xi_i \geq 0 $$
with slack variables \(\xi_i\) to handle non-separable data.
In summary, I have designed a comprehensive intelligent system for automatic inspection and modeling of solar panels using UAVs. The system combines cutting-edge computer vision and machine learning with reliable hardware, delivering fast, accurate, and practical fault detection. It significantly reduces the burden of manual inspection, improves maintenance efficiency, and contributes to the sustainable growth of the photovoltaic industry. The modular design allows easy adaptation to various scales of solar power plants, and the cloud-based data storage facilitates continuous improvement through big data analytics. I am confident that this intelligent system will become an essential tool for modern solar panel maintenance and will play a crucial role in advancing renewable energy utilization worldwide.