In the operation and maintenance of solar power plants, the inspection of solar panel surfaces has traditionally been a labor-intensive and inefficient process. Manual inspection often leads to delayed fault detection and high costs. To address these challenges, we propose a novel method that combines unmanned aerial vehicles (UAVs) with a state-of-the-art object detection algorithm to intelligently identify foreign objects on solar panel surfaces. The study focuses on using the Single Shot Multibox Detector (SSD) algorithm to detect anomalies such as dust, bird droppings, leaves, and other debris that can significantly reduce the power generation efficiency of solar panels. By leveraging UAVs for aerial imagery and deep learning for automated analysis, we aim to construct an intelligent maintenance system for solar panels, thereby improving their operational efficiency and reducing manual intervention.
The remainder of this paper is organized as follows: Section 1 introduces the background and related work. Section 2 describes the data collection process. Section 3 details the SSD algorithm and its application. Section 4 presents the experimental setup and results. Section 5 provides an evaluation and discussion. Section 6 concludes the study.
1. Introduction
The rapid growth of the photovoltaic industry has brought increasing pressure on the operation and maintenance (O&M) of solar panel arrays. Conventional O&M methods, such as manual inspection and patrolling, suffer from long fault identification times and low accuracy. In recent years, object detection techniques have evolved rapidly, giving rise to efficient methods like R-CNN, Faster R-CNN, YOLO, and SSD. Among these, the SSD algorithm offers an excellent balance between speed and accuracy, making it suitable for real-time detection tasks. However, its application in the specific domain of solar panel surface inspection remains limited.
International studies have primarily focused on the theoretical relationship between solar panel components and power generation efficiency, with emphasis on material science for cleaning. Domestic research includes probabilistic fault tree analysis, least-squares support vector regression (Ls-SVR) for fault detection, and automated fault detection systems. Despite these efforts, the integration of SSD with UAV-based inspection has been largely unexplored. Our work bridges this gap by applying the SSD algorithm to UAV-captured visible-light images of solar panels, enabling automatic identification and localization of surface foreign objects.
2. Data Source
Our study was conducted on a rooftop photovoltaic array at an industrial facility. A UAV equipped with a visible-light camera was programmed to fly along a fixed route over the solar panel field. The UAV captured both video and still images under normal daylight conditions. The collected data recorded various types of foreign objects on the solar panel surfaces, including dust accumulation, organic matter, and physical obstructions. This dataset forms the foundation for training and evaluating the detection model.
3. Model Methodology
3.1 SSD Algorithm Network Structure
The SSD network architecture is based on the VGG16 backbone, with additional convolutional layers of varying sizes. The key idea is to replace the fully connected layers of traditional classifiers with convolutional layers, enabling the network to output predictions for multiple feature maps at different scales. The input to the model is a raw visible-light image captured by the UAV. After passing through the VGG16 feature extractor, the network processes the feature maps through a series of progressively smaller convolutional layers. Each of these layers generates default boxes (also known as anchor boxes) of specific scales and aspect ratios. For each default box, the model predicts both the class probabilities and the offsets relative to the box coordinates.
The SSD algorithm was first proposed by Liu et al. (2016). It combines the anchor mechanism of Faster R-CNN with the regression approach of YOLO, achieving end-to-end training in a single shot. Compared with YOLO, SSD offers significantly improved localization and classification accuracy. The loss function of SSD consists of two components: the confidence loss (classification) and the localization loss (bounding box regression). The total loss is a weighted sum:
$$L(x, c, l, g) = \frac{1}{N} \left( L_{conf}(x, c) + \alpha L_{loc}(x, l, g) \right)$$
where \(N\) is the number of matched default boxes (including both positive and negative samples), \(\alpha\) is the weighting factor (default 1), and \(x_{ij}^p\) indicates whether the \(i\)-th default box matches the \(j\)-th ground-truth box of class \(p\). The localization loss uses the smooth L1 loss function:
$$\text{smooth}_{L1}(x) = \begin{cases} 0.5x^2, & \text{if } |x| < 1 \\ |x| – 0.5, & \text{otherwise} \end{cases}$$
Thus, the localization loss is defined as:
$$L_{loc}(x, l, g) = \sum_{i \in Pos}^N \sum_{m \in \{cx, cy, w, h\}} x_{ij}^k \text{smooth}_{L1}\left( l_i^m – \hat{g}_j^m \right)$$
where
$$\hat{g}_j^{cx} = \frac{g_j^{cx} – d_i^{cx}}{d_i^w}, \quad \hat{g}_j^{cy} = \frac{g_j^{cy} – d_i^{cy}}{d_i^h}, \quad \hat{g}_j^{w} = \log \left( \frac{g_j^w}{d_i^w} \right), \quad \hat{g}_j^{h} = \log \left( \frac{g_j^h}{d_i^h} \right)$$
The confidence loss is a softmax loss over multiple classes:
$$L_{conf}(x,c) = -\sum_{i \in Pos}^N x_{ij}^p \log \left( \hat{c}_i^p \right) – \sum_{i \in Neg} \log \left( \hat{c}_i^0 \right)$$
with \(\hat{c}_i^p = \frac{\exp(c_i^p)}{\sum_p \exp(c_i^p)}\). Each default box is matched to ground-truth boxes using the intersection-over-union (IoU) criterion.
3.2 Intersection-over-Union (IoU)
IoU is a key metric in object detection that measures the overlap between a predicted bounding box and a ground-truth box. It is defined as the ratio of the area of intersection to the area of union:
$$\text{IoU} = \frac{G_t \cap D_r}{G_t \cup D_r}$$
where \(G_t\) is the ground-truth box and \(D_r\) is the detection result. In the SSD training strategy, a default box is considered a positive sample if its IoU with any ground-truth box exceeds a threshold (typically 0.5); otherwise, it is a negative sample. After generating many candidate boxes, non-maximum suppression (NMS) is applied to select the most confident detections and eliminate redundant ones.
4. Experimental Process
4.1 Data Preparation
The UAV-collected videos were processed using Adobe Premiere Pro CC to extract frames. We segmented the videos into clips containing foreign objects and those without. From the relevant segments, frames were exported at a step size of 6 frames, yielding a total of 1,363 images showing foreign objects on solar panels. Each image was annotated using the Labellmg tool, which generated corresponding XML files containing bounding box coordinates and class labels. The dataset was then split into training and testing sets as shown in Table 1.
| Data Source | Images | XML Files |
|---|---|---|
| Training Set | 1,030 | 1,030 |
| Testing Set | 313 | 313 |
| Total | 1,363 | 1,363 |
4.2 Experimental Procedure
We implemented the SSD algorithm using the TensorFlow Object Detection API. The positive-to-negative sample ratio was set to approximately 1:3. The XML files were first converted to CSV format and then to TFRecord format for efficient training. The model was trained with a MobileNet V1 backbone (pre-trained on COCO) for 30,000 steps on a GeForce RTX 2060 GPU. The architecture consists of a base network (MobileNet), auxiliary convolutional layers, and prediction layers. During training, the model matches default boxes to ground-truth boxes using the IoU criterion. Only default boxes with IoU > 0.5 are considered positive samples. To balance the number of samples, negative samples are randomly selected so that the positive-to-negative ratio is about 1:3. The total loss is minimized using stochastic gradient descent.
After training, the model with the smallest total loss among the last five checkpoints was selected and exported as a frozen inference graph (`.pb` file) for deployment. The training process was monitored using TensorBoard, and the loss curve is shown in the Results section.
5. Results and Evaluation
The overall loss value decreased steadily during training, as illustrated by the TensorBoard visualization. The model at 30,000 steps achieved a sufficiently low loss and was chosen for final testing. We evaluated the trained model on the test set of 313 images. For each test image, the model predicted bounding boxes and class labels. The performance was measured using the IoU metric: a detection was considered correct if its IoU with the ground-truth box exceeded 0.5. The average IoU across all true positive detections was above 75%.
Table 2 summarizes the training results and final accuracy.
| Model | GPU | Dataset | Training Time | Training Steps | Average IoU (%) |
|---|---|---|---|---|---|
| ssd_mobilenet_v1_pets | GeForce RTX 2060 | 1,363 images | 4.5 hours | 30,000 | 75 |
We also conducted live tests with new images captured by the UAV. The model successfully detected foreign objects on solar panels, drawing bounding boxes with confidence scores. Figure 1 (represented by the above image) shows a sample detection result, where a bird dropping on a solar panel is correctly identified and boxed. The IoU of this detection was 78%, indicating high localization accuracy. In general, detections with IoU greater than 75% accounted for the majority of true positives, demonstrating the robustness of the proposed method.
6. Discussion
The experimental results confirm that the SSD algorithm, when applied to UAV-captured visible-light images, can effectively recognize foreign objects on solar panel surfaces. The average IoU of 75% is competitive with state-of-the-art detection methods, and the inference speed is sufficient for real-time UAV processing. Compared with manual inspection, our approach reduces the need for human labor and minimizes downtime for solar panels. Furthermore, the model can be integrated with automated cleaning robots to trigger localized cleaning actions, thereby improving the overall energy yield of the photovoltaic system.
Several factors contribute to the success of the method. First, the use of a pretrained MobileNet backbone accelerates convergence and reduces overfitting. Second, the multi-scale feature maps in SSD allow detection of objects of various sizes, which is important because foreign objects on solar panels can range from small dust specks to large leaves. Third, the data augmentation techniques (e.g., random cropping, flipping) applied during training enhance generalization. However, there is still room for improvement. For example, the performance under low-light conditions or on textured surfaces (e.g., bifacial solar panels) may degrade. Future work could explore adding infrared or multispectral channels to improve robustness. Additionally, incorporating temporal information from video sequences could further reduce false positives.
7. Conclusion
In this study, we proposed an intelligent detection system for foreign objects on solar panel surfaces using UAV imagery and the SSD object detection algorithm. We collected a dataset of 1,363 annotated images from a rooftop photovoltaic plant, trained an SSD-MobileNet model, and achieved an average IoU of 75% on the test set. The system is capable of real-time detection and can be seamlessly integrated into automated O&M workflows for solar panel arrays. By reducing manual inspection efforts and enabling swift cleaning interventions, this approach contributes to higher efficiency and lower costs in solar energy production. Future enhancements will focus on multi-modal sensing and edge deployment to further improve the practicality and scalability of the system.