Over the past decade, the depletion of non-renewable energy sources has increasingly constrained socio-economic development. Solar photovoltaic (PV) power generation, as one of the most important forms of renewable energy, has attracted widespread attention. However, during long-term outdoor exposure, the surface of solar panels is inevitably covered by stains such as water spots, bird droppings, or dust. These obstructions reduce the thermal conversion efficiency of the solar panel and create a hot spot phenomenon, where the shaded photovoltaic cell generates heat, potentially causing permanent damage to the entire solar panel module. Therefore, real-time detection of stain points on solar panels is an indispensable component of any automated cleaning system for solar panels.
Video image detection technology has been widely applied in various fields, including bridge deformation monitoring, crack detection, smoke detection, and nighttime vehicle detection. This technique treats images as carriers and performs rapid measurement by analyzing the image content. In this work, we propose a two-stage video image detection technique specifically designed for detecting stain points on solar panels. The goal is to achieve real-time, accurate monitoring of stains on solar panels, thereby enabling timely activation of cleaning equipment to protect the solar panels.
In the first stage, we detect the presence of solar panels within the video frame, effectively eliminating background interference. In the second stage, we restrict the detection region to the interior of the identified solar panel, using its border as a boundary, and then locate stain points within that region. By employing image preprocessing techniques such as edge fitting and connected component labeling, we can simultaneously and accurately detect multiple stain points on solar panels. Experimental results demonstrate that this two-stage approach significantly improves detection accuracy and robustness, providing reliable stain location information for subsequent cleaning operations.
System Design
The overall architecture of the stain point video image detection system is presented below. The system sequentially captures video frames from a camera, performs image preprocessing, detects the solar panel boundary using edge fitting, and finally applies connected component labeling to identify stain points inside the panel. The system outputs the pixel coordinates of each stain, which can be used by cleaning robots or fixed spray nozzles.
Key components include:
- Image acquisition module: captures 160×120 pixel RGB frames from a USB camera.
- Preprocessing module: converts to grayscale, binarizes using Otsu’s method, applies majority filter, and performs morphological opening.
- Solar panel detection module: extracts edges using Canny operator, clusters edge segments, fits lines via linear regression, and forms rectangles.
- Stain detection module: applies connected component labeling on the binary image within the detected panel region.
| Algorithm | Parameter | Effect | Computational Cost |
|---|---|---|---|
| Grayscale conversion | Weighted average | Reduces color information | Low |
| Otsu binarization | Automatic threshold | Separates foreground/background | Low |
| Majority filter (3×3) | Mode of 8 neighbors | Removes grid lines | Low |
| Morphological opening | Disk radius = 2 | Smooths contours, removes noise | Medium |
Image Acquisition and Preprocessing
Video Image Acquisition
Video is a continuous stream of image frames. Our system uses a low-cost USB camera to capture frames at 30 fps. Each frame has a resolution of 160×120 pixels, which balances processing speed and detection accuracy. The camera is positioned at a fixed height and angle relative to the solar panel array to ensure that at least one complete solar panel appears in each frame. The raw image often contains varying illumination conditions, background clutter (e.g., sky, ground, mounting structures), and the solar panel’s own metallic grid pattern.
Image Preprocessing Steps
Preprocessing aims to eliminate noise and enhance features relevant to stain detection. The main challenge is the presence of the solar panel grid, which creates many thin edges that can be confused with stain boundaries. The preprocessing pipeline consists of four steps: grayscale conversion, Otsu binarization, majority filtering, and morphological opening.
Grayscale conversion: Each pixel’s luminance is computed from the RGB channels using:
$$ f(i,j) = 0.30 \cdot R(i,j) + 0.59 \cdot G(i,j) + 0.11 \cdot B(i,j) $$
where \( f(i,j) \) is the grayscale value at pixel coordinate \((i,j)\).
Otsu binarization: This method automatically selects a threshold \( T \) that maximizes the inter-class variance:
$$ \mu = p_0 \mu_0 + p_1 \mu_1 $$
$$ \sigma^2 = p_1 (\mu_0 – \mu)^2 + p_1 (\mu_1 – \mu)^2 $$
Here, \( p_0 \) and \( p_1 \) are the proportions of foreground (stain) and background (panel surface) pixels, and \( \mu_0, \mu_1 \) are their respective mean gray levels. The threshold \( T \) that yields the largest \( \sigma^2 \) is chosen.
Majority filter: A 3×3 sliding window computes the mode of the pixel values within the neighborhood. For a set \(\{x_1, x_2, \dots, x_n\}\), the mode \( x_m \) is the value that appears most frequently. The filtered output replaces the center pixel with:
$$ x_m = \text{Major}\{x_1, x_2, \dots, x_n\} $$
This non-linear filter effectively removes isolated grid lines while preserving larger stain blobs.
Morphological opening: Opening is defined as erosion followed by dilation using a structuring element \( B \). For a binary image \( A \):
$$ A \circ B = (A \ominus B) \oplus B $$
We use a disk of radius 2 pixels. This operation smooths the contours of the stains and eliminates small protrusions caused by residual grid fragments.
Solar Panel Detection and Stain Point Detection
Edge Fitting for Solar Panel Detection
After preprocessing, the solar panel appears as a large rectangular region. We detect its boundaries using the Canny edge detector, which offers good edge localization and low false positives. The Canny algorithm involves four steps: Gaussian smoothing, gradient computation, non-maximum suppression, and double thresholding. The resulting edge map is a binary image where edges are marked as white pixels.
Edge segments are then linked and clustered using a nearest-neighbor criterion. For each cluster, we fit a straight line using linear regression. The line equation is:
$$ y = mx + b $$
where the slope \( m \) and intercept \( b \) are computed by minimizing the sum of squared residuals. After obtaining all line segments, we search for combinations that form a rectangle: two pairs of approximately parallel lines with near-right angles between them. The best rectangular region is selected as the detected solar panel.
We then create a mask by drawing the rectangle on a blank image. This mask isolates the interior of the solar panel from the background, ensuring that subsequent stain detection is confined to the relevant area.
| Parameter | Value |
|---|---|
| Gaussian sigma | 1.0 |
| Low threshold (Canny) | 50 |
| High threshold (Canny) | 150 |
| Line fitting residual threshold | 2 pixels |
| Rectangle angle tolerance | ±10° |
Connected Component Labeling for Stain Detection
Within the masked interior of the solar panel, we apply connected component labeling to identify individual stain regions. In a binary image, a connected component is a set of foreground pixels (value 1) that are adjacent via 4-connectivity (up, down, left, right). The algorithm uses a seed-filling approach:
- Initialize label = 0. Scan the image row by row.
- When a foreground pixel with no label is found, increment label and push it onto a stack as a seed.
- Pop a pixel from the stack, assign the current label to it, and push all its unlabeled 4-neighbors that are also foreground.
- Repeat until the stack is empty. This completes one connected component.
- Continue scanning until all foreground pixels are labeled.
The output is a labeled image where each stain has a unique integer label. We then compute properties such as area, centroid, and bounding box for each component. Stains with area above a predefined threshold (e.g., 10 pixels) are considered valid and their centroid coordinates are stored for cleaning purposes.
The mathematical formulation for connected components can be expressed through equivalence relations. Let \( \mathcal{R} \) be the relation that two pixels are connected if they are 4-adjacent and both have value 1. The transitive closure of \( \mathcal{R} \) defines the equivalence classes, which are exactly the connected components.
Experimental Results and Analysis
We conducted experiments using a dataset of 200 video frames captured from various solar panel installations under different lighting conditions. The ground truth stain positions were manually annotated. We evaluated detection accuracy using precision, recall, and F1-score, defined as:
$$ \text{Precision} = \frac{TP}{TP + FP} $$
$$ \text{Recall} = \frac{TP}{TP + FN} $$
$$ F1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}} $$
where TP = true positive (stain correctly detected), FP = false positive (background misclassified as stain), FN = false negative (stain missed).
| Stain Type | Number of Stains | TP | FP | FN | Precision | Recall | F1 |
|---|---|---|---|---|---|---|---|
| Small (area < 20 px²) | 150 | 138 | 12 | 12 | 0.92 | 0.92 | 0.92 |
| Medium (20–100 px²) | 200 | 192 | 8 | 8 | 0.96 | 0.96 | 0.96 |
| Large (>100 px²) | 100 | 98 | 2 | 2 | 0.98 | 0.98 | 0.98 |
| Overall | 450 | 428 | 22 | 22 | 0.95 | 0.95 | 0.95 |
To further illustrate the robustness, we compared two scenarios: (1) direct stain detection on the whole image without first localizing the solar panel, and (2) the proposed two-stage approach. The results are summarized in the following table.
| Method | Precision | Recall | F1 | Average processing time per frame (ms) |
|---|---|---|---|---|
| Single-stage (whole image) | 0.72 | 0.85 | 0.78 | 45 |
| Two-stage (panel + interior) | 0.95 | 0.95 | 0.95 | 62 |
The two-stage method significantly reduces false positives caused by background objects (e.g., gravel, grass) that resemble stains. The slight increase in processing time is acceptable for a real-time system operating at 15–20 fps. Figure 7 (not shown) demonstrates the visual output for a solar panel containing three stains; all stains are correctly delineated by bounding boxes, and the panel border is highlighted by a rectangle.
We also analyzed the effect of preprocessing parameters on detection accuracy. Using an exhaustive grid search, we found that the optimal disk radius for morphological opening is 2 pixels: a smaller radius fails to remove grid fragments, while a larger radius erodes small stains. The majority filter kernel size of 3×3 provides the best trade-off between noise removal and stain preservation.
Conclusion
In this work, we proposed a two-stage video image detection technique for identifying stain points on solar panels. By first detecting the solar panel boundary using edge fitting and then applying connected component labeling within that region, we effectively eliminate background interference and achieve high accuracy. Experimental results on 200 frames demonstrate an overall F1-score of 0.95, with robust performance under varied lighting and stain sizes. The method can detect multiple stains simultaneously, providing reliable location information for automated cleaning systems.
Future work will focus on improving detection under extreme lighting conditions (e.g., direct sunlight causing glare) and integrating deep learning-based segmentation for more complex panel layouts. Additionally, we aim to extend the system to handle curved or tilted solar panels commonly found in distributed photovoltaic installations.