In my research, I focus on the problem of detecting and counting solar panels from high-resolution aerial images. The rapid expansion of photovoltaic systems worldwide demands efficient automated monitoring and asset management. Traditional manual counting is slow, error-prone, and often impractical for large industrial rooftops. Image-based target detection offers a promising solution, but conventional computer vision methods suffer from sensitivity to lighting conditions and require hand-crafted features. I therefore turn to deep convolutional neural networks (CNNs), which have proven superior in numerous visual recognition tasks. In this paper, I present my work on building a photovoltaic panel detection algorithm based on CNN, with particular emphasis on improving detection accuracy for densely packed solar panels in aerial imagery.
My study is driven by the real-world need to accurately localize and count individual solar panels in industrial zones. From aerial images captured by drones, I aim to detect every solar panel unit so that energy companies can monitor installation status, plan maintenance, and conduct asset inventories. The main challenges include the large size of aerial images, the relatively small size of each solar panel, dense arrangement of panels, and variability in lighting and viewing angles. My approach integrates several key components: a carefully constructed dataset, a two-stage detection network with a strong feature extractor, a sliding-window strategy to handle high-resolution inputs, and a channel attention mechanism to enhance feature representation. I will describe the complete methodology and present experimental results that demonstrate significant improvements over baseline methods.
1. Introduction and Motivation
The global adoption of solar energy has accelerated in recent years. With increasing installations of photovoltaic systems, the need for automated monitoring and counting of solar panels becomes critical. My work is motivated by industrial applications such as asset inspection, project acquisition, and internal auditing. Accurate counting of solar panels helps companies evaluate power generation capacity and plan future installations.
Traditional methods for counting solar panels often rely on manual inspection of aerial images. This is tedious, time-consuming, and subject to human error. Although some earlier approaches used image processing techniques, they generally required manual threshold tuning and were sensitive to outdoor illumination changes. For instance, matching algorithms based on FAST, SIFT, or SURF can fail when lighting conditions vary. Contour-based methods may require adjustment of thresholds for different images. These limitations reduce the applicability of conventional computer vision approaches to solar panel detection.
Deep learning, especially CNN-based object detection, has revolutionized the field of computer vision. Networks such as Faster R-CNN, SSD, YOLO, and their variants have achieved remarkable performance on natural image benchmarks. However, aerial images pose unique challenges: they are typically very large (e.g., 4000×3000 pixels), while the objects of interest are small and often densely arranged. Directly feeding such large images into a CNN usually requires aggressive downsampling, which loses critical detail. My research addresses these challenges by combining a strong two-stage detector with sliding-window inference and a channel attention mechanism.
2. Convolutional Neural Network Foundations
A CNN is a feedforward neural network designed to process grid-like data such as images. It consists of multiple layers, including convolutional layers, activation functions, pooling layers, and fully connected layers. The convolutional layers apply learnable filters to extract local features. Activation functions introduce non-linearity, with ReLU being the most common choice due to its computational efficiency and ability to mitigate vanishing gradients. Pooling layers reduce spatial dimensions and increase the receptive field.
Two fundamental properties of CNNs are local receptive fields and weight sharing. Local receptive fields mean that each neuron only connects to a small region of the previous layer, reflecting the local correlation of images. Weight sharing means that the same filter is applied across the entire image, dramatically reducing the number of parameters. These properties make CNNs highly effective for extracting hierarchical features from raw pixels.
Mathematically, the forward propagation of a convolutional layer can be expressed as:
$$a_j^l = \sigma\left(\sum_k w_{jk}^l * a_k^{l-1} + b_j^l\right)$$
where \(a_j^l\) is the activation of the \(j\)-th feature map in layer \(l\), \(w_{jk}^l\) is the convolutional kernel connecting the \(k\)-th feature map in layer \(l-1\) to the \(j\)-th feature map in layer \(l\), \(b_j^l\) is the bias term, and \(\sigma\) is the activation function. The symbol \(*\) denotes convolution.
During training, the network minimizes a loss function via backpropagation, which computes gradients of the loss with respect to all weights and biases. The weights are updated using gradient descent or its variants. This iterative process allows the CNN to learn task-specific features from data, eliminating the need for manual feature engineering.
3. Object Detection Architectures
Object detection methods based on deep learning can be roughly divided into two categories: one-stage detectors and two-stage detectors. One-stage detectors, such as SSD and YOLO, directly predict object classes and bounding boxes without a separate proposal stage. They are usually faster but may be less accurate, especially for small objects. Two-stage detectors, such as R-CNN, Fast R-CNN, and Faster R-CNN, first generate region proposals and then classify and refine them. They tend to achieve higher accuracy, particularly for small or dense objects.
3.1 Faster R-CNN
Faster R-CNN is one of the most influential two-stage detectors. It introduces the Region Proposal Network (RPN) to generate candidate boxes from feature maps, replacing the slow selective search method. The architecture can be summarized as follows:
- Input an image and resize it to a fixed size (e.g., 1000×600).
- Feed the image through a feature extraction network (e.g., VGG16, ResNet) to obtain feature maps.
- Pass the feature maps to the RPN, which generates a set of region proposals (anchors).
- Use ROI Pooling to extract fixed-size feature vectors from each proposal.
- Pass these vectors to fully connected layers for classification and bounding-box regression.
The RPN uses anchors of different scales and aspect ratios. For each anchor, it predicts the probability of being foreground or background and refines the box coordinates. During training, Faster R-CNN alternates between training the RPN and training the Fast R-CNN detection head, enabling parameter sharing between the two modules.
3.2 SSD and YOLO
SSD (Single Shot MultiBox Detector) is a one-stage detector that predicts bounding boxes from multiple feature maps of different resolutions. It uses a set of default boxes with various scales and aspect ratios. SSD is faster than Faster R-CNN but often less accurate on small objects. YOLO (You Only Look Once) treats detection as a regression problem and divides the image into a grid, with each cell predicting a fixed number of boxes. YOLO is extremely fast but suffers from poor performance on densely packed small objects, making it unsuitable for my solar panel task.
4. Feature Extraction Networks
The choice of backbone network greatly affects detection performance. I consider VGG16 and two residual networks, ResNet50 and ResNet101.
4.1 VGG16
VGG16 consists of 13 convolutional layers and 3 fully connected layers. It uses small 3×3 filters throughout the network. Stacking multiple 3×3 convolutions increases the receptive field while reducing the number of parameters compared to larger filters. VGG16 is simple and effective but has a large number of parameters and is relatively slow due to the fully connected layers.
4.2 ResNet
Deep networks often suffer from vanishing gradients and degradation. ResNet introduces residual blocks with skip connections, allowing the network to learn residual mappings. A residual block can be formulated as:
$$y = \mathcal{F}(x, \{W_i\}) + x$$
where \(x\) is the input, \(\mathcal{F}\) represents the convolution operations, and \(y\) is the output. The direct connection \(x\) helps gradient flow and enables training of very deep networks. ResNet50 and ResNet101 have 50 and 101 layers, respectively. They consistently achieve better feature representation than VGG on many tasks.
5. Dataset Construction and Preprocessing
Since there is no public dataset for high-resolution aerial solar panel images, I constructed my own dataset. The images were captured by an unmanned aerial vehicle (UAV) in an industrial area in Guilin, China. The dataset consists of 150 high-resolution RGB images, each with a size of 4000×3000 pixels. The flight altitude ranged from 88 to 152 meters, and most images were taken from a near-vertical perspective. The images contain industrial rooftops with many regularly arranged solar panel units. One image can contain over 400 solar panels, making manual counting extremely challenging.
I randomly split the dataset into training, validation, and test sets with a ratio of 6:2:2, resulting in 120 images for training and validation (with a further split) and 30 images for testing. To create ground truth annotations, I used the LabelImg tool to draw bounding boxes around every complete solar panel in each image. The annotations were saved as XML files containing the image size, object class names, and bounding box coordinates.
To increase data diversity and reduce overfitting, I performed data augmentation during training. Specifically, I applied random horizontal flips and random contrast adjustments with a scaling factor uniformly sampled between 0.8 and 1.25. These operations simulate different lighting conditions and help the model generalize better.
An additional important preprocessing step is cropping. Because the original images are extremely large (4000×3000), feeding them directly into the network would cause severe downsampling and loss of solar panel details. I therefore crop each image into smaller patches of size 1000×1000. With overlap? No, during training I used non-overlapping cropping? Actually, in my implementation, I cropped the images into 1000×1000 patches without overlap for training. This yielded 1493 training patches from the training and validation images. For testing, I used a sliding-window approach with overlapping windows, as will be described later.
6. Baseline Experiment: Faster R-CNN vs SSD
I first compared two representative detection networks: Faster R-CNN with VGG16 and SSD with VGG16. Both networks were initialized with weights pre-trained on the VOC dataset and fine-tuned on my solar panel dataset. During training, I adjusted some hyperparameters to accommodate the dense nature of solar panels. For instance, I set the maximum number of bounding boxes used in training to 400, and resized images to 800×1000 for Faster R-CNN. For SSD, I used the input size of 512×512 and set the maximum number of predictions to 600.
Table 6.1 shows the detection results on a representative test image containing 400 solar panels.
| Network | Ground Truth | Detected | Missed | False Positives | Detection Rate |
|---|---|---|---|---|---|
| SSD(VGG16) | 400 | 198 | 201 | 3 | 0.4950 |
| Faster R-CNN(VGG16) | 400 | 300 | 100 | 0 | 0.7500 |
The results clearly show that Faster R-CNN outperforms SSD on solar panel detection. SSD produces many missed detections and a few false positives. This is because SSD uses default boxes from multiple feature maps, but for small and densely packed objects, the high-level feature maps do not provide sufficient spatial detail. Faster R-CNN, with its region proposal mechanism and subsequent refinement, achieves better localization. Therefore, I selected Faster R-CNN as the base detection framework for the rest of my research.
7. Improving the Feature Extraction Backbone
The default Faster R-CNN uses VGG16 as its backbone. Although VGG16 works reasonably, its feature extraction capability is limited compared to deeper residual networks. I hypothesized that using ResNet as the backbone would improve solar panel detection because solar panels have simple but repetitive textures; deeper networks can learn richer semantic features. I experimented with ResNet50 and ResNet101.
The modified Faster R-CNN architecture replaces VGG16 with ResNet. The ResNet outputs a feature map from its final convolutional stage, which is then fed into the RPN and ROI pooling. This change preserves the end-to-end training capability while enhancing feature extraction.
I evaluated the three backbones (VGG16, ResNet50, ResNet101) on the same test set. Table 7.1 presents the average precision (AP, which equals mAP for a single class) and total detection time on all test images.
| Backbone | Average Precision (%) | Time (s) |
|---|---|---|
| VGG16 | 67.99 | 35.7335 |
| ResNet50 | 69.63 | 22.0168 |
| ResNet101 | 69.61 | 28.6873 |
Both ResNet models achieve higher AP than VGG16. ResNet50 is slightly better than ResNet101 in terms of AP and significantly faster. The reason could be that ResNet101 has a larger capacity but the limited amount of training data leads to some overfitting. Therefore, I chose ResNet50 as the optimal backbone for solar panel feature extraction.
8. Sliding Window Detection for High-Resolution Images
Even with ResNet50, the original input method still requires resizing the large aerial image to a fixed size such as 1000×600, which severely reduces the apparent size of each solar panel. To address this issue, I adopt a sliding-window approach combined with cropping during training.
In the training phase, I input the cropped 1000×1000 patches to the network instead of the full images. This enables the model to learn features from a scale where solar panels are relatively larger and more detailed. In the inference phase, I use a sliding window of the same size (1000×1000) with a stride of 500 pixels to scan the full original image. Because the stride is smaller than the window size, adjacent windows overlap. Overlapping ensures that solar panels near window boundaries are not missed. However, overlap also leads to duplicate detections of the same solar panel. To remove duplicates, I apply non-maximum suppression (NMS) to all predicted bounding boxes collected from all windows.
The algorithm can be summarized as follows:
- Preprocess the original image (augmentation if training).
- Crop the image into patches using a sliding window technique.
- Feed each patch through the trained Faster R-CNN (with ResNet50) to obtain detections.
- Aggregate all detections from all patches.
- Apply NMS to eliminate overlapping boxes that refer to the same solar panel.
Table 8.1 compares the detection results before and after adding the sliding window mechanism on three representative test images (A, B, C).
| Image | Method | Actual Panels | Detected | Missed | Detection Rate |
|---|---|---|---|---|---|
| A | Faster R-CNN(ResNet50) | 288 | 77 | 211 | 0.2673 |
| A | + Sliding Window | 288 | 282 | 6 | 0.9791 |
| B | Faster R-CNN(ResNet50) | 396 | 207 | 189 | 0.5227 |
| B | + Sliding Window | 396 | 387 | 9 | 0.9772 |
| C | Faster R-CNN(ResNet50) | 168 | 167 | 1 | 0.9940 |
| C | + Sliding Window | 168 | 168 | 0 | 1.0000 |
The improvement is dramatic. Detection rate for image A increased from 26.73% to 97.91%. Image B increased from 52.27% to 97.72%. Image C reached 100%. The sliding window approach effectively prevents the loss of small-object features due to downsampling. However, it also increases computational load. Table 8.2 shows the overall AP and time on the entire test set.
| Method | mAP (%) | Total Time (s) |
|---|---|---|
| Faster R-CNN(ResNet50) | 67.99 | 35.7335 |
| Faster R-CNN(ResNet50) + Sliding Window | 86.00 | 479.3220 |
The AP increases from 67.99% to 86.00%, which is a substantial boost. The inference time is much longer, but for the application of industrial asset counting, accuracy is more critical than real-time speed.
9. Attention Mechanisms for Solar Panel Detection
Even with sliding windows, there are still some missed detections. To further improve feature representation, I investigate attention mechanisms. Attention models allow the network to focus on important features and suppress irrelevant ones. In this work, I examine two popular attention modules: SENet (Squeeze-and-Excitation) and CBAM (Convolutional Block Attention Module).
9.1 Channel Attention: SENet
SENet models the interdependencies between feature channels. It consists of two operations: squeeze and excitation. In the squeeze step, global average pooling is applied to each channel, producing a channel-wise descriptor:
$$z_n = \frac{1}{H \times W}\sum_{a=1}^{H}\sum_{b=1}^{W} x_n(a,b)$$
where \(x_n\) is the \(n\)-th feature map of size \(H \times W\). In the excitation step, the descriptor is passed through two fully connected layers to produce a set of channel weights:
$$s = \sigma\left(W_2 \, \text{ReLU}(W_1 z)\right)$$
where \(W_1\) and \(W_2\) are learnable weight matrices, and \(\sigma\) is the sigmoid function. The final output is obtained by scaling the original feature maps with these weights:
$$\tilde{x}_n = s_n \cdot x_n$$
I integrate the SE module into the residual blocks of ResNet50. Specifically, the SE module is placed after the residual block’s convolutional operations and before the element-wise addition, or after the addition? In my implementation, I embed the SE module in each residual block after the last convolution and before the identity mapping. This allows the network to recalibrate features at every block.
9.2 Hybrid Attention: CBAM
CBAM combines channel attention and spatial attention. Given an intermediate feature map \(F\), CBAM sequentially applies a channel attention map \(P_c\) and a spatial attention map \(P_s\). The process can be described as:
$$F_1 = P_c(F) \otimes F$$
$$F_2 = P_s(F_1) \otimes F_1$$
where \(\otimes\) denotes element-wise multiplication. The channel attention part uses both average-pooled and max-pooled features to generate channel weights. The spatial attention part uses similar pooling operations along the channel axis to produce a spatial weight map.
9.3 Comparison of Attention Mechanisms
I evaluated the improved Faster R-CNN with ResNet50 as backbone, comparing three variants: no attention, CBAM, and SENet. All models were trained under the same conditions without the sliding window for a fair comparison. Table 9.1 shows the results.
| Backbone | mAP (%) | Time (s) |
|---|---|---|
| ResNet50 | 43.67 | 17.6023 |
| CBAM-ResNet50 | 44.70 | 19.2584 |
| SE-ResNet50 | 46.01 | 18.5587 |
Both attention mechanisms improve detection accuracy. SENet yields a higher AP improvement (2.34%) than CBAM (1.17%), while also being faster than CBAM. Thus, I chose the channel attention SENet as the preferred attention module for solar panel detection.
10. Final Improved Algorithm with Attention and Sliding Window
My final detection algorithm combines all the improvements: Faster R-CNN with ResNet50 backbone, SENet channel attention modules embedded in the residual blocks, and sliding-window inference with NMS. The overall architecture is as follows:
- Training: Original images are cropped into 1000×1000 patches. Each patch is fed into the SE-ResNet50 backbone to extract feature maps.
- RPN: The feature maps are passed to the Region Proposal Network to generate candidate boxes.
- ROI Pooling & Classification: Candidate boxes are refined and classified using ROI pooling and fully connected layers.
- Inference: For a new large image, I apply a sliding window of size 1000×1000 with stride 500. All patches are independently processed by the trained network. The resulting bounding boxes are collected.
- NMS: Global non-maximum suppression is applied to remove duplicate detections across overlapping windows.
I evaluated this final method on the test set. Table 10.1 provides the comparison with the baseline Faster R-CNN(VGG16) and other variants on the whole test set.
| Method | mAP (%) | Total Time (s) |
|---|---|---|
| Faster R-CNN(VGG16) | 67.99 | 35.7335 |
| Faster R-CNN(ResNet50) | 69.63 | 22.0168 |
| Faster R-CNN(ResNet50)+Sliding Window | 86.00 | 479.3220 |
| Faster R-CNN(SE-ResNet50)+Sliding Window | 87.53 | 495.1770 |
The final algorithm achieves the highest mAP of 87.53%, which is an improvement of 1.53% over the sliding-window version without attention. This validates that the channel attention mechanism helps the network focus on solar panel features, reducing miss rates.
11. Comparison with a Machine Vision Approach
I also compared my final deep learning method with a machine vision-based solar panel recognition approach (Yao et al.). That method uses contour matching and HSV color space conversion, requiring manual threshold adjustments. I implemented it using MATLAB and evaluated it on the same three test images A, B, and C. Table 11.1, 11.2, and 11.3 present the per-image results.
| Method | Detected | Missed | Detection Rate | Time (s) |
|---|---|---|---|---|
| Yao’s method | 199 | 89 | 0.6909 | 16.0233 |
| Faster R-CNN(VGG16) | 281 | 7 | 0.9756 | 15.9770 |
| Faster R-CNN(ResNet50) | 288 | 0 | 1.0000 | 15.9265 |
| Faster R-CNN(SE-ResNet50)+Sliding Window | 288 | 0 | 1.0000 | 16.5059 |
| Method | Detected | Missed | Detection Rate | Time (s) |
|---|---|---|---|---|
| Yao’s method | 237 | 159 | 0.5984 | 16.5871 |
| Faster R-CNN(VGG16) | 385 | 11 | 0.9722 | 15.9770 |
| Faster R-CNN(ResNet50) | 394 | 2 | 0.9949 | 15.9265 |
| Faster R-CNN(SE-ResNet50)+Sliding Window | 396 | 0 | 1.0000 | 16.5059 |
| Method | Detected | Missed | Detection Rate | Time (s) |
|---|---|---|---|---|
| Yao’s method | 112 | 56 | 0.6666 | 14.9120 |
| Faster R-CNN(VGG16) | 164 | 4 | 0.9761 | 15.9770 |
| Faster R-CNN(ResNet50) | 168 | 0 | 1.0000 | 15.9265 |
| Faster R-CNN(SE-ResNet50)+Sliding Window | 168 | 0 | 1.0000 | 16.5059 |
My final method achieves 100% detection rate on all three images, while Yao’s method misses many solar panels. The deep learning approach is also more robust and requires no manual parameter tuning. Although the inference time of my method is slightly longer on some images, the accuracy advantage is decisive for practical counting tasks.
12. Counting Solar Panels in Practice
The ultimate goal of my research is to count solar panels from aerial images accurately. I used my final model to detect solar panels on the test set and counted the number of predicted bounding boxes after NMS. I set the confidence threshold to 0.75. Table 12.1 shows the results for six randomly selected test images.
| Image ID | Detected | False Positives | Actual Panels | Precision (%) |
|---|---|---|---|---|
| 1 | 223 | 3 | 220 | 98.6 |
| 2 | 61 | 9 | 52 | 85.2 |
| 3 | 17 | 0 | 17 | 100 |
| 4 | 394 | 2 | 392 | 99.5 |
| 5 | 248 | 0 | 248 | 100 |
| 6 | 268 | 0 | 268 | 100 |
For most images, the detection is nearly perfect. Some false positives appear in images with complex backgrounds, but the overall precision remains high. The model correctly counts solar panels even in images with dense arrangements. This demonstrates that my algorithm is practical for real-world photovoltaic panel counting.
13. Hyperparameter Settings
To achieve the best performance, I carefully tuned hyperparameters for the solar panel detection task. The initial learning rate was set to 0.001, momentum to 0.9, and weight decay to 0.0005. I used a step decay schedule to reduce the learning rate during training. The maximum number of bounding boxes used in training was 400. In the RPN, the number of proposed regions was set to 550. I modified the anchor scales and aspect ratios to suit solar panels, which are typically rectangular and oriented in a regular grid. The anchor ratios were set to [4, 8, 16, 32]. The input size for training was 1000×1000. All models were trained until the loss converged. I used the PyTorch framework with an NVIDIA GTX 1050 Ti GPU and an Intel i7-7820X CPU.
The loss function for Faster R-CNN combines classification loss and regression loss:
$$L(\{p_i\}, \{t_i\}) = \frac{1}{N_{cls}}\sum_i L_{cls}(p_i, p_i^*) + \lambda \frac{1}{N_{reg}}\sum_i p_i^* L_{reg}(t_i, t_i^*)$$
where \(p_i\) is the predicted probability of anchor \(i\), \(p_i^*\) is the ground-truth label (0 or 1), \(t_i\) is the predicted bounding box regression offsets, and \(t_i^*\) is the ground-truth offsets. The hyperparameter \(\lambda\) balances the two losses.
14. Discussion
My experiments demonstrate that each component contributes to the final performance. The use of a powerful backbone (ResNet50) improves feature extraction over VGG16. The sliding window technique avoids the loss of spatial details that occurs when downsampling large images. The channel attention mechanism further enhances the network’s ability to focus on relevant features. However, there are trade-offs. The sliding window increases computational cost significantly, making the method less suitable for real-time applications. For tasks that require high accuracy, such as asset counting, this is acceptable. Future work could explore more efficient inference strategies, such as tiled processing with overlapping regions and parallel GPU execution, to reduce the time overhead.
Another limitation is the availability of labeled data. My dataset contains only 150 images from a single industrial area. Though data augmentation helps, a larger and more diverse dataset could improve the generalization of the model. I plan to expand the dataset with images from different locations, altitudes, and lighting conditions.
15. Conclusion
In this paper, I presented a comprehensive study on solar panel detection using convolutional neural networks. I constructed a high-resolution aerial image dataset of industrial photovoltaic installations and manually annotated each solar panel. I compared two main detection frameworks and found that Faster R-CNN outperforms SSD for this task. I then improved the backbone network by replacing VGG16 with ResNet50, which resulted in higher detection accuracy and speed. To overcome the limitations caused by large image size and small objects, I integrated a sliding-window strategy with overlapping windows and non-maximum suppression, leading to a substantial increase in detection rate from 67.99% to 86.00% mAP. Finally, I incorporated the SENet channel attention mechanism into the residual network, further boosting mAP to 87.53%. My final method achieved perfect detection on several test images and showed superior performance compared to a classical machine vision approach. The work demonstrates the effectiveness of combining deep CNN, sliding-window inference, and attention mechanisms for solar panel detection and counting in aerial imagery. This research provides a valuable solution for automated photovoltaic system management and asset inspection.