In the face of escalating global environmental challenges, the transition towards a clean, low-carbon, safe, and efficient energy system has become a paramount goal for the energy sector worldwide. For too long, electricity generation has relied heavily on traditional fossil fuels. To mitigate environmental degradation and slow global warming, ambitious targets have been set, such as achieving carbon peak before 2030 and carbon neutrality before 2060. The widespread adoption of new energy sources is a critical pathway to these “dual-carbon” goals. Among renewable technologies, photovoltaic (PV) power generation, with its sustainable, non-polluting, and energy-saving advantages, has emerged as one of the most rapidly developing sectors. Statistics indicate a massive and continuously growing installed capacity of PV systems globally. However, this proliferation brings significant operational challenges. The maintenance, inspection, and inventory management of vast numbers of solar panels through traditional manual field surveys are prohibitively time-consuming, labor-intensive, and costly. Efficiently geolocating, sizing, and assessing the condition of solar panels has thus become a pressing issue.
Remote sensing technology offers a powerful, large-scale, and non-contact solution. However, the precise extraction of solar panels from complex high-resolution imagery remains a formidable task. Traditional image processing methods, which often rely on low-level features like grayscale or texture for region- or edge-based segmentation, struggle with the intricate backgrounds and fine details present in such imagery, leading to poor edge delineation and low extraction accuracy.
The advent of deep learning, particularly semantic segmentation models, has revolutionized this field. By learning high-level semantic features end-to-end, these models can perform pixel-wise classification directly from raw images, enabling automated and high-precision target extraction. Pioneering architectures like Fully Convolutional Networks (FCN), U-Net, PSPNet, and DeepLabV3+ have laid a strong foundation. U-Net’s encoder-decoder structure with skip connections helps preserve spatial details. PSPNet employs a pyramid pooling module to aggregate multi-scale context. DeepLabV3+ leverages atrous (dilated) convolutions within an Atrous Spatial Pyramid Pooling (ASPP) module to capture multi-scale contextual information while maintaining feature map resolution, followed by a decoder to refine object boundaries.
Despite these advances, directly applying these models to extract solar panels from very high-resolution (VHR) remote sensing images presents specific challenges. The ASPP module using standard atrous convolutions with large dilation rates can suffer from the “gridding issue,” where the sampled pixels are too sparse, potentially missing fine local details crucial for separating closely spaced solar panels or capturing their precise edges. Furthermore, during the decoding phase, standard upsampling may not adequately focus on the most informative features for the target solar panels, especially against cluttered backgrounds.
To address these limitations, we propose a novel semantic segmentation architecture, DeepLab-HDCA (DeepLab with Hybrid Dilation Convolution and Attention), based on an enhanced DeepLabV3+ framework. Our primary contributions are twofold, each targeting a key challenge in solar panel extraction:
- To overcome the information loss associated with standard atrous convolutions in multi-scale feature extraction, we introduce a Hybrid Dilated Convolution based Atrous Spatial Pyramid Pooling (HASPP) module.
- To enhance the model’s focus on discriminative features and improve boundary recovery during upsampling, we integrate a Convolutional Block Attention Module (CBAM) into the decoder.
Methodology: The DeepLab-HDCA Framework
The overall pipeline of our proposed DeepLab-HDCA follows an encoder-decoder structure. The encoder, typically a backbone network like ResNet, extracts hierarchical features. The core of our innovation lies in the modified ASPP and the attentive decoder.
1. Hybrid Dilated Convolution for ASPP (HASPP)
Standard dilated convolutions insert “holes” (zeros) between kernel elements to exponentially expand the receptive field without increasing parameters or losing resolution. A 3×3 kernel with a dilation rate \( r \) has an effective receptive field size \( R \) given by:
$$ R = 2 \cdot (r \cdot (k-1) + 1) – 1 $$
where \( k \) is the kernel size (e.g., 3). For \( r=2 \), \( R=7 \); for \( r=3 \), \( R=11 \). However, with large dilation rates, the sampled pixels become non-contiguous, creating a checkerboard pattern. When multiple such layers are stacked, large continuous areas within the theoretical receptive field may never be involved in computation, leading to loss of local continuity and fine-grained details. This is detrimental for segmenting small or densely packed objects like solar panels.
Hybrid Dilated Convolution (HDC) is designed to alleviate this gridding effect. Instead of using a single large dilation rate, we use a cascade of smaller rates designed such that the final combined receptive field is fully covered without gaps. In our HASPP module, we replace the standard parallel atrous convolutions (e.g., rates 6, 12, 18) with parallel branches, each containing a cascade of 3×3 convolutions with carefully chosen dilation rates. For instance, one branch may use rates [1, 3], another [1, 3, 5], and a third [1, 3, 5, 7]. The design ensures that for an N-layer cascade with dilation rates \( r_1, r_2, …, r_N \), the maximum distance between any two adjacent sampled pixels in the final effective kernel is 1. This rule can be expressed as:
$$ M_i = max[r_{i+1} – r_i, r_{i+1}] $$
for the i-th layer, and the condition \( M_i \leq K \) (kernel size) should hold to avoid gridding. This design allows the module to capture multi-scale context from effectively larger receptive fields while maintaining dense pixel sampling, thereby preserving the local structural information essential for accurate solar panel delineation.
The output feature maps from different HDC branches and a global average pooling branch are concatenated and processed by a 1×1 convolution to generate rich, multi-scale, and locally dense contextual features.
2. Attention-Guided Decoder with CBAM
The decoder in DeepLabV3+ combines low-level features from the encoder with the high-level contextual features from the ASPP output to recover spatial details. To make this fusion more effective, we incorporate the Convolutional Block Attention Module (CBAM) before the final concatenation and upsampling steps. CBAM sequentially infers attention maps along both the channel and spatial dimensions, which are then multiplied with the input feature map for adaptive feature refinement.
Given an intermediate feature map \( \mathbf{F} \in \mathbb{R}^{C \times H \times W} \), the CBAM process is:
- Channel Attention Module (CAM): It focuses on “what” is meaningful. The module aggregates spatial information using both average-pooling and max-pooling to produce two different spatial context descriptors. These are forwarded to a shared multi-layer perceptron (MLP) to generate the channel attention map \( \mathbf{M}_c \in \mathbb{R}^{C \times 1 \times 1} \).
$$ \mathbf{M}_c(\mathbf{F}) = \sigma \left( \text{MLP}(\text{AvgPool}(\mathbf{F})) + \text{MLP}(\text{MaxPool}(\mathbf{F})) \right) $$
$$ \mathbf{F}’ = \mathbf{M}_c(\mathbf{F}) \otimes \mathbf{F} $$
where \( \sigma \) is the sigmoid activation and \( \otimes \) denotes element-wise multiplication. - Spatial Attention Module (SAM): It focuses on “where” the informative parts are. The channel-refined feature \( \mathbf{F}’ \) is processed by applying average-pooling and max-pooling along the channel axis and concatenating the results. A convolutional layer then produces the spatial attention map \( \mathbf{M}_s \in \mathbb{R}^{1 \times H \times W} \).
$$ \mathbf{M}_s(\mathbf{F}’) = \sigma \left( f^{7 \times 7}( [\text{AvgPool}(\mathbf{F}’); \text{MaxPool}(\mathbf{F}’)] ) \right) $$
$$ \mathbf{F}” = \mathbf{M}_s(\mathbf{F}’) \otimes \mathbf{F}’ $$
where \( f^{7 \times 7} \) denotes a convolution with a 7×7 filter.
The final output \( \mathbf{F}” \) is the attentively refined feature. By integrating CBAM into the decoder, the model learns to suppress irrelevant background noise and highlight features corresponding to solar panels and their edges, leading to sharper and more accurate segmentation masks after upsampling.
Experimental Setup and Datasets
We evaluated our proposed DeepLab-HDCA model on a public multi-resolution PV dataset. This dataset contains VHR imagery with three spatial resolutions: 0.1 m (from UAVs, mainly rooftop solar panels), 0.3 m (from aerial imagery, mixed rooftop and ground-mounted), and 0.8 m (from satellite imagery, large-scale ground-mounted plants). The backgrounds are highly diverse, including various roof types (brick, concrete, steel), vegetation (grass, shrubs, farmland), bare soil, and water bodies, presenting a comprehensive testbed for model robustness.
The dataset was split into training and validation sets. We employed standard data augmentation techniques (rotation, flipping) and trained our model using common deep learning frameworks. The performance was measured using standard semantic segmentation metrics: Intersection over Union (IoU), Precision, and Recall, calculated from the confusion matrix of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN).
$$ \text{IoU} = \frac{TP}{TP + FP + FN}, \quad \text{Precision} = \frac{TP}{TP + FP}, \quad \text{Recall} = \frac{TP}{TP + FN} $$
Results and Analysis
1. Performance Across Different Resolutions
Our first experiment compared the proposed DeepLab-HDCA (HASPP+CBAM) against the baseline DeepLabV3+ (with standard ASPP) across the three resolution levels. The results clearly demonstrate the effectiveness of our modifications.
| Dataset (Resolution) | Model Variant | IoU (%) | Precision (%) | Recall (%) |
|---|---|---|---|---|
| PV01 (0.1m) | Baseline (ASPP) | 91.90 | 95.75 | 95.81 |
| Ours (HASPP+CBAM) | 92.54 | 96.43 | 95.82 | |
| PV03 (0.3m) | Baseline (ASPP) | 77.89 | 87.38 | 87.76 |
| Ours (HASPP+CBAM) | 79.91 | 87.03 | 90.75 | |
| PV08 (0.8m) | Baseline (ASPP) | 74.85 | 85.88 | 85.78 |
| Ours (HASPP+CBAM) | 76.27 | 85.65 | 87.44 |
The proposed model consistently achieved higher IoU across all resolutions, with notable gains of approximately 2 percentage points on the 0.3m and 0.8m data. More importantly, the Recall saw significant improvements, especially at lower resolutions (0.3m: +2.99%, 0.8m: +1.66%), indicating that our model is better at correctly identifying solar panel pixels and reducing false negatives. This is crucial for complete inventory surveys. Visually, our model successfully separates closely spaced solar panels that the baseline model tends to merge, thanks to the HASPP’s ability to capture fine local distinctions and the CBAM’s focus on relevant features.
2. Performance in Different Background Contexts
We further analyzed model performance based on background type (rooftop vs. ground-mounted) within specific resolution sets. This is critical because distributed rooftop solar panels are often smaller, more scattered, and surrounded by complex urban textures, posing a greater challenge than larger, more homogeneous ground-mounted plants.
| Background Context (at 0.3m) | Model Variant | IoU (%) |
|---|---|---|
| Rooftop | Baseline (ASPP) | 77.89 |
| Rooftop | Ours (HASPP+CBAM) | 79.91 |
| Farmland | Baseline (ASPP) | 94.14 |
| Farmland | Ours (HASPP+CBAM) | 94.27 |
| Grass/Shrub | Baseline (ASPP) | ~91.23* |
| Grass/Shrub | Ours (HASPP+CBAM) | ~91.98* |
The improvement is most pronounced for rooftop solar panels, where IoU increased by over 2 percentage points. For ground-mounted solar panels on relatively uniform backgrounds like farmland, the improvement, while positive, is more modest. This confirms that our enhancements are particularly effective for the more challenging task of extracting distributed, small-scale rooftop PV installations, where edge details and separation from background clutter are paramount.
3. Ablation Study
To validate the contribution of each component, we conducted an ablation study on the 0.3m resolution dataset, measuring IoU for different model variants.
| Model Configuration | IoU (%) | Description |
|---|---|---|
| Baseline (ASPP only) | 77.89 | Original DeepLabV3+ |
| ASPP + CBAM | 78.31 | Adds only attention to decoder |
| HASPP only | 79.08 | Replaces ASPP with HASPP |
| HASPP + CBAM (Full Model) | 79.91 | Our proposed DeepLab-HDCA |
The results show that both the HASPP module and the CBAM module individually contribute to performance gains. The HASPP module provides a larger boost, underscoring the importance of resolving the gridding issue for multi-scale feature extraction of solar panels. The CBAM module further refines the result. Their combination yields the best performance, demonstrating a synergistic effect where enhanced multi-scale features are then attentively processed to focus on the most relevant details for segmenting solar panels.
4. Comparative Analysis with Other Models
We compared our full DeepLab-HDCA model against other classic semantic segmentation architectures: U-Net, PSPNet, and the baseline DeepLabV3+. The test focused on the challenging rooftop subsets across all three resolutions.
| Model | 0.1m Rooftop IoU (%) | 0.3m Rooftop IoU (%) | 0.8m Rooftop IoU (%) |
|---|---|---|---|
| U-Net | 88.64 | 77.07 | 55.76 |
| PSPNet | 79.97 | 69.50 | 53.53 |
| DeepLabV3+ (Baseline) | 91.90 | 77.89 | 74.85 |
| DeepLab-HDCA (Ours) | 92.54 | 79.91 | 76.27 |
Our model consistently outperforms all others. U-Net, while effective at preserving edges, may lose broader contextual information. PSPNet’s fixed grid pooling can struggle with the irregular shapes and layouts of rooftop solar panel arrays. The baseline DeepLabV3+ performs well but is surpassed by our enhancements. The superior performance of DeepLab-HDCA, especially at lower resolutions (0.3m and 0.8m), validates the effectiveness of combining hybrid dilated convolutions for dense multi-scale feature extraction with an attention mechanism for feature refinement in the specific domain of solar panel extraction.
Conclusion
In this work, we have presented DeepLab-HDCA, an advanced semantic segmentation framework tailored for the precise extraction of solar panels from very high-resolution remote sensing imagery. The core of our method addresses two key technical hurdles: the loss of local detail in multi-scale context modules and the lack of focus on discriminative features during boundary recovery. The Hybrid Dilated Convolution based ASPP (HASPP) module effectively mitigates the gridding artifact of standard atrous convolutions, enabling the capture of both broad context and fine-grained local patterns essential for separating individual solar panels. The integration of the CBAM attention mechanism into the decoder guides the model to emphasize features relevant to solar panels and their edges, suppressing background interference and leading to sharper segmentation masks.
Extensive experiments on a multi-resolution PV dataset demonstrate that our proposed model achieves state-of-the-art performance, particularly in the challenging scenario of extracting distributed rooftop solar panels against complex backgrounds. It shows consistent improvements over the baseline DeepLabV3+ and other classic models like U-Net and PSPNet across different spatial resolutions. This research provides a robust, automated, and cost-effective solution for large-scale PV infrastructure inventory and monitoring, offering significant data support for the operation and maintenance of the rapidly growing global solar energy sector.