The rapid global expansion of solar energy infrastructure has positioned photovoltaic power plants as critical assets in the renewable energy landscape. Ensuring their operational efficiency and longevity requires robust and frequent inspection regimes. Unmanned Aerial Vehicles (UAVs), or drones, have emerged as a transformative tool for this purpose, offering a cost-effective, flexible, and high-resolution platform for visual data acquisition. While significant research has advanced the automated detection and pixel-level segmentation of defects within solar panel images captured by drones, a crucial challenge remains: the precise geographical localization of identified faulty panels, particularly in complex, non-flat terrains like hillsides. Pinpointing the exact geographic coordinates of a specific malfunctioning solar panel is paramount for efficient field maintenance, directly impacting dispatch planning, repair times, and overall operational costs.
Traditional methods for UAV-based target geolocation often face limitations in this specific context. Geometric modeling approaches, which rely on a simple pinhole camera model and assume a flat ground plane, produce significant errors in sloped environments. Techniques dependent on LiDAR (Light Detection and Ranging) offer high accuracy but are prohibitively expensive and complex to integrate for routine inspections. Methods leveraging pre-existing Geographic Information System (GIS) data or Digital Elevation Models (DEMs) are not feasible in unknown or unmapped sites. Multi-view stereopsis or triangulation methods from multiple drones or positions increase logistical complexity and processing time. Therefore, there is a compelling need for a method that can rapidly and accurately determine the 3D world coordinates of a solar panel from a single UAV image, using only the standard sensor suite (camera, IMU, GPS) available on commercial drones.
This work addresses this gap by proposing a novel visual geolocation framework that synergizes modern computer vision with photogrammetric principles. The core innovation lies in leveraging a deep learning-based Monocular Depth Estimation (MDE) network to infer the missing range information from a single image. The proposed system operates in two primary stages. First, a specifically designed MDE network, which we term SwinDenseDepth, analyzes the input UAV image to predict a per-pixel depth map. Subsequently, a coordinate transformation model utilizes this predicted distance to the target solar panel, along with the camera’s intrinsic parameters and the UAV’s extrinsic pose (position and orientation), to convert the panel’s 2D pixel coordinates into precise 3D geodetic coordinates (latitude, longitude, altitude). This approach provides a lightweight, fast, and accurate solution suitable for integration into real-time or post-processing inspection pipelines.
Proposed Geolocation Methodology
The overarching architecture of the proposed solar panel geolocation system is designed for practicality and accuracy. The input is a single image from a UAV flight over a photovoltaic plant, coupled with the standard telemetry data logged during capture. The output is the geodetic coordinates of a user-specified or automatically detected solar panel within that image. The process can be decomposed into two interconnected modules: the Monocular Depth Estimation Network and the Coordinate System Transformation Model.
The workflow begins with the acquisition of a UAV image. A solar panel of interest is identified within this image, either manually or through an automated object detection algorithm, yielding its pixel coordinates \( (u, v) \). Concurrently, the following metadata is recorded from the UAV’s onboard sensors: the geographic position of the UAV \( (\phi_{UAV}, \lambda_{UAV}, h_{UAV}) \), its attitude angles \( (\phi, \theta, \psi) \) for roll, pitch, and yaw respectively, and the gimbal/camera attitude angles \( (\alpha, \beta, \gamma) \). The intrinsic camera parameters \( (f_x, f_y, u_0, v_0) \), obtained via prior calibration, are also known. The core challenge is to find the missing scalar: the radial distance \( D \) from the camera center to the specific point on the target solar panel. This is where the deep learning model intervenes.
Coordinate System Transformation Model
The transformation from a 2D pixel to a 3D world coordinate is a well-established process in photogrammetry, involving a chain of coordinate system transformations. The mathematical model forms the deterministic backbone of our geolocation system. The involved coordinate frames are:
- Image Pixel Coordinate System (\(o_p – uv\)): The 2D coordinates of the target point in the image.
- Camera Coordinate System (\(o_c – x_c y_c z_c\)): Origin at the camera’s optical center, \(z_c\)-axis along the optical axis.
- Gimbal/Platform Coordinate System (\(o_g – x_g y_g z_g\)): Attached to the camera gimbal, accounting for its pan, tilt, and roll relative to the UAV body.
- UAV Body Coordinate System (\(o_v – x_v y_v z_v\)): Fixed to the UAV’s body, typically with \(x_v\) forward, \(y_v\) right, and \(z_v\) down.
- World Geodetic Coordinate System (WGS84, \(O_E – X_E Y_E Z_E\)): The Earth-centered, Earth-fixed reference frame providing latitude (\(\phi\)), longitude (\(\lambda\)), and height (\(h\)).
The transformation is achieved through a series of matrix multiplications that project a pixel ray into the world, scaled by the estimated distance \(D\). The core equation is given by:
$$
\begin{bmatrix} X_E \\ Y_E \\ Z_E \end{bmatrix}_T = \begin{bmatrix} X_E \\ Y_E \\ Z_E \end{bmatrix}_{UAV} + D \cdot \mathbf{R}_e^d \cdot \mathbf{R}_d^n \cdot \mathbf{R}_n^p \cdot \begin{bmatrix} (u – u_0)/f_x \\ (v – v_0)/f_y \\ 1 \end{bmatrix}_{normalized}
$$
Here, \( [X_E, Y_E, Z_E]_{UAV} \) is the UAV’s position in Earth-Centered Earth-Fixed (ECEF) coordinates, derived from its GPS/INS data. The column vector on the right represents the direction of the pixel ray in the normalized camera coordinate system. The critical parameter \(D\) is the Euclidean distance from the camera center to the target point along this ray. The rotation matrices \( \mathbf{R}_n^p \), \( \mathbf{R}_d^n \), and \( \mathbf{R}_e^d \) perform the sequential rotations between the coordinate frames. Specifically:
1. Intrinsic/Projection Transformation (\(\mathbf{R}_n^p\)): This is effectively the inverse of the camera projection using the intrinsic matrix \( \mathbf{K} \). The normalized image coordinates are obtained.
2. Gimbal Attitude Transformation (\(\mathbf{R}_d^n\)): Rotates from the camera frame to the UAV body frame, based on the gimbal angles \( (\alpha, \beta, \gamma) \). It is typically composed as a sequence of elementary rotations (e.g., \( \mathbf{R}_z(\alpha) \cdot \mathbf{R}_y(\beta) \cdot \mathbf{R}_x(\gamma) \)).
3. UAV Attitude Transformation (\(\mathbf{R}_e^d\)): Rotates from the UAV body frame to the local North-East-Down (NED) or ECEF frame, based on the UAV’s roll, pitch, and yaw \( (\phi, \theta, \psi) \). This matrix aligns the body axes with the world reference frame.
The accuracy of the final geodetic coordinates \( (\phi_T, \lambda_T, h_T) \) is highly sensitive to the accuracy of the distance \(D\). An error in \(D\) propagates linearly along the sightline, leading to a significant positional error on the ground, especially from high altitudes. Therefore, the primary research focus is on obtaining the most accurate estimate of \(D\) possible from a single monocular image, which leads us to the design of a specialized depth estimation network.
SwinDenseDepth: A Monocular Depth Estimation Network for UAV Imagery
Standard MDE networks trained on datasets like KITTI (road scenes) perform poorly when applied directly to UAV imagery. The challenges are distinct: extreme variations in viewing angles (nadir to oblique), large ranges of operation (30m to 100m+), significant scale variation of objects like solar panels within a single image, and complex background terrain. To address these issues, we designed SwinDenseDepth, a transformer-based architecture optimized for depth perception in aerial scenarios.
The network follows an encoder-decoder structure. The encoder’s role is to extract powerful, hierarchical feature representations from the input UAV image. Recognizing the limitations of pure Convolutional Neural Networks (CNNs) in modeling long-range dependencies crucial for understanding large-scale aerial scenes, we employ a Swin Transformer backbone as the encoder. The Swin Transformer introduces a hierarchical design with shifted windows, allowing it to efficiently compute self-attention both locally within non-overlapping windows and globally across the image through the shifting mechanism in successive layers. This capability is vital for a solar panel inspection scene, where the model must understand the spatial layout of multiple panel arrays and their relationship to the terrain. The encoder outputs multi-scale feature maps \( \{ \mathbf{x}_0, \mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3 \} \) at different resolutions.
The decoder is tasked with progressively upsampling these features to the original image resolution while refining the depth prediction. To maximize information flow and mitigate the vanishing gradient problem, we adopt a densely connected structure within the decoder. Instead of simply summing features from the encoder, the decoder at each stage concatenates features from all previous decoder blocks at the same resolution, as well as the corresponding encoder feature. This dense fusion encourages the reuse of features and helps preserve fine-grained details necessary for predicting sharp boundaries around solar panels.
Furthermore, to allow the network to selectively focus on more informative features across both channel and spatial dimensions, we integrate a Channel-Spatial Attention Fusion (CSFusion) module at each stage of the decoder. The module works in two sequential steps:
Channel Attention: Generates a 1D channel attention vector by aggregating spatial information of a feature map via both global average pooling and global max pooling. These descriptors are then processed by a shared multi-layer perceptron to produce the channel attention weights \( \mathbf{M}_c \).
Spatial Attention: Generates a 2D spatial attention map by aggregating channel information. It applies average-pooling and max-pooling along the channel axis and concatenates the results. A convolutional layer followed by a sigmoid activation produces the spatial attention weights \( \mathbf{M}_s \).
The overall attention-refined feature \( \mathbf{F}’ \) is computed as:
$$
\mathbf{F}’ = \mathbf{M}_s(\mathbf{M}_c(\mathbf{F}) \otimes \mathbf{F}) \otimes (\mathbf{M}_c(\mathbf{F}) \otimes \mathbf{F})
$$
where \( \mathbf{F} \) is the input feature map to the CSFusion module and \( \otimes \) denotes element-wise multiplication. This dual-attention mechanism enables the network to emphasize “what” and “where” is important—for instance, focusing on the channels that represent geometric edges of solar panels and the spatial regions where the distance gradient is high.
The network is trained using a supervised loss function. Given the predicted depth map \( \hat{D} \) and the ground truth depth map \( D_{gt} \), a common and effective loss is a combination of the scale-invariant logarithmic loss and an L1 gradient loss to encourage smoothness. One typical formulation is the BerHu loss or a simple L1 loss. For the purposes of this study, we primarily utilized the Mean Squared Error (MSE) loss for its stability:
$$
\mathcal{L}_{MSE} = \frac{1}{N} \sum_{i=1}^{N} (D_{gt}^{(i)} – \hat{D}^{(i)})^2
$$
where \( N \) is the total number of valid pixels. The training objective is to minimize this loss over the constructed dataset of UAV imagery and corresponding depth maps.
Experimental Design and Implementation
To validate the proposed geolocation framework, a comprehensive experimental setup was devised, encompassing dataset creation, network training, depth estimation evaluation, and final geolocation accuracy testing.
Dataset Generation and Network Training
Acquiring real-world UAV imagery with perfectly aligned, accurate pixel-level ground truth depth is exceptionally challenging. To overcome this, we leveraged the AirSim simulation platform, built on Unreal Engine, to create a highly realistic virtual environment. A detailed 3D model of a photovoltaic power plant on sloped terrain was constructed. A simulated drone was then programmed to fly autonomous missions at varying altitudes (30m to 60m) and with diverse camera angles (nadir and oblique), mimicking a real inspection routine. For each captured synthetic RGB image, AirSim provides a perfectly registered depth map based on the simulation’s ground truth geometry. This process yielded a high-quality dataset of 2,280 image-depth pairs. The dataset was split into training (70%), validation (20%), and test (10%) sets.
| Parameter | Specification |
|---|---|
| Simulation Platform | AirSim / Unreal Engine |
| Scene | Photovoltaic Plant on Sloped Terrain |
| UAV Flight Altitude | 30m, 40m, 50m, 60m |
| Camera Angles | Nadir (0°), Oblique (15°-45°) |
| Total Image Pairs | 2,280 |
| Train / Val / Test Split | 1,596 / 456 / 228 |
| Image Resolution | 768 x 768 pixels |
| Depth Map Format | Linear depth in meters |
The SwinDenseDepth network was implemented using PyTorch. The Swin-T variant was used as the encoder backbone, pre-trained on ImageNet. The decoder was built from scratch with the proposed dense connections and CSFusion modules. During training, input images were resized to 768×768. Data augmentation techniques including random horizontal flipping and color jittering were applied to improve generalization. The model was optimized using the Adam optimizer with an initial learning rate of 1e-4, decayed via a cosine annealing schedule over 35 epochs.
Evaluation Metrics
To rigorously evaluate the depth estimation performance, standard metrics from the MDE literature were employed, with slight adjustments for the UAV context. Let \( \hat{d}_i \) be the predicted depth and \( d_i^* \) be the ground truth depth for pixel \( i \), and \( N \) be the total number of pixels evaluated.
| Metric | Formula | Description |
|---|---|---|
| Absolute Relative Error (AbsRel) | $$ \frac{1}{N} \sum_{i=1}^{N} \frac{|d_i^* – \hat{d}_i|}{d_i^*} $$ | Average ratio of absolute error to true depth. |
| Squared Relative Error (SqRel) | $$ \frac{1}{N} \sum_{i=1}^{N} \frac{(d_i^* – \hat{d}_i)^2}{d_i^*} $$ | Emphasizes larger errors more heavily. |
| Root Mean Squared Error (RMSE) | $$ \sqrt{ \frac{1}{N} \sum_{i=1}^{N} (d_i^* – \hat{d}_i)^2 } $$ | Standard deviation of the prediction errors. |
| Accuracy under threshold (\(\delta_k\)) | $$ \% \text{ of } \hat{d}_i \text{ s.t. } \max(\frac{\hat{d}_i}{d_i^*}, \frac{d_i^*}{\hat{d}_i}) < \delta_k $$ | Percentage of predictions within a strict tolerance (\(\delta_1=1.05, \delta_2=1.05^2, \delta_3=1.05^3\)). |
For the final geolocation accuracy, the Haversine formula was used to compute the great-circle distance between the predicted geographic coordinate \( (\phi_p, \lambda_p) \) and the ground truth coordinate \( (\phi_{gt}, \lambda_{gt}) \), incorporating altitude difference:
$$
\text{Error}_{2D} = 2r \cdot \arcsin\left( \sqrt{ \sin^2\left(\frac{\phi_{gt} – \phi_p}{2}\right) + \cos(\phi_{gt})\cos(\phi_p) \sin^2\left(\frac{\lambda_{gt} – \lambda_p}{2}\right) } \right)
$$
$$
\text{Error}_{3D} = \sqrt{ (\text{Error}_{2D})^2 + (h_{gt} – h_p)^2 }
$$
where \( r \) is the Earth’s radius (approximately 6,378,137 m).
Results and Analysis
Ablation Study on Depth Estimation Network
To validate the effectiveness of each component in SwinDenseDepth, an ablation study was conducted on the test set. A baseline model was established using a standard Vision Transformer (ViT) encoder with a simple summation-based decoder (akin to DPT). Components were incrementally added: 1) Replacing ViT with Swin Transformer (Swin), 2) Incorporating the Dense connection structure in the decoder (Dense), and 3) Adding the Channel-Spatial Attention Fusion modules (CSFusion).
| Model Configuration | \(\delta_1 \uparrow\) | \(\delta_2 \uparrow\) | \(\delta_3 \uparrow\) | AbsRel \(\downarrow\) | SqRel \(\downarrow\) | RMSE \(\downarrow\) |
|---|---|---|---|---|---|---|
| Baseline (ViT + Simple Decoder) | 0.648 | 0.781 | 0.813 | 0.209 | 0.007 | 0.011 |
| + Swin Transformer Encoder | 0.677 | 0.784 | 0.829 | 0.194 | 0.004 | 0.009 |
| + Dense Connections | 0.682 | 0.789 | 0.821 | 0.180 | 0.003 | 0.009 |
| + CSFusion Modules (SwinDenseDepth) | 0.729 | 0.809 | 0.834 | 0.110 | 0.001 | 0.008 |
The results demonstrate clear and cumulative improvements. The Swin Transformer encoder provides a significant boost in accuracy (\(\delta_1\) from 0.648 to 0.677), highlighting its superior ability to model the aerial scene’s geometry compared to the standard ViT. The dense connections further refine the features, reducing the AbsRel error. The final addition of the CSFusion modules yields the most substantial gain, improving \(\delta_1\) by 4.7 percentage points and halving the AbsRel error compared to the previous configuration. This confirms that the attention mechanism successfully guides the network to focus on semantically meaningful regions, such as the edges and surfaces of solar panels, leading to sharper and more accurate depth discontinuities.
Comparative Analysis with State-of-the-Art MDE Methods
We compared SwinDenseDepth against several prominent MDE networks, including both self-supervised and supervised methods, after training them on our custom UAV dataset. The selected benchmarks were Monodepth2 (self-supervised), Lite-Mono (a lightweight self-supervised CNN-Transformer hybrid), AdaBins (supervised adaptive binning), and GLPDepth (a recent high-performance supervised method).
| Method | Supervision | \(\delta_1 \uparrow\) | \(\delta_2 \uparrow\) | \(\delta_3 \uparrow\) | AbsRel \(\downarrow\) | RMSE \(\downarrow\) |
|---|---|---|---|---|---|---|
| Monodepth2 | Self | 0.293 | 0.577 | 0.819 | 0.692 | 0.138 |
| Lite-Mono | Self | 0.277 | 0.539 | 0.770 | 0.647 | 0.125 |
| AdaBins | Supervised | 0.303 | 0.521 | 0.683 | 0.501 | 0.042 |
| GLPDepth | Supervised | 0.701 | 0.801 | 0.814 | 0.189 | 0.009 |
| SwinDenseDepth (Ours) | Supervised | 0.729 | 0.809 | 0.834 | 0.110 | 0.008 |
The table reveals several key insights. First, self-supervised methods (Monodepth2, Lite-Mono) struggle significantly in the UAV domain, as the photometric consistency assumption often breaks down with large viewpoint changes and complex textures, resulting in high error metrics. Supervised methods perform markedly better. While AdaBins shows improvement, it is outperformed by both GLPDepth and our method. SwinDenseDepth achieves the best performance across nearly all metrics, notably surpassing GLPDepth by 2.8% in \(\delta_1\) and reducing AbsRel by 42%. This demonstrates the efficacy of our architectural choices—the Swin Transformer’s hierarchical global modeling combined with dense, attention-guided feature fusion—is particularly well-suited for the challenges inherent in UAV-based solar panel inspection imagery.
Geolocation Accuracy in Simulation
To evaluate the end-to-end geolocation system, multiple known ground control points were placed within the simulated photovoltaic plant. The drone was flown over these points at various altitudes and orientations. For each captured image, the pixel coordinate of a point was manually selected, its depth was estimated by SwinDenseDepth, and the full coordinate transformation was applied to compute its geodetic coordinates. These were compared against the known simulation ground truth. As a baseline, the traditional geometric modeling method, which assumes a flat terrain at the drone’s altitude, was also applied.
| Test Point | Drone Altitude (m) | View Angle | Geometric Modeling Error (m) | Proposed Method Error (m) |
|---|---|---|---|---|
| 1 | 20.2 | Oblique | 2.41 | 1.05 |
| 2 | 32.7 | Oblique | 5.61 | 1.12 |
| 3 | 57.6 | Oblique | 7.87 | 1.31 |
The results are conclusive. The flat-earth assumption of the geometric model leads to substantial errors that grow with terrain slope and drone altitude, rendering it unsuitable for hillside solar panel farms. In contrast, our method, which implicitly learns and accounts for the terrain geometry through depth estimation, maintains sub-1.5 meter accuracy across different flight conditions. This level of precision is sufficient to uniquely identify a specific solar panel within an array.
Validation on Real-World UAV Inspection Data
To assess practical applicability, the system was tested on real UAV imagery collected from an operational photovoltaic plant on sloped terrain. Several solar panels were physically surveyed to obtain their accurate GPS coordinates. A drone was flown at approximately 35m, 40m, and 60m altitudes, capturing images from various oblique angles. Since ground truth depth maps for real images are unavailable, the depth prediction accuracy was indirectly validated by comparing the final geolocation error. The coordinate transformation used real sensor data (GPS, IMU) from the drone’s flight log. For comparison, geolocation was also performed using: 1) the baseline geometric method, and 2) the coordinate transformation using depth maps from the baseline DPT network instead of SwinDenseDepth.
| Flight Altitude (m) | Avg. Geo. Modeling Error (m) | Avg. Error with DPT Depth (m) | Avg. Error with SwinDenseDepth (Ours) (m) |
|---|---|---|---|
| 35 | 13.08 | 1.07 | 1.03 |
| 40 | 11.14 | 3.35 | 1.18 |
| 60 | 15.02 | 4.05 | 1.48 |
The real-world tests corroborate the findings from simulation. The geometric modeling error is unacceptably high (over 11 meters), confirming its failure in non-flat terrain. Using a depth map from a network (DPT) not optimized for UAV scenes already provides a massive improvement, bringing errors down to the 1-4 meter range. However, our optimized SwinDenseDepth network consistently delivers the best performance, achieving an average geolocation error between 1.0 and 1.5 meters across the tested altitude range. This meets the practical requirement of identifying the specific solar panel for maintenance. The slight increase in error with altitude is expected due to the inherent difficulty of estimating depth for distant, smaller objects and potential slight inaccuracies in the UAV’s sensor data (GPS drift, IMU noise).
Conclusion and Future Work
This work presented a novel and effective framework for the geographical localization of solar panels from monocular UAV imagery, specifically targeting the challenging case of sloped terrain. The core contribution is the integration of a specialized monocular depth estimation network, SwinDenseDepth, into a standard photogrammetric coordinate transformation pipeline. By accurately predicting the distance to the target, the system overcomes the fundamental limitation of traditional single-image geolocation methods. The SwinDenseDepth network itself incorporates several key innovations—a Swin Transformer encoder for global scene understanding, a densely connected decoder for feature reuse, and a channel-spatial attention fusion module for contextual refinement—which collectively enable it to produce high-quality depth estimates from the unique perspective of UAV-based solar panel inspection images.
Comprehensive experiments conducted in both a controlled simulation environment and on real-world UAV data demonstrate the system’s robustness and accuracy. The proposed method consistently achieved geolocation errors in the range of 1 to 2 meters for flight altitudes between 30m and 60m, a significant improvement over flat-earth geometric modeling and a clear advancement over using off-the-shelf MDE networks. This level of accuracy is operationally valuable, enabling maintenance crews to be dispatched directly to the correct solar panel or row, thereby reducing search time and increasing the efficiency of photovoltaic plant operations and maintenance.
Future research directions are multifaceted. First, exploring self-supervised or semi-supervised learning paradigms for the MDE network could reduce dependency on simulated or expensively acquired ground truth depth data. Second, the current system localizes a point on a panel. Extending it to predict the bounding polygon or footprint of an entire solar panel in geographic coordinates would provide even more utility for mapping and inventory purposes. Third, investigating temporal fusion techniques to integrate depth and location estimates from multiple frames in a video sequence could further enhance robustness and accuracy. Finally, optimizing the network for lower computational complexity and power consumption would facilitate its deployment directly on the drone’s onboard computer for real-time, edge-based geolocation during flight.