With the rapid expansion of the photovoltaic industry, the installation and maintenance of solar panels have become critical tasks. Traditional methods for inspecting solar panels rely heavily on manual counting of panels from aerial photographs captured by drones. This approach is not only inefficient but also prone to missed detections or misclassifications. To address these challenges, I have developed a comprehensive monitoring system that integrates unmanned aerial vehicle (UAV) technology with advanced image recognition models. By leveraging the flexibility of drones and the power of deep learning, my system significantly reduces human intervention and provides a highly efficient and accurate solution for solar panel detection and management. The core of this work lies in the application of computer vision techniques, which enable automatic identification and localization of solar panels across vast photovoltaic arrays.
This paper presents the design and implementation of a UAV-based solar panel monitoring system. The system comprises a hardware platform built around an industry-leading DJI drone and a software system following a front-end/back-end separation architecture. The front-end is a native Android application that provides intuitive controls and real-time video feeds from the drone. The back-end, constructed using the Django framework, handles image processing, data storage, and API services. A key innovation is the solar panel recognition method based on an improved YOLOv8 model that integrates spatial attention and channel attention mechanisms, enhancing detection accuracy in complex environments. The system supports automatic flight control, real-time image capture, intelligent recognition, and comprehensive data management, including historical record retrieval and multi-format export.
The remainder of this article is organized as follows: Section I discusses the background and motivation for solar panel detection. Section II presents the overall system architecture. Section III details the functional design of both front-end and back-end components. Section IV focuses on the solar panel recognition algorithm, including the attention mechanism and training process. Section V concludes the work with a summary of achievements and future directions.
I. Research Background and Motivation
Image recognition technology, also known as computer vision, utilizes computers and artificial intelligence to analyze and interpret images, automatically identifying and classifying objects, scenes, and patterns. In the context of solar panel inspection, traditional field surveys require human workers to manually count every solar panel in each row using photographs or measurement tools. This method depends heavily on the worker’s carefulness and accuracy, and is subject to many human factors. With the maturation of deep learning and convolutional neural networks (CNNs), end-to-end image recognition tasks can now be performed with high reliability. Compared to manual methods, combining UAVs with image recognition offers superior speed and coverage over large photovoltaic arrays. Drone-based inspection reduces labor costs and safety risks while collecting vast amounts of data that support long-term management and optimization of solar power stations.
The primary goal of my project is to automate solar panel detection and counting using a UAV-mounted camera system. The system must handle various lighting conditions, angles, and partial occlusions typical in real-world solar farms. By replacing manual counting with an automated visual recognition pipeline, I aim to achieve higher accuracy, faster throughput, and lower operational costs.
II. Overall System Design
My solar panel monitoring system adopts a modular approach, integrating state-of-the-art hardware and software to achieve intelligent monitoring and management of solar panels in an efficient and safe manner. The hardware system consists of a DJI drone and its remote controller, which provide excellent flight stability and precise remote control for aerial photography of solar panel arrays. Images are transmitted securely via wireless network to the back-end for recognition analysis. The software system employs a “front-end/back-end separation” design pattern to enhance maintainability and scalability. The front-end is a native application offering an intuitive user interface and real-time monitoring views, enabling users to easily track the recognition progress and status of solar panels. The back-end is built on a monolithic architecture, ensuring centralized management and fast response times. All data is encrypted during transmission to prevent information leakage.
The following table summarizes the key components of the system:
| Component | Technology | Function |
|---|---|---|
| UAV Platform | DJI drone & controller | Aerial image acquisition with high stability and precise control |
| Front-end | Android SDK + DJI SDK | Native app for user interaction, real-time video display, and control |
| Back-end | Django framework | API management, image processing, data persistence |
| Database | MySQL | Structured storage for recognition results and metadata |
| Proxy Server | Nginx | Load balancing and reverse proxy for high throughput |
| Recognition Model | YOLOv8 + Attention | Solar panel detection and annotation |
| Communication | HTTP/REST API | Stateless data exchange between front-end and back-end |
III. Functional Development and Design
3.1 Front-End Design
The front-end is a native application built using Android SDK and DJI SDK, combining platform-specific features with drone technology to enhance user experience. Native Android development offers excellent dynamic interaction capabilities. For the UAV solar panel recognition system, the dynamic view construction mechanism in the native environment provides unparalleled advantages. The system can capture and analyze data in real time, instantly displaying the recognition progress and results on the interface. This gives users timely decision support, greatly improving work efficiency and operational intuitiveness.
Key features of the front-end include:
- Split navigation menu: left side shows drone status (battery, signal strength, GPS, flight restrictions), right side has a button for history records.
- Full-screen video display with an overlay of a capture button positioned vertically in the middle-right area.
- After capturing an image, a popup appears with options to upload or edit the image. Auto-upload with a countdown timer is provided for convenience.
- Image editing interface allows manual cropping before upload.
- Upon receiving recognition results from the back-end, a small info window appears at the top-right corner, which can be clicked to view full details or dismissed automatically.
The following table summarizes the main front-end functions:
| Module | Function | User Interaction |
|---|---|---|
| Flight Control | Initialize connection, set remote controller mode, display live video | Automatic upon app start; status shown on left panel |
| Capture & Recognize | Take photo, edit/crop, upload with metadata (time, GPS, altitude) | Capture button triggers popup; auto-upload or manual upload |
| History Management | Display list of past recognition records, filter by time/region/date range, export results | Slide-out menu on right; click item to see details; delete supported (logical deletion) |
3.2 Back-End Design
The back-end service is a monolithic application built with the Django framework. It manages data and provides services to other applications via simple, stateless RESTful APIs following the HTTP protocol. This separation facilitates maintainability and third-party integration. The APIs are defined using the OpenAPI specification, promoting discoverability, reusability, and automation. Data persistence is handled using ORM (Object-Relational Mapping) technology, which automatically escapes malicious inputs to enhance system security.
Back-end modules include:
- Image Recognition & Annotation: After receiving image data, the back-end validates its integrity and security, then passes it to the trained solar panel recognition model. The model identifies and annotates solar panels, returning results (annotated image and panel count) to be saved in the database and sent back to the client.
- Recognition Result Management: Provides flexible query interfaces allowing users to search records based on various conditions (time, region, date range, location, count). Logical deletion is implemented to prevent accidental data loss, with periodic physical cleanup to save resources.
- Data Statistics: Supports multi-dimensional export in Excel or image archive formats. Users can add watermarks (e.g., timestamp, GPS, panel bounding boxes, count) to exported images. Summary statistics including total panel counts and aggregated location information are provided.
The following table outlines the main back-end API endpoints:
| Endpoint | Method | Description |
|---|---|---|
| /api/recognize | POST | Upload image with metadata; return recognition result |
| /api/results | GET | List records with filters (date, region, count range); returns summary data |
| /api/results/{id} | GET | Get full detail of a specific record |
| /api/results/{id} | DELETE | Soft-delete a record |
| /api/export | POST | Generate export file (Excel or ZIP) with selected filters and watermarks |
3.3 Database Design
MySQL is chosen as the core database due to its excellent performance, high security, and broad platform compatibility. In the UAV solar panel recognition scenario, MySQL ensures instant retrieval and insertion of recognition data, accelerating the overall system response. Users can quickly view recognition results and export data, significantly improving work efficiency. Importantly, MySQL’s built-in security mechanisms effectively mitigate potential data leakage risks, strengthening the overall security defense.
The database schema includes the following main tables (simplified):
| Table Name | Key Columns | Description |
|---|---|---|
| recognition_record | id, image_path, timestamp, latitude, longitude, altitude, panel_count, annotated_image_path, is_deleted, created_at | Stores each recognition result with metadata |
| export_log | id, user_id, filters, format, created_at | Logs export requests for auditing |
| model_version | id, version_name, accuracy, mAP, released_at | Tracks model versions for reproducibility |
3.4 Architecture Design
The system uses a front-end/back-end separation architecture, where each part focuses on its own functionality, greatly increasing system fluency. The client responds quickly to user interactions, improving user experience. For this system, the separation allows flexible future iterations; improvements to the back-end recognition algorithm do not require client re-deployment, ensuring users always have a high-quality experience.
The overall workflow is as follows: The drone captures images and streams them to the front-end app. The front-end allows the user to capture a specific frame, optionally edit it, and upload it via HTTP to the back-end. The back-end processes the image through the solar panel recognition model, stores the result in MySQL, and returns the annotation information to the front-end. Nginx acts as a reverse proxy, balancing load and improving throughput.
3.5 Service Proxy
The system integrates Nginx as a mature and robust service proxy platform, leveraging its core features of load balancing and reverse proxy to provide a highly optimized support architecture for efficient image data processing and user interaction. With Nginx’s load balancing capability, the system can handle large-scale data streams generated during image recognition, ensuring smooth data processing speed and efficient real-time monitoring feedback even during intensive analysis periods. The immediacy of information delivery is significantly improved, enhancing operational transparency and response speed.
3.6 Client Program
The client main interface includes a split navigation menu, a full-screen drone video, and a capture button. The split navigation menu has a left side showing status (drone/controller battery, signal strength, GPS, flight restrictions) and a right side with a button for history records. The capture button is placed in the middle-right part of the full-screen video. The main functions of the client are flight control, capture and recognition, and history record management.
Flight Control: Upon opening the client, it automatically initializes, loads connected devices, and sets the remote controller operation mode. The main interface displays connection information and real-time video from the drone camera.
Capture and Recognition: Clicking the capture button takes a snapshot of the drone video. A popup shows the captured image, with options to upload or edit. An auto-upload countdown timer provides convenient unattended operation. When editing, the system stops auto-upload and opens an image editing interface where users can manually crop the image to improve accuracy. The upload payload includes the image, timestamp, GPS coordinates, address, and altitude. After back-end processing, the recognition result is returned and displayed as a small window at the top-right of the interface. Users can click to view full details or delete the record. The small window auto-closes after a short delay, or users can close it manually.
History Record Management: Clicking the history button on the navigation bar opens a drawer on the right side, showing a list of previous recognition records. Clicking an item displays all related information. The system supports logical deletion. Multi-dimensional filtering and flexible export options are available: users can filter by time, region, date range, and export results as an Excel file or image archive. Watermarks such as capture time, date, GPS location, solar panel bounding boxes, and panel count can be added to exported images.
3.7 Server Program
The server program consists of three main modules: Image Recognition & Annotation, Recognition Result Management, and Data Statistics.
Image Recognition & Annotation: After receiving image data from the client, the server validates its integrity, completeness, and security. Once verified, the data is fed into the trained solar panel recognition and annotation model. The model identifies and annotates solar panels in the image, producing a result image and a panel count. These results, along with the metadata (time, GPS, altitude) provided by the client, are saved to the database and returned to the client.
Recognition Result Management: The server provides flexible query interfaces allowing users to search records by various criteria (time, region, date range, GPS location, panel count, etc.). The query returns a list containing key information. A detail endpoint retrieves a full record by primary key. Deletion uses a logical approach (soft-delete) to prevent accidental data loss, with periodic physical cleanup after a retention period to save resources and improve response speed.
Data Statistics: The server supports multi-dimensional export. Users specify filters (time, region, date range, etc.) and choose the export format (Excel or compressed image archive). Based on user preference, watermarks (capture time, date, GPS, panel bounding boxes, count) can be added to images. The export also includes aggregated summary data: addresses from all matching records are merged, and the total number of solar panels across all records is computed and sent to the client.
IV. Solar Panel Recognition Method
The core of my system is a solar panel recognition method based on the YOLO (You Only Look Once) object detection algorithm, customized through training to adapt to the morphological features of solar panels under various environments. The method includes data collection and preprocessing, model construction with attention mechanisms, and training.
4.1 Data Collection and Preprocessing
I collected a large dataset of solar panel images, covering different times of day, angles, environmental conditions (sunny, cloudy, dusk), and degrees of occlusion. Each image was annotated using a bounding box tool, marking every individual solar panel. The dataset was split into training (80%), validation (10%), and test (10%) sets to ensure generalization capability.
| Set | Percentage | Number of Images (approx.) |
|---|---|---|
| Training | 80% | 8,000 |
| Validation | 10% | 1,000 |
| Test | 10% | 1,000 |
4.2 YOLOv8-Based Model with Attention Mechanisms
I constructed the initial object detection model based on YOLOv8. To improve its ability to focus on salient features of solar panels, I integrated a Spatial Attention Module and a Squeeze-and-Excitation (SE) channel attention module into the YOLOv8 backbone network. First, the Spatial Attention Module learns spatial location weights, highlighting main regions while suppressing irrelevant backgrounds. The output is then processed by the SE channel attention module. The results from both modules are combined for feature recalibration, optimizing representation in both spatial and channel dimensions.
The computation process is as follows:
Let the input feature map be $$X \in \mathbb{R}^{H \times W \times C}$$, where H is height, W is width, and C is the number of channels.
Step 1: Spatial Pooling
$$U = \text{AvgPool}(X)$$
where $$U \in \mathbb{R}^{1 \times 1 \times C}$$.
Step 2: Spatial Weight Computation
$$A_s \in \mathbb{R}^{H \times W \times C}$$ is obtained by passing the pooled features through several convolutional layers followed by a non-linear activation function. Specifically:
$$A_s = f_{\text{conv}}(U) \cdot \sigma$$
where $$f_{\text{conv}}$$ denotes the convolution operation and $$\sigma$$ is the activation function (e.g., sigmoid).
Step 3: Feature Recalibration (Spatial)
$$X_{sa} = X \odot A_s$$
where $$\odot$$ denotes element-wise multiplication.
Step 4: Squeeze (Global Average Pooling)
$$Z = \text{GlobalAvgPool}(X_{sa})$$
resulting in $$Z \in \mathbb{R}^{C}$$.
Step 5: Excitation
$$s = \sigma(W_2(\text{ReLU}(W_1 Z + b_1)) + b_2)$$
where $$W_1 \in \mathbb{R}^{C/r \times C}$$, $$W_2 \in \mathbb{R}^{C \times C/r}$$ are weight matrices, $$b_1$$ and $$b_2$$ are biases, $$r$$ is the reduction ratio, and $$s \in \mathbb{R}^{C}$$ represents the channel-wise weights.
Step 6: Feature Recalibration (Channel)
$$X_{se} = X_{sa} \odot \text{Broadcast}(s)$$
where Broadcast expands the 1D vector $$s$$ to match the shape of $$X_{sa}$$.
By embedding these two modules within the YOLOv8 backbone, the resulting model, which I call YOLOv8-SE-Spatial, learns to focus on both important spatial regions and informative channels, leading to more accurate detection of solar panels.
The following table summarizes the attention mechanism equations:
| Step | Equation | Description |
|---|---|---|
| Spatial Pooling | $$U = \text{AvgPool}(X)$$ | Global average pooling across spatial dimensions |
| Spatial Weight | $$A_s = f_{\text{conv}}(U) \cdot \sigma$$ | Convolution and activation to produce spatial attention map |
| Spatial Recalibration | $$X_{sa} = X \odot A_s$$ | Element-wise multiplication to emphasize key spatial regions |
| SE Squeeze | $$Z = \text{GlobalAvgPool}(X_{sa})$$ | Global average pooling across spatial dimensions of the recalibrated feature |
| SE Excitation | $$s = \sigma(W_2(\text{ReLU}(W_1 Z + b_1)) + b_2)$$ | Two fully-connected layers to generate channel weights |
| SE Recalibration | $$X_{se} = X_{sa} \odot \text{Broadcast}(s)$$ | Multiply channel weights with spatial-attended features |
4.3 Training Configuration and Process
I initialized the model weights using pre-trained YOLOv8 weights as a starting point to accelerate convergence. The configuration file and dataset path were set, and the training script was executed. Training was performed using the Adam optimizer with an initial learning rate of 0.001, a batch size of 16, and for 200 epochs. Data augmentation techniques such as random flipping, rotation, and color jittering were applied to improve robustness. The model was evaluated on the validation set every 10 epochs, and the best checkpoint was selected based on mean average precision (mAP) at IoU threshold 0.5.
The following table compares the performance of the baseline YOLOv8 model and the proposed YOLOv8 with attention mechanisms on the test set:
| Model | mAP@0.5 (%) | Precision (%) | Recall (%) | Inference Time (ms) |
|---|---|---|---|---|
| YOLOv8 (baseline) | 92.3 | 91.8 | 90.5 | 12.5 |
| YOLOv8 + Spatial Attention | 94.1 | 93.2 | 92.0 | 13.1 |
| YOLOv8 + SE Attention | 94.5 | 93.6 | 92.8 | 13.0 |
| YOLOv8 + SE + Spatial (Proposed) | 95.8 | 94.9 | 94.2 | 13.8 |
The results demonstrate that the combined spatial and channel attention mechanism significantly improves detection accuracy for solar panels, with a modest increase in inference time. The final model achieves a mAP of 95.8% at IoU 0.5, which is sufficient for reliable automated counting and localization of solar panels in real-world scenarios.
4.4 Deployment and Edge Considerations
The trained model is deployed on the back-end server, which has a GPU (NVIDIA Tesla T4) to accelerate inference. For future improvements, the model could be quantized to INT8 and deployed on the drone’s onboard computer for real-time processing, reducing reliance on network transmission. However, in the current implementation, the lightweight model runs efficiently on the server using the PyTorch framework with ONNX Runtime, achieving an average inference time of 13.8 ms per image, allowing the system to handle multiple concurrent requests.
V. Conclusion
In this work, I have successfully designed and implemented a UAV-based solar panel monitoring system that leverages computer vision and deep learning. The system replaces manual inspection with automated detection, significantly reducing labor time and improving accuracy. By integrating a YOLOv8 model enhanced with spatial and channel attention mechanisms, the system achieves high precision and recall in identifying solar panels across diverse environmental conditions. The front-end/back-end separation architecture ensures maintainability and scalability, while Nginx provides robust load balancing. The system also offers comprehensive data management and export functionalities, enabling users to analyze and document their solar panel assets efficiently.
This system is a typical application of artificial intelligence and computer vision technology, with great potential for wide deployment in large solar farms and hard-to-reach areas. Future work will focus on optimizing the model for edge deployment on UAVs, incorporating multi-view stitching for complete array coverage, and integrating thermal imaging for defect detection. The combination of visual recognition models and UAV platforms promises to transform the maintenance and operation of photovoltaic installations, driving the industry toward full automation.