In modern photovoltaic (PV) power generation systems, the reliable and efficient operation of solar panels is paramount. However, over long-term operation, solar panels are inevitably subject to performance degradation and failures due to environmental factors, installation stresses, and maintenance issues. Traditional methods for detecting abnormal states in these solar panels, often reliant on intelligent image analysis, frequently lack a robust process for analyzing the correlation between key sample features. Moreover, neural network-based recognition processes can become trapped in iterative cycles when dealing with similar nonlinear features, leading to unacceptably high rates of missed and false detections for abnormal states. This significantly impacts the overall power generation efficiency and economic return of a PV plant. Therefore, developing a more accurate and reliable detection method is of critical importance for operational maintenance.
We propose a novel method for detecting abnormal states in solar panels within photovoltaic power plants, leveraging the combined strengths of covariance matrix analysis and fuzzy neural networks (FNN). The core innovation lies in its holistic approach: it not only captures the visual symptoms of failure from the panel’s surface but also analyzes the intricate correlations within the panel’s internal operational data. This dual-path analysis significantly enhances detection accuracy. The method begins with the acquisition of both surface images and internal operational data from the solar panels. The initial images undergo preprocessing steps including distortion correction, image filtering, and enhancement to improve quality. Subsequently, a covariance matrix is employed to rigorously analyze the inter-relationships between different internal operational parameters, such as irradiance, current, and voltage, extracting rich features that signal internal faults like short circuits or degraded connections. Concurrently, a fuzzy neural network is constructed to process the panel’s operational data, adeptly capturing complex, nonlinear changes and isolating abnormal patterns. This network is also instrumental in extracting robust features from the preprocessed surface images, handling uncertainties like partial shadows or uneven illumination. Finally, based on established detection criteria, the extracted internal and external features are matched against known fault signatures to determine the current abnormal state and type of the solar panel, enabling precise and comprehensive fault diagnosis.
The data acquisition flow for a holistic assessment of solar panel status is illustrated below. We systematically collect both the physical appearance data through imaging and the real-time electrical performance data. This comprehensive dataset forms the foundation for our subsequent covariance and neural network analysis.
We begin our work by acquiring comprehensive state data from the solar panels, encompassing both internal operational parameters and external visual conditions. This forms the foundational input for our detection system.
Internal Operational Data Acquisition for Solar Panels
To diagnose internal faults in solar panels, a deep understanding of their working principle is essential. We collect a suite of internal operational data, including but not limited to: irradiance (W), temperature (T), output voltage (U) and current (I), power output (P), and derived parameters. The relationship between these parameters is key to identifying faults like low irradiance due to soiling, low open-circuit voltage, abnormal short-circuit current, or reduced output power.
The fundamental electrical model of a solar panel, accounting for non-ideal factors, is represented by a single-diode model with series and parallel resistances. The output current of a solar panel can be described as:
$$I_t = I_{ph} – I_0 \left[ \exp\left(\frac{U_t + I_t R_s}{a \cdot k}\right) – 1 \right] – \frac{U_t + I_t R_s}{R_p}$$
where \(I_{ph}\) is the photogenerated current (proportional to irradiance), \(I_0\) is the diode reverse saturation current, \(R_s\) and \(R_p\) are the series and shunt resistances, \(a\) is the ideality factor, and \(k\) is Boltzmann’s constant. The real-time voltage \(U_t\) is measured relative to the maximum power point or open-circuit conditions. Effective irradiance \(W_t\) accounts for direct, diffuse, and reflected components:
$$W_t = k_{cover}(W_{projectivity} + W_{scattering} + W_{reflect})$$
The instantaneous power \(P_t\) is calculated as \(P_t = U_t \cdot I_t\). Data acquisition devices are connected to the solar panels to log these parameters continuously. To ensure consistency and improve analysis, the raw collected data samples are standardized using Min-Max normalization:
$$d’ = \frac{d – \min(d)}{\max(d) – \min(d)}$$
where \(d\) is the original data point and \(d’\) is the normalized value scaled between 0 and 1.
External Image Acquisition for Solar Panels
Internal data alone cannot reveal external defects. Therefore, we simultaneously capture visual and thermal images of the solar panel surfaces. We employ an imaging system capable of capturing both visual spectrum and infrared (IR) images. The IR imaging is crucial for detecting thermal anomalies like hot spots, which are indicative of series resistance faults, shading, or cell cracks. The principle involves measuring the infrared radiation emitted from the panel’s surface, which is related to its temperature and emissivity. The pixel intensity at a location \((x_{panel}, y_{panel})\) in the generated image can be quantitatively related to the panel’s temperature \(T\) and the radiation constants \(\beta_1, \beta_2\) for the IR spectrum wavelength \(\lambda\):
$$\delta = \beta_1 \lambda^{-5} \left( \exp\left(\frac{\beta_2}{\lambda T}\right) – 1 \right)^{-1}$$
$$x_{image} = x_{panel} \cdot \delta, \quad y_{image} = y_{panel} \cdot \delta$$
By stitching together the pixel data according to spatial relationships, a complete visual and thermal image of the solar panel is formed for analysis.
Image Preprocessing for Solar Panels
The raw images often contain noise, lens distortion, and uneven contrast, which can hinder accurate feature extraction. We implement a three-stage preprocessing pipeline specifically tailored for solar panel imagery.
1. Distortion Correction: Lens distortions, both radial and tangential, are corrected. If \((x, y)\) is a point in the original image, its corrected coordinates \((\Delta x, \Delta y)\) are found by solving:
$$x_{distorted} = x(1 + \zeta_j r^2 + \zeta_q r^4) + 2\zeta_1 xy + \zeta_2(r^2 + 2x^2)$$
$$y_{distorted} = y(1 + \zeta_j r^2 + \zeta_q r^4) + 2\zeta_2 xy + \zeta_1(r^2 + 2y^2)$$
where \(\zeta_j, \zeta_q\) are radial distortion coefficients, \(\zeta_1, \zeta_2\) are tangential distortion coefficients, and \(r^2 = x^2 + y^2\). The correction parameters are obtained through calibration.
2. Image Filtering: We employ frequency-domain filtering to remove periodic noise (like striping) and other high-frequency artifacts. The 2D Fourier Transform \(I_{Fourier}(\Delta x, \Delta y)\) of the distortion-corrected image \(I(\Delta x, \Delta y)\) is computed:
$$I_{Fourier}(u, v) = \sum_{\Delta x=0}^{M-1} \sum_{\Delta y=0}^{N-1} I(\Delta x, \Delta y) e^{-j2\pi (u\Delta x/M + v\Delta y/N)}$$
A suitable frequency-domain filter \(\vartheta(u,v)\) is applied, and the inverse transform yields the filtered image:
$$I_{filtering}(\Delta x, \Delta y) = \mathcal{F}^{-1}\{I_{Fourier}(u, v) \cdot \vartheta(u, v)\}$$
3. Image Enhancement: Histogram equalization is used to improve contrast. The transformation function is calculated from the cumulative distribution of the filtered image’s histogram:
$$s_k = (L-1) \sum_{i=0}^{k} \frac{n_i}{MN}$$
where \(L\) is the number of grey levels, \(n_i\) is the count of pixels at grey level \(i\), and \(M, N\) are image dimensions. The final enhanced image \(I_{Equalize}\) is obtained by mapping original intensities to \(s_k\).
Feature Extraction from Solar Panel Data
With clean data in hand, we proceed to extract discriminative features that signal abnormal conditions in the solar panels. We use two complementary techniques: covariance matrix analysis for internal data and fuzzy neural networks for image data.
Covariance Matrix Analysis for Internal Operational Features
The normalized internal operational data (e.g., \(U_t\), \(I_t\), \(P_t\), \(W_t\)) for \(e\) samples, each with \(o\) features, is arranged into a matrix \(\mathbf{\Psi}\) of size \(e \times o\). The covariance matrix \(\mathbf{C}\) is computed to reveal the linear relationships between all feature pairs:
$$\mathbf{C} = \frac{1}{e-1} (\mathbf{\Psi} – \bar{\mathbf{\Psi}})^T (\mathbf{\Psi} – \bar{\mathbf{\Psi}})$$
where \(\bar{\mathbf{\Psi}}\) is the mean vector across samples. The diagonal elements of \(\mathbf{C}\) are the variances of individual features, while off-diagonal elements are covariances indicating how features change together. For instance, under normal conditions, current and irradiance are highly correlated. A breakdown in this correlation, visible in the covariance matrix, can be a strong indicator of an internal fault like a cracked cell or faulty bypass diode. We extract a feature vector \(\boldsymbol{\tau}_{work}\) from this matrix that encapsulates these relational characteristics, which is more robust than looking at parameters in isolation.
Fuzzy Neural Network for External Image Feature Extraction
Solar panel surface images contain uncertainties like blurred edges, varying illumination, and ambiguous defect boundaries. Fuzzy Neural Networks (FNN) are ideal for handling such imprecise information. Our FNN structure consists of five layers: Input, Fuzzification, Rule, Defuzzification, and Output.
1. Fuzzification Layer: Each input pixel intensity or derived texture measure from \(I_{Equalize}\) is fuzzified. For the \(i\)-th input variable with \(h\) fuzzy sets, the membership degree \(\psi_h(i)\) is calculated using a Gaussian membership function:
$$\psi_h(i) = \exp\left[-\left(\frac{I_{Equalize}(x,y)_i – \chi_i}{\zeta_i}\right)^2\right]$$
where \(\chi_i\) and \(\zeta_i\) are the center and width of the \(h\)-th fuzzy set for that input.
2. Rule Layer & Defuzzification: The firing strength \(\gamma\) of a rule is computed as the product of incoming membership degrees. These are normalized to \(\mu_i\).
3. Output Layer: The final clear output, representing a specific image feature (e.g., texture entropy, contour shape descriptor), is obtained via weighted averaging:
$$\varphi_{ij} = f_{activation}\left(\sum (\omega_{ij} \cdot \mu_i) – b_{ij}\right)$$
where \(\omega_{ij}\) and \(b_{ij}\) are weights and biases learned during training. The network is trained to output robust features like \(\tau_{texture} = \varphi(I_{Equalize}(x,y))\) for texture, and similarly for geometric and shape features. All extracted image features are fused into a comprehensive external state vector \(\boldsymbol{\tau}_{SurfaceImaging}\).
Abnormal State Detection for Solar Panels
The final stage involves synthesizing the extracted internal and external features to diagnose the state of the solar panel.
External Abnormal State Detection via Feature Matching
Common external anomalies for solar panels include fragmentation, cracks, and hot spots. We have pre-established feature templates \(\boldsymbol{\tau}_{standard}\) for each of these anomaly types from a known database. The matching degree \(s\) between the extracted panel features \(\boldsymbol{\tau}_{SurfaceImaging}\) and a template is computed using a similarity measure, such as the cosine similarity:
$$s = \frac{\boldsymbol{\tau}_{SurfaceImaging} \cdot \boldsymbol{\tau}_{standard}}{\|\boldsymbol{\tau}_{SurfaceImaging}\| \|\boldsymbol{\tau}_{standard}\|}$$
If \(s\) exceeds a threshold \(s_0\) for a particular fault template, the panel is flagged with that external anomaly. Furthermore, for surface defects, the approximate affected area \(\sigma_{surface}\) can be calculated from the segmented defect region in the image using polygon area formulas:
$$\sigma_{surface} = \frac{1}{2} \left| \sum_{i=1}^{n} (x_i y_{i+1} – x_{i+1} y_i) \right|$$
where \((x_i, y_i)\) are the vertices of the defect polygon.
Internal Abnormal State Detection
Internal faults primarily include short-circuit and open-circuit conditions within the solar panel cells or connections. Characteristic signatures for these faults are defined in the internal feature space. For example, a significant deviation in the relationship between voltage and current, as captured by the covariance-based features \(\boldsymbol{\tau}_{work}\), can indicate a short circuit (\(\boldsymbol{\tau}_{short}\)) or open circuit (\(\boldsymbol{\tau}_{open}\)). The fault condition is triggered if the measured internal feature vector shows a high similarity to a fault template or deviates beyond a statistically defined threshold from the normal cluster. The magnitude of an internal current anomaly, for instance, is quantified as:
$$\sigma_{internal} = | I_t – I_{normal}(\boldsymbol{\tau}_{work}) |$$
where \(I_{normal}(\boldsymbol{\tau}_{work})\) is the expected current given the other operational features under normal conditions.
The final decision on the solar panel’s health integrates evidence from both detection pathways, providing a comprehensive diagnosis: Normal, External Anomaly (with type and area), or Internal Anomaly (with type and severity). This integrated approach significantly improves reliability over methods using only one data source.
Experimental Analysis and Performance Evaluation
To validate the effectiveness of our proposed method for solar panels, we conducted a series of experiments comparing it against two established techniques: a method based on semantic segmentation of images and a method based on an improved YOLOv5s algorithm. The evaluation focused on detection accuracy for both internal and external faults in various solar panels.
Experimental Setup
The experiment was conducted using a dataset from a operational PV plant. We selected 500 solar panels of two types: monocrystalline silicon and multi-compound thin-film. All panels were installed at a 5° tilt. A subset of panels was artificially induced with controlled faults, while others were kept in normal condition. Internal data (I-V curves, irradiance) was logged, and simultaneous visual/IR images were captured using a calibrated thermal imager. Fault types and their magnitudes are summarized below.
| Panel Type | Panel ID | Fault Type | Fault Severity |
|---|---|---|---|
| Monocrystalline | D01 | Hot Spot | Area: 0.25 m² |
| D03 | Micro-crack | Length: 0.8 m | |
| D05 | Fragment | Area: 0.2 m² | |
| D07 | Short Circuit | Current: 1.5 A | |
| Multi-Compound | Q01 | Hot Spot | Area: 1.5 m² |
| Q03 | Short Circuit | Current: 3.7 A | |
| Q05 | Cell Discoloration | Area: 1.2 m² | |
| Q07 | Partial Shading | Power Loss: ~30% |
The parameters for the Fuzzy Neural Network were set as follows: input/output nodes=5, nodes in hidden layers=30, learning rate=0.1, iterations=200. The covariance matrix was computed using a sliding window of operational data.
Performance Metrics
We evaluated the methods using three key metrics specific to solar panel fault diagnosis:
1. Fault Magnitude Estimation Error (\(\varepsilon\)): The absolute difference between the detected fault magnitude and the actual induced magnitude. For current faults: \(\varepsilon_{internal} = |\sigma_{internal} – \sigma_{internal-set}|\). For area faults: \(\varepsilon_{surface} = |\sigma_{surface} – \sigma_{surface-set}|\).
2. Missed Detection Rate (\(\eta_{missed}\)): The percentage of truly faulty solar panels that were incorrectly classified as normal.
$$\eta_{missed} = \frac{N_{faulty} – N_{detected}}{N_{faulty}} \times 100\%$$
3. False Detection Rate (\(\eta_{false}\)): The percentage of normal solar panels that were incorrectly flagged as faulty.
$$\eta_{false} = \frac{N_{false-alarm}}{N_{normal}} \times 100\%$$
Results and Discussion
The performance comparison for internal short-circuit current estimation is shown below. Our method demonstrates superior accuracy in quantifying the fault severity.
| Solar Panel ID | Actual Fault Current (A) | Semantic Segmentation Method | Improved YOLOv5s Method | Our Proposed Method |
|---|---|---|---|---|
| D07 | 1.5 | 1.8 | 1.3 | 1.5 |
| D08 | 2.0 | 2.6 | 2.1 | 2.0 |
| Q01 | 3.5 | 3.9 | 3.7 | 3.5 |
| Q03 | 3.7 | 3.1 | 3.2 | 3.6 |
| Average Error (A) | 0.47 | 0.28 | 0.03 | |
Similarly, for external surface faults, our method provides the most accurate estimation of the defective area, which is crucial for planning maintenance actions.
| Solar Panel ID | Actual Fault Area (m²) | Semantic Segmentation Method | Improved YOLOv5s Method | Our Proposed Method |
|---|---|---|---|---|
| D01 | 0.25 | 0.28 | 0.27 | 0.22 |
| D02 | 0.40 | 0.45 | 0.43 | 0.41 |
| D06 | 0.80 | 0.88 | 0.85 | 0.80 |
| Q04 | 1.50 | 1.57 | 1.54 | 1.51 |
| Average Error (m²) | 0.048 | 0.026 | 0.008 | |
The most critical metrics for operational reliability are the missed and false detection rates. The comparative results across a batch of solar panels with mixed faults clearly show the advantage of our integrated approach.
| Evaluation Metric | Semantic Segmentation Method | Improved YOLOv5s Method | Our Proposed Method |
|---|---|---|---|
| Missed Detection Rate (\(\eta_{missed}\)) | 8.4% | 5.2% | 1.8% |
| False Detection Rate (\(\eta_{false}\)) | 6.1% | 4.7% | 1.2% |
The results unequivocally demonstrate that our proposed method, which synergistically combines covariance matrix analysis of operational data with fuzzy neural network processing of imagery, significantly outperforms the traditional single-modality detection methods. The average error for internal current fault estimation was reduced by approximately 0.35 A compared to the next best method, and the surface area estimation error was reduced by about 0.03 m². More importantly, the missed and false detection rates were markedly lower. This is directly attributable to the method’s dual-path analysis: the covariance matrix effectively captures the degrading correlations between electrical parameters that precede or accompany failures, while the fuzzy neural network robustly handles the visual ambiguities and nonlinear patterns in both image and time-series data. This comprehensive diagnostic capability ensures that subtle or nascent faults in solar panels are identified early and accurately, preventing power loss and potential safety hazards.
Conclusion
Solar panels are critical assets in photovoltaic power plants, and their health directly impacts energy yield and system longevity. This work has presented a novel, robust framework for detecting abnormal states in solar panels by integrating multi-source data analysis. The method’s core strength lies in its hybrid approach: utilizing a covariance matrix to extract revealing relational features from internal operational time-series data and employing a fuzzy neural network to extract resilient features from external visual and thermal imagery. This combination effectively addresses the limitations of prior methods, such as the neglect of parameter interdependencies and the poor handling of nonlinear, uncertain patterns. Experimental validation on real-world solar panel data confirms the superior performance of our method. It achieves significantly higher accuracy in fault magnitude estimation and, crucially, dramatically lower rates of missed and false detections compared to state-of-the-art image-only techniques. By enabling precise and early fault diagnosis, this method provides a powerful tool for the predictive maintenance and optimization of photovoltaic power plants, contributing directly to improved grid stability and the economic viability of solar energy.