In this thesis, I investigate the problem of segmenting infrared images of solar panels under low-contrast conditions. The goal is to accurately extract the solar panel regions from complex backgrounds, which is essential for automated fault detection and maintenance of photovoltaic systems. I propose two complementary approaches: the first is a traditional method based on local entropy combined with level-set bias correction and image enhancement; the second is a deep learning method based on the Mask-RCNN network. Both methods are designed to handle insufficient contrast, color non-uniformity, and background noise. Through extensive experiments, I demonstrate that the proposed methods achieve better precision and recall than several existing segmentation algorithms.
1 Introduction
Solar panels are the core components of photovoltaic power generation systems. In practical maintenance, it is critical to inspect solar panels for defects such as hot spots, which appear as bright regions in infrared images. Using drones to capture infrared images of solar panels allows rapid inspection, but manually examining thousands of images is inefficient and error-prone. Therefore, computer-aided segmentation of solar panels from infrared images is highly desirable. However, these images often suffer from low contrast, uneven illumination, and complex backgrounds, making segmentation challenging.
Traditional segmentation methods such as thresholding, edge detection, and region growing have limited performance under such conditions. Recent advances in deep learning have provided powerful tools for semantic segmentation, but they require large annotated datasets and careful preprocessing. In this work, I aim to develop robust segmentation algorithms specifically for low-contrast infrared images of solar panels.
The remainder of this paper is organized as follows. Section 2 reviews the background of image enhancement and segmentation. Section 3 presents the local entropy based method. Section 4 introduces the Mask-RCNN based method. Section 5 reports experimental results and comparisons. Section 6 concludes the work.
2 Background and Related Work
2.1 Image Enhancement
Image enhancement is often a necessary preprocessing step for low-contrast images. Several classical techniques exist, including grayscale transformation, histogram equalization, Retinex-based methods, and low-light image enhancement (LIME).
Grayscale transformation maps each pixel value through a function \(g(x,y)=T[f(x,y)]\). For a linear mapping, we have \(g(x,y)=a f(x,y)+b\), where \(a\) controls the slope and \(b\) controls the offset. Nonlinear transforms include logarithmic and exponential functions: \(g(x,y)=c \log(1+f(x,y))\) and \(g(x,y)=c f(x,y)^\gamma\). These simple methods may not handle global contrast well.
Retinex theory assumes that an image \(S(x,y)\) can be decomposed into reflectance \(R(x,y)\) and illumination \(L(x,y)\): \(S(x,y)=L(x,y)\cdot R(x,y)\). By estimating the illumination using Gaussian filters, the reflectance can be recovered. The single-scale Retinex formula is:
$$
r(x,y)=\log S(x,y)-\log\left[F(x,y)*S(x,y)\right]
$$
where \(F\) is a Gaussian kernel. Multi-scale Retinex combines several scales:
$$
R(x,y)=\sum_{i=1}^{K} \omega_i\left\{\log S(x,y)-\log\left[F_i(x,y)*S(x,y)\right]\right\}
$$
The LIME method (Guo et al.) estimates only the illumination component, rather than both illumination and reflectance, which reduces computational cost. The model is \(L=R\circ T\), where \(T\) is the illumination map. The initial estimate is \(T'(x)=\max_{c\in\{R,G,B\}} L_c(x)\). Then an optimization problem is solved to refine \(T\).
2.2 Image Segmentation
Thresholding is the simplest segmentation approach. It selects a threshold \(T\) to separate foreground from background. Otsu’s method chooses \(T\) by maximizing the between-class variance. However, thresholding fails for low-contrast or non-uniform images.
Active contour models (Snakes) minimize an energy functional along a curve. The original Snakes energy is:
$$
E_{snake} = \int_0^1 \left[ E_{internal}(v(s)) + E_{external}(v(s)) \right] ds
$$
where \(E_{internal}\) enforces smoothness and \(E_{external}\) attracts the curve to edges. Level set methods represent the contour implicitly as the zero level set of a function \(\phi(x,y)\). Region-based level set models, such as the one by Li et al., handle intensity inhomogeneity through a bias field.
Deep learning based segmentation has become prominent. Fully convolutional networks (FCN), U-Net, and Mask-RCNN are widely used. Mask-RCNN extends Faster R-CNN by adding a mask branch for pixel-wise segmentation.
3 Local Entropy Based Segmentation for Solar Panels
In this section, I present the first proposed method, which combines contrast enhancement, level-set bias correction, and local entropy segmentation. The overall flowchart consists of the following steps: (1) enhance the original infrared image using histogram equalization and the V channel from HSV color space; (2) perform bias correction using a modified level set algorithm with an edge stopping function; (3) compute local entropy maps and fuse them; (4) threshold the entropy map to obtain an initial segmentation; (5) apply morphological operations and contour filtering to extract the final solar panel regions.
3.1 Contrast Enhancement
Given a grayscale image \(I\) of size \(M\times N\) with pixel values in \([0,L-1]\), the probability of pixel value \(k\) is \(p(k) = n_k / (M N)\), where \(n_k\) is the number of pixels with value \(k\). The cumulative distribution function is \(c(k)=\sum_{i=0}^{k} p(i)\). Histogram equalization maps each pixel to \(h(k)=c(k)(L-1)\).
In addition, I convert the image from RGB to HSV color space. The V channel is defined as:
$$
V = \max(R, G, B)
$$
The V channel captures brightness information and is more robust to color variations. I combine the histogram-equalized grayscale image and the V channel image for subsequent processing.
3.2 Level Set Bias Correction
The level set method by Li et al. models an image \(I(x)\) as \(I(x)=b(x)c(x)+n(x)\), where \(b\) is the bias field, \(c\) is the true signal, and \(n\) is noise. The energy functional is:
$$
\mathcal{F}(\phi,c,b) = \varepsilon(\phi,c,b) + \nu L(\phi) + \mu R_p(\phi)
$$
where
$$
\varepsilon(\phi,c,b) = \int \sum_{i=1}^{N} \int K(y-x) \left| I(x) – b(x) c_i \right|^2 M_i(\phi(x)) dx dy
$$
$$
L(\phi) = \int |\nabla H(\phi(x))| dx
$$
$$
R_p(\phi) = \int p(|\nabla \phi|) dx
with \(H\) being the Heaviside function and \(M_i\) the membership functions. To speed up convergence, I add an edge stopping function:
$$
g_k(x) = \frac{1}{1 + \left( \frac{|\nabla I(x)|^2}{K} \right)}
$$
where \(K = \lambda \max\{|\nabla I|^2\}\) with \(\lambda \in (0,0.1]\). The evolution equation becomes:
$$
\frac{\partial \phi}{\partial t} = -\delta(\phi) (e_1 – e_2) g_k + \nu \delta(\phi) \operatorname{div}\left( g_k \frac{\nabla \phi}{|\nabla \phi|} \right) + \mu \operatorname{div}\left( d_p(|\nabla \phi|) \nabla \phi \right)
$$
After solving for \(\phi\), the bias field is updated and the image is corrected.
3.3 Local Entropy Segmentation
The local entropy of an image measures the randomness of pixel intensities in a local window. For a window of size \(9\times 9\), I compute the probability of each gray level in the window and then calculate the entropy:
$$
H_{local} = – \sum_{i=0}^{255} p_i \log p_i
$$
For each pixel, I compute the entropy using the histogram-equalized image and the V channel image, obtaining \(H_h\) and \(H_v\). The fused entropy is:
$$
H_f = 0.3 H_h + 0.7 H_v
$$
I then compute the mean entropy \(H_m\) of the whole image and apply the following rule:
$$
\begin{cases}
H_f \le 0.78 H_m, & \text{target pixel}\\
H_f > 0.78 H_m, & \text{background pixel}
\end{cases}
$$
This separates smooth solar panel regions (low entropy) from noisy backgrounds (high entropy).
3.4 Post-processing and Large Panel Assembly
After thresholding, the initial mask contains small solar panel modules (small plates) and some noise. I apply morphological erosion and dilation to remove small noise. Then I detect contours and filter them based on geometric constraints: area, aspect ratio, and the ratio of the contour area to the minimum bounding rectangle area. Let \(contArea\) be the contour area and \(boundRectArea\) be the area of the minimum bounding rectangle. I define:
$$
r_{ratio} = \frac{boundRectArea}{contArea}, \quad w_{ratio} = \frac{boundRectHeight}{boundRectWidth}
$$
Valid small plates satisfy \(contArea > 500\), \(r_{ratio} \le 4\), and \(0.2 < w_{ratio} < 5\). I then dilate the image to merge small plates into large panels. A large panel is accepted if it contains at least two valid small plates; otherwise it is discarded as noise. The final mask is used to extract the solar panel regions.
4 Mask-RCNN Based Segmentation for Solar Panels
The second proposed method utilizes the Mask-RCNN network, which is a state-of-the-art instance segmentation framework. The process includes data augmentation, contrast enhancement, and model training.
4.1 Data Augmentation
Since the original dataset is limited and the solar panels appear mostly in a vertical alignment, I augment the training and validation sets to increase diversity. The augmentation operations include random cropping with different scales, rotation by various angles, flipping, and adjusting brightness/saturation. For example, I perform center cropping with sizes \(410\times410\), \(358\times448\), random cropping with sizes \(400\times430\), \(358\times400\), horizontal/vertical/diagonal flipping, rotations of 90°, 180°, and 270°, and brightness/saturation adjustments. After augmentation, all images are resized to \(512\times640\) pixels. The augmented dataset helps the model generalize better to panels of different orientations and scales.
4.2 Contrast Enhancement Using Caip and Guided Filter
To improve the visibility of low-contrast infrared images, I combine the Caip enhancement algorithm with the guided filter method. The Caip algorithm (Ying et al.) fuses multiple exposure images using a weight map. The fused result \(R\) is computed as:
$$
R = \sum_{k=1}^{K} W_k \left( g(P, k) \right)
$$
More specifically, for a single input image \(P\), the algorithm estimates a weight map \(W = T^p\), where \(T\) is the scene illumination map obtained by optimizing:
$$
\min_T \left\| T – \tilde{T} \right\|_2^2 + \alpha \sum_{x} \frac{\left| \nabla T(x) \right|^2}{L(x)}
$$
The brightness transform function is \(g(P,k) = P + k \cdot P(1-P)\), where \(k\) is the exposure ratio determined by maximizing the entropy of the transformed image.
The guided filter is an edge-preserving smoothing filter. Given a guidance image \(I\) and an input image \(P\), the output \(Q\) is modeled as a linear transform of \(I\) in a local window \(w_k\):
$$
Q_i = a_k I_i + b_k, \quad \forall i \in w_k
$$
The coefficients \(a_k\) and \(b_k\) are obtained by minimizing:
$$
E(a_k,b_k) = \sum_{i \in w_k} \left( (a_k I_i + b_k – P_i)^2 + \epsilon a_k^2 \right)
$$
The solution is:
$$
a_k = \frac{\frac{1}{|\omega|} \sum_{i \in w_k} I_i P_i – \mu_k \bar{P}_k}{\sigma_k^2 + \epsilon}, \quad b_k = \bar{P}_k – a_k \mu_k
$$
where \(\mu_k\) and \(\sigma_k^2\) are the mean and variance of \(I\) in \(w_k\), and \(\bar{P}_k\) is the mean of \(P\). The enhanced image \(Q\) is then obtained by combining the filtered output with the original image. Finally, the Caip result \(R\) and the guided filter result \(Q\) are fused as:
$$
X = 0.6 R + 0.4 Q
$$
This combined enhancement improves contrast while preserving edges, which benefits the subsequent segmentation.
4.3 Mask-RCNN Model
Mask-RCNN consists of a backbone network (ResNet101 with Feature Pyramid Network) for feature extraction, a Region Proposal Network (RPN) for generating candidate regions, and a head network that performs classification, bounding box regression, and mask prediction. The RoIAlign layer replaces RoIPool to avoid misalignment. The loss function is a combination of classification loss, bounding box loss, and mask loss:
$$
L = L_{cls} + L_{box} + L_{mask}
$$
For our binary segmentation task, the mask branch outputs a binary mask for the solar panel class. I train the model with a batch size of 1, a learning rate of 0.001, and 43 epochs with a step size of 160. The pretrained model uses 40 epochs with a step size of 120 on the original data. Data augmentation and enhancement are applied to the training and validation sets. The final model is used to segment test images.
5 Experimental Results
5.1 Dataset and Evaluation Metrics
I use a dataset of 98 infrared images of solar panels, each of size \(512\times640\). The training set consists of 923 images and the validation set of 294 images before augmentation. After augmentation, I select 1650 training images and 580 validation images. The test set contains the 98 images. For quantitative evaluation, I use the following metrics: True Positive (TP), True Negative (TN), False Positive (FP), False Negative (FN), Recall (Re), Specificity (Sp), False Positive Rate (FPR), False Negative Rate (FNR), Percentage of Wrong Classification (PWC/PBC), Precision (Pr), and F-measure (F1). They are defined as:
$$
Re = \frac{TP}{TP+FN}, \quad Sp = \frac{TN}{TN+FP}, \quad FPR = \frac{FP}{FP+TN}
$$
$$
FNR = \frac{FN}{TP+FN}, \quad PWC = 100 \times \frac{FN+FP}{TP+FN+FP+TN}
$$
$$
Pr = \frac{TP}{TP+FP}, \quad F1 = \frac{2 \cdot Pr \cdot Re}{Pr+Re}
$$
5.2 Qualitative Comparison
I compare the proposed local entropy method (ours3) and the Mask-RCNN method (ours4) with several existing techniques: Graphcut, Tufenge, Localized active contours, U-Net, and different enhancement methods (Retinex, LIME, Caip, Cdsa). Figure 3.3 (not shown) illustrates the impact of different enhancement techniques; our combined enhancement yields less background noise and more complete solar panel regions. Figure 3.4 demonstrates that each step in the local entropy method is necessary; removing any step degrades the segmentation. Figure 4.6 shows the necessity of data augmentation and enhancement for the Mask-RCNN method. Figure 4.7 shows that our method outperforms other segmentation methods in terms of completeness and noise reduction.
5.3 Quantitative Results
In this section, I present the quantitative results in tables. All results are averaged over the 98 test images.
| Enhancement | Re | Sp | FPR | FNR | PWC | Pr | F1 |
|---|---|---|---|---|---|---|---|
| Cdsa | 0.8769 | 0.8723 | 0.1277 | 0.0612 | 12.8095 | 0.7793 | 0.7994 |
| Caip | 0.8252 | 0.7905 | 0.2095 | 0.0737 | 19.4955 | 0.6449 | 0.7050 |
| LIME | 0.8517 | 0.8859 | 0.1141 | 0.0677 | 12.2533 | 0.7975 | 0.7991 |
| Retinex | 0.6532 | 0.8535 | 0.1465 | 0.1462 | 20.4344 | 0.6509 | 0.6304 |
| Ours (local entropy) | 0.9703 | 0.9023 | 0.0977 | 0.0174 | 7.4548 | 0.8387 | 0.8951 |
Table 3.1 shows that our local entropy method achieves the highest recall (0.9703) and F1 score (0.8951), while maintaining a low false positive rate. This indicates that the proposed enhancement step is effective for low-contrast solar panel images.
| Method | Re | Sp | FPR | FNR | PWC | Pr | F1 |
|---|---|---|---|---|---|---|---|
| Only HSV | 0.9615 | 0.8687 | 0.1313 | 0.0197 | 10.1297 | 0.7872 | 0.8550 |
| Combined (Ours) | 0.9703 | 0.9023 | 0.0977 | 0.0174 | 7.4548 | 0.8387 | 0.8951 |
| Only HistEq | 0.7915 | 0.8940 | 0.1060 | 0.1101 | 14.2354 | 0.7829 | 0.7675 |
| Without local entropy | 0.9656 | 0.3100 | 0.6900 | 0.0161 | 46.6822 | 0.4299 | 0.5801 |
| Without level set | 0.7963 | 0.8255 | 0.1745 | 0.1012 | 18.6768 | 0.6896 | 0.7235 |
Table 3.2 demonstrates that every component in the local entropy method contributes to the final performance. Without the local entropy step, the specificity drops dramatically (0.31), meaning many background pixels are incorrectly classified as foreground. Without the level set bias correction, the recall and precision both decrease significantly.
| Method | Re | Sp | FPR | FNR | PWC | Pr | F1 |
|---|---|---|---|---|---|---|---|
| Graphcut | 0.6466 | 0.7864 | 0.2136 | 0.1645 | 25.9111 | 0.6382 | 0.6058 |
| Tufenge | 0.5963 | 0.7682 | 0.2318 | 0.1648 | 26.6805 | 0.5921 | 0.5465 |
| Localized | 0.9584 | 0.7528 | 0.2472 | 0.0216 | 16.6882 | 0.7021 | 0.8034 |
| Ours (local entropy) | 0.9703 | 0.9023 | 0.0977 | 0.0174 | 7.4548 | 0.8387 | 0.8951 |
Table 3.3 shows that the proposed local entropy method outperforms all traditional segmentation methods in terms of precision and F1 score, while obtaining comparable recall to the Localized method but with much higher specificity.
For the Mask-RCNN method, I perform a similar ablation study. The results are shown in Table 4.1.
| Method | Re | Sp | FPR | FNR | PWC | Pr | F1 |
|---|---|---|---|---|---|---|---|
| Original (no augmentation) | 0.8487 | 0.9675 | 0.0328 | 0.0736 | 6.9674 | 0.9135 | 0.8753 |
| Data augmentation | 0.8989 | 0.9773 | 0.0227 | 0.0535 | 4.8800 | 0.9472 | 0.9203 |
| Test set enhancement | 0.8992 | 0.9797 | 0.0203 | 0.0526 | 4.7025 | 0.9511 | 0.9223 |
| All data enhancement (Ours) | 0.9040 | 0.9830 | 0.0170 | 0.0506 | 4.3493 | 0.9579 | 0.9286 |
Table 4.1 confirms that each step (data augmentation and contrast enhancement) improves the network’s performance. The final model with all enhancements achieves the best F1 score of 0.9286 and precision of 0.9579, while maintaining a recall of 0.9040.
Table 4.2 compares the Mask-RCNN based method with other segmentation algorithms and with the local entropy method from the previous chapter.
| Method | Re | Sp | FPR | FNR | PWC | Pr | F1 |
|---|---|---|---|---|---|---|---|
| Graphcut | 0.6466 | 0.7864 | 0.2136 | 0.1645 | 25.9111 | 0.6382 | 0.6058 |
| U-Net | 0.8174 | 0.9665 | 0.0335 | 0.0932 | 8.4963 | 0.9039 | 0.8489 |
| Localized | 0.9584 | 0.7528 | 0.2472 | 0.0216 | 16.6882 | 0.7021 | 0.8034 |
| Chapter 3 method | 0.9703 | 0.9023 | 0.0977 | 0.0174 | 7.4548 | 0.8387 | 0.8951 |
| Chapter 4 method (Mask-RCNN) | 0.9040 | 0.9830 | 0.0170 | 0.0506 | 4.3493 | 0.9579 | 0.9286 |
From Table 4.2, it is evident that the Mask-RCNN method achieves the highest precision (0.9579) and F1 score (0.9286) among all compared methods. The local entropy method has a slightly higher recall (0.9703) because it tends to include more foreground pixels, but it also has a higher false positive rate and lower precision. The Mask-RCNN method produces cleaner boundaries and fewer background errors.
6 Conclusion and Future Work
In this thesis, I have addressed the challenging problem of segmenting solar panels from low-contrast infrared images. I proposed two effective methods: (1) a traditional approach based on local entropy, level set bias correction, and morphological post-processing; (2) a deep learning approach based on Mask-RCNN with data augmentation and a combined contrast enhancement strategy.
The local entropy method is fully automatic and does not require labeled training data. It achieves high recall and robust segmentation by combining histogram equalization, HSV V-channel, bias correction, and local entropy thresholding. However, it may include some background noise when noise contours have similar shapes to valid panels. Future work could improve the contour filtering or use adaptive thresholds for entropy.
The Mask-RCNN method requires manual annotation and training time, but it yields higher precision and F1 scores. Data augmentation and the Caip-guided-filter enhancement are crucial to its success. The method occasionally fails on very small or partially occluded solar panels at image boundaries. Future improvements might include more advanced augmentation strategies, a lighter backbone network, or post-processing to reconnect broken panels.
In summary, both proposed methods significantly outperform several existing segmentation algorithms on the given dataset. The choice between them depends on the application: if no training data is available, the local entropy method is preferable; if annotated data can be prepared, the Mask-RCNN method provides cleaner segmentation results. I believe this work contributes to the field of solar panel inspection and low-contrast image segmentation.