In the context of the current global energy crisis and environmental protection, the search for sustainable energy alternatives to fossil fuels has become crucial. Solar energy, as a dominant form of new energy, offers broad application prospects. Solar panels are the core components of solar power generation systems, and their performance and reliability directly determine the efficiency of power generation. However, due to long-term operation and harsh environmental conditions, various types of defects may appear on solar panels. These defects can degrade the efficiency of the solar panel, cause energy losses, and even damage the entire photovoltaic system. Therefore, timely detection and repair of these defects are essential. Automating the classification of solar panel defects has become an important research area. Traditional manual inspection methods are time-consuming, labor-intensive, and prone to subjective errors. Consequently, much research is now focused on developing automated classification systems based on computer vision and machine learning techniques, enabling accurate and efficient defect identification.
In recent years, to avoid the high cost of manual annotation, self-supervised learning has attracted widespread attention. Among the various self-supervised methods, contrastive learning stands out due to its high performance and versatility. Instance-level contrastive learning is one of the most typical categories within contrastive learning. Compared with other contrastive paradigms, instance-level contrastive learning is simpler to implement and possesses excellent sample discrimination capability. Moreover, solar panel defect data often exhibits complex characteristics, requiring strong sample processing capacity. Therefore, instance-level contrastive learning has significant potential for the practical task of solar panel defect classification. This article focuses on an improved instance-level contrastive learning approach tailored for solar panel defect classification.
1. Introduction
The demand for energy has been growing globally over recent decades. Traditional fossil fuels are limited, cause severe environmental pollution, and contribute to climate change through greenhouse gas emissions. Fluctuating prices and supply insecurity further challenge economic and energy stability. Renewable energy sources, especially solar energy, offer a sustainable path forward. Unlike fossil fuels, solar energy is inexhaustible, and the operational cost of photovoltaic systems is relatively low. Solar panels, also known as photovoltaic panels, are the core devices that convert sunlight into electricity. They are composed of multiple solar cells that absorb photons and generate electric current. The reliability of such panels is vital for the long-term performance of solar power plants.
Solar panels can develop various defects during manufacturing, transportation, and operation. Cracks, scratches, bubbles, black spots, broken grids, and mismatch problems are among the most common. These defects reduce the conversion efficiency and the lifespan of the solar panels. For example, surface scratches may cause light reflection and reduce absorption; cracks may lead to current leakage and hot spots; and internal structural defects may accelerate degradation. Because different defect types require different maintenance strategies, accurately classifying defects is of paramount importance. A robust classification system enables on-time preventive maintenance and thereby improves the overall reliability of solar energy systems.
Traditional machine vision methods rely on hand-crafted features. Typical techniques include threshold segmentation, morphological operations, and filtering. These approaches depend on expert knowledge and often fail under complex illumination or variable background conditions. Deep learning methods, on the other hand, automatically learn hierarchical features from raw data and have achieved state-of-the-art results on many image classification tasks. However, most existing deep learning models are trained in a fully supervised manner, which requires large amounts of manually labeled data. In many real-world applications, especially in industrial environments, obtaining sufficiently large and accurately labeled datasets is expensive and time-consuming. Furthermore, annotations can introduce information loss because only the annotated objects are used for supervision while rich contextual information is ignored.
Self-supervised learning offers a promising alternative. It leverages unlabeled data and constructs supervisory signals from the data itself. A model trained with self-supervised objectives learns meaningful representations without human annotation. Contrastive learning is a particularly successful self-supervised paradigm. Its core idea is to embed samples into a latent space where similar samples are pulled closer and dissimilar samples are pushed apart. This property is well suited to defect classification, because identifying subtle differences between normal and defective solar panel regions is essential.
This paper investigates the application of instance-level contrastive learning to solar panel defect classification. The main contributions are threefold. First, we employ SE-ResNet50 as the feature extraction backbone to enhance channel-wise attention and improve representation power. Second, we replace the commonly used NT-Xent loss with the Contrastive Loss to better utilize limited data and improve model convergence. Third, we introduce a global self-attention mechanism into the classifier to capture global contextual dependencies, increasing classification accuracy while adding only a small number of parameters. We empirically validate the proposed method on both a benchmark dataset and a solar panel defect dataset, demonstrating consistent improvements.
2. Contrastive Learning Theory and Methods
2.1 Self-Supervised Learning
Self-supervised learning is an umbrella term for unsupervised methods that define auxiliary tasks based on unlabeled data. The training process does not rely on human-provided labels; instead, the supervision comes from some property of the data itself. A typical self-supervised pipeline consists of three stages: designing a proxy task, training an encoder to solve the proxy task, and transferring the learned representations to downstream tasks. The proxy task is not the final objective, but it forces the model to capture informative structures in the input data. For images, common proxy tasks include image colorization, jigsaw puzzle solving, rotation prediction, and contrastive instance discrimination. The quality of the learned representations heavily depends on the choice of proxy task.
Self-supervised methods can be broadly divided into three categories: generative learning, contrastive learning, and generative-contrastive (adversarial) learning. Generative methods, such as autoencoders and generative adversarial networks, attempt to reconstruct the input or synthesize realistic images. They are widely used in natural language processing, for example in word2vec and BERT. However, generative models are often difficult to train and require substantial computational resources. They also operate in high-dimensional pixel space, which may be unnecessary for learning compact representations. Contrastive learning, on the other hand, learns to distinguish between similar and dissimilar samples without explicitly generating data. The contrastive objective encourages the encoder to produce representations that are invariant to transformations while discarding irrelevant details. Adversarial learning balances a generator and a discriminator, but it suffers from instability and mode collapse. For the defect classification of solar panels, contrastive learning is particularly attractive because it directly exploits inter-sample relations, which helps distinguish normal and defective regions with fine granularity.
2.2 Proxy Tasks
A proxy task is a secondary task designed to support the primary task. In self-supervised learning, the proxy task generates pseudo-labels from the data itself. For example, colorization predicts the color channels from a grayscale input; jigsaw completion predicts the arrangement of shuffled patches; rotation prediction estimates the angle of rotation applied to an image. Contrastive learning itself can be viewed as a proxy task: given an anchor sample, a transformed version of it serves as the positive sample, and other samples serve as negatives. The model learns to identify the positive among a set of negatives. This process encourages invariance to the applied transformations while maintaining discriminative power among different samples.
2.3 Categories of Contrastive Learning
Contrastive learning methods can be categorized by the granularity of the contrastive units. We examine three levels: context-level, instance-level, and cluster-level.
2.3.1 Context-level Contrastive Learning
Context-level methods maximize mutual information between global and local features of the same image. The Deep InfoMax (DIM) approach encodes an entire image and compares its global feature vector with local feature vectors from the same image. The goal is to make local features compatible with the global representation. The CMC method extends this idea by maximizing mutual information across multiple views of the same image. These methods excel in learning spatially sensitive representations but tend to require more complex training procedures.
2.3.2 Instance-level Contrastive Learning
Instance-level contrastive learning treats each image as an instance class. Given an anchor, the transformed view is the positive, while all other images in the dataset (or a batch) are negatives. The objective is to make the representation of an image robust to data augmentation while distinguishing it from unrelated images. This paradigm can be implemented with negative samples or without negative samples. Methods with negative sampling include memory-bank-based approaches and end-to-end approaches. For example, MoCo maintains a queue of encoded negatives and uses momentum updates for the key encoder. SimCLR uses a large batch size to provide sufficient negatives and incorporates a projection head to improve downstream accuracy. Methods without negative samples, such as BYOL and SimSiam, employ asymmetric predictor or stop-gradient mechanisms to prevent collapse. Instance-level methods are popular due to their simplicity and excellent performance on large-scale benchmarks.
2.3.3 Cluster-level Contrastive Learning
Cluster-level contrastive learning, also known as prototype contrastive learning, clusters the embedded features and contrasts samples against cluster prototypes. Methods like PCL and SwAV assign images to online-generated clusters and encourage views of the same image to be assigned to the same cluster. These approaches combine clustering and representation learning. However, they inherently require a sufficient amount of data to form stable prototypes. In the solar panel defect domain, the amount of unlabeled data is often limited, making cluster-level methods less effective.
Given the characteristics of solar panel defect datasets, instance-level contrastive learning is the most suitable choice. It performs well with relatively small datasets, does not require a large memory bank or stable clustering, and can still leverage cross-sample information through negative pairs. Therefore, this paper focuses on improving instance-level contrastive learning for solar panel defect classification.
3. Improved Instance-level Contrastive Learning
3.1 Original Method
In the original instance-level contrastive learning framework, an input image is randomly augmented twice to form a positive pair. Both augmented views are fed into a convolutional neural network (typically ResNet) to obtain feature vectors. An additional multilayer perceptron (MLP) projection head maps the features to a lower-dimensional embedding space. The loss is computed on these embeddings using the NT-Xent (Normalized Temperature-Scaled Cross Entropy) loss. After pretraining, the projection head is discarded and the backbone network is frozen. A linear classifier (fully connected layer) is then trained on top of the frozen features using labeled data for classification.
3.2 Improvements and Objectives
Three main improvements are introduced to adapt instance-level contrastive learning to solar panel defect classification. First, SE-ResNet50 is adopted as the backbone to improve feature extraction. Second, the Contrastive Loss is proposed to replace NT-Xent, thereby improving sample utilization and model accuracy. Third, a global self-attention classifier replaces the conventional fully connected layer to capture global feature interactions.
3.3 Feature Extraction Module: SE-ResNet50
ResNet50 is a deep convolutional network built from residual blocks. By adding skip-connections, it alleviates the vanishing gradient problem and facilitates training of deep architectures. For solar panel defect classification, the backbone should be sufficiently deep to model complex defect patterns. However, deeper networks can still suffer from redundant feature representations. To improve the discriminative power, we integrate the Squeeze-and-Excitation (SE) block into each residual unit. The SE block performs adaptive recalibration of channel-wise feature responses. The operation can be expressed as:
$$ \mathbf{z}_c = \frac{1}{H \times W} \sum_{i=1}^{H} \sum_{j=1}^{W} \mathbf{u}_c(i,j), $$
$$\ \mathbf{s} = \sigma \left( \mathbf{W}_2 \, \delta \left( \mathbf{W}_1 \mathbf{z} \right) \right), $$
$$ \tilde{\mathbf{u}}_c = s_c \cdot \mathbf{u}_c, $$
where \(\mathbf{u}_c\) is the \(c\)-th feature map, \(\mathbf{z}_c\) is the result of global average pooling, \(\mathbf{W}_1\) and \(\mathbf{W}_2\) are weights of the two fully connected layers, \(\delta\) is a ReLU activation, and \(\sigma\) is the sigmoid function. The resulting channel weights \(s_c\) emphasize important channels and suppress less informative ones. This mechanism enables the network to highlight defect-specific features that are critical for classification. The overall architecture of SE-ResNet50 is presented below.
3.4 Loss Function Module: Contrastive Loss
The standard NT-Xent loss computes the cosine similarity between two augmented views and treats all other samples as negatives. Let the encoded vectors be \(\mathbf{z}_i\) and \(\mathbf{z}_j\) for two augmented views of the same image. The NT-Xent loss for a positive pair \((i,j)\) is:
$$ \ell_{i,j} = – \log \frac{ \exp(\mathrm{sim}(\mathbf{z}_i, \mathbf{z}_j)/\tau) }{ \sum_{k=1}^{2N} \mathbb{1}_{[k\ne i]} \exp(\mathrm{sim}(\mathbf{z}_i, \mathbf{z}_k)/\tau) }, $$
where \(\mathrm{sim}(\mathbf{u},\mathbf{v}) = \mathbf{u}^\top \mathbf{v} / (\|\mathbf{u}\| \|\mathbf{v}\|)\) is the cosine similarity, \(\tau\) is a temperature parameter, and \(2N\) is the total number of views in a batch. This loss depends heavily on a large batch size to provide enough negatives. In many real-world scenarios such as solar panel defect classification, available computational resources are limited, and the batch size must be kept small. Under these conditions, the NT-Xent loss may lead to insufficient supervision and unstable convergence.
To overcome this issue, we replace NT-Xent with the Contrastive Loss, which directly optimizes the Euclidean distance between paired samples. For a pair of samples \((\mathbf{x}_i, \mathbf{x}_j)\), the Contrastive Loss is defined as:
$$ L = \frac{1}{2N} \sum_{n=1}^{N} \left[ y \, d^2 + (1-y) \, \max(\mathrm{margin} – d, 0)^2 \right], $$
where \(d = \|\mathbf{f}_i – \mathbf{f}_j\|_2\) is the Euclidean distance between the two feature vectors, \(y\) indicates whether the pair is a true pair (\(y=1\)) or an impostor pair (\(y=0\)), and \(\mathrm{margin}\) is a user-defined threshold. If \(y=1\), the loss encourages the distance to be small, pulling the features together. If \(y=0\), the loss encourages the distance to be at least the margin, pushing the features apart. This loss function does not require an extremely large batch size because it only needs paired samples. It naturally fits the solar panel defect classification scenario where data are limited but still need to be fully exploited. Furthermore, the Contrastive Loss is simpler to compute and provides more stable gradients for small-batch training.
3.5 Classifier Module: Global Self-Attention
After pretraining, the projection head is removed and the backbone is frozen. A classifier is trained on top of the frozen feature map. In the original approach, a fully connected layer is used as the classifier. However, a fully connected layer has restrictions: it requires a fixed input dimension and cannot effectively model the spatial relationships that are important for defect recognition. Solar panel defects often have irregular shapes, and their surrounding context may contain crucial information. To capture global contextual dependencies, we introduce a global self-attention mechanism into the classifier.
The self-attention mechanism computes attention scores between every pair of spatial or sequential positions. Given an input feature map \(\mathbf{X} \in \mathbb{R}^{C \times N}\) (where \(N = H \times W\) is the number of spatial positions), we compute query, key, and value matrices:
$$ \mathbf{Q} = \mathbf{W}_Q \mathbf{X}, \quad \mathbf{K} = \mathbf{W}_K \mathbf{X}, \quad \mathbf{V} = \mathbf{W}_V \mathbf{X}. $$
The attention output is:
$$ \mathrm{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V}) = \mathrm{softmax}\left( \frac{\mathbf{Q}^\top \mathbf{K}}{\sqrt{d_k}} \right) \mathbf{V}, $$
where \(d_k\) is the dimension of the key vectors. This operation allows the model to attend to all positions when producing the output feature, thus capturing global relationships. In the context of solar panel defects, self-attention can relate a small crack to a distant busbar or a dark area caused by a cell failure. By aggregating such global information, the classifier becomes more discriminative.
The proposed classifier architecture consists of a global average pooling layer, a global self-attention block, and a final fully connected layer that outputs the class probabilities. The self-attention block adds a modest number of parameters but substantially improves classification accuracy by modeling long-range dependencies.
4. Experiments on a Benchmark Dataset
To validate the proposed improvements, we first conduct experiments on the STL10 benchmark dataset. STL10 contains 10 classes with 500 labeled training images per class and 800 test images per class. Additionally, it provides 100,000 unlabeled images, which is suitable for self-supervised pretraining. The experiments are designed to compare the original instance-level contrastive learning with our improved version under controlled settings.
The experimental environment is listed below.
| Component | Configuration |
|---|---|
| Operating System | Ubuntu 20.04 |
| CPU | AMD Ryzen 7 5800H 3.20 GHz |
| GPU | NVIDIA RTX 3060 6G |
| Memory | 16 GB |
| Programming Language | Python 3.9.0 (Anaconda) |
| Deep Learning Framework | PyTorch, Sklearn, OpenCV |
We compare four configurations: ResNet50 with NT-Xent, SE-ResNet50 with NT-Xent, ResNet50 with Contrastive Loss, and SE-ResNet50 with Contrastive Loss. Each configuration is evaluated with two classifier types: a plain fully connected layer (FC) and the proposed global self-attention classifier (Self-Attention). The pretraining batch size is set to 32, the projection head outputs 128 dimensions, and the learning rate is \(3\times10^{-4}\). The results are reported in terms of Top-1 and Top-5 accuracy.
| Backbone | Loss Function | Classifier | Top-1 Acc. | Top-5 Acc. |
|---|---|---|---|---|
| ResNet50 | NT-Xent | FC | 69.04% | 97.71% |
| ResNet50 | NT-Xent | Self-Attention | 72.01% | 98.48% |
| ResNet50 | Contrastive Loss | FC | 72.37% | 98.50% |
| ResNet50 | Contrastive Loss | Self-Attention | 74.36% | 99.03% |
| SE-ResNet50 | NT-Xent | FC | 72.79% | 98.56% |
| SE-ResNet50 | NT-Xent | Self-Attention | 74.17% | 99.25% |
| SE-ResNet50 | Contrastive Loss | FC | 73.11% | 98.93% |
| SE-ResNet50 | Contrastive Loss | Self-Attention | 75.07% | 99.35% |
From the results, we observe that SE-ResNet50 consistently outperforms ResNet50 by 1% to 3% in Top-1 accuracy under otherwise identical settings. This demonstrates the effectiveness of the SE attention mechanism. Moreover, replacing NT-Xent with Contrastive Loss improves accuracy in all cases, especially when the batch size is limited to 32. The global self-attention classifier adds another 2% improvement over the plain FC classifier. These findings confirm the viability of the proposed modifications on a general benchmark dataset.
5. Solar Panel Defect Classification
5.1 Dataset Description
To further evaluate the method on the actual application, we use a solar panel defect dataset containing 2,624 grayscale images of size \(300 \times 300\) pixels collected from 44 different photovoltaic modules. The dataset includes 20 common defect categories, such as short circuit, black spot, scratch, pulse crackle, over welding, light and shade, damaged, cryptic fissure, bright spot, lobes, stain, blackhorn, broken grid, chipping, fluff wires, faulty soldering, dislocation, cut angle, black border, and mismatch. Representative images of these categories were used in the experiments. Due to copyright and clarity considerations, the detailed examples are not reproduced here, but the dataset provides a challenging benchmark for defect classification.
5.2 Data Augmentation
Data augmentation is an essential part of contrastive learning. For solar panels, simple transformations such as horizontal flips, vertical flips, random rotation, random cropping, scaling, brightness and contrast adjustment, and additive noise are appropriate. These augmentations simulate various capture conditions and improve robustness. However, aggressive transformations such as random erasing, translation, or strong geometric distortion can remove or modify crucial defect regions, potentially misleading the model. Therefore, we exclude these harmful augmentations and use only the ones that preserve the semantic content of a defect. An example of augmented solar panel images is shown below.
5.3 Implementation Details
The algorithm consists of two stages. In the pretraining stage, SE-ResNet50 is trained with Contrastive Loss on the entire unlabeled dataset. The training parameters are listed in the table below.
| Parameter | Value |
|---|---|
| Pretraining epochs | 800 |
| Batch size | 32 |
| MLP output dimension | 128 |
| Learning rate | 0.0003 |
After pretraining, the MLP projection head is removed. The remaining parameters of the backbone are frozen. A classifier consisting of global average pooling, a global self-attention block, and a fully connected output layer is appended. The classifier is trained on a small labeled subset. For this experiment, we randomly select 260 defect images (10% of the dataset) and manually label all 20 categories, with 13 images per category. The classifier training uses the following settings.
| Parameter | Value |
|---|---|
| Training epochs | 150 |
| Batch size | 32 |
| Number of classes | 20 |
| Learning rate | 0.0003 |
5.4 Pretraining Stage Analysis
To evaluate the contributions of the backbone and the loss function, we compare four combinations: ResNet50 with NT-Xent, SE-ResNet50 with NT-Xent, ResNet50 with Contrastive Loss, and SE-ResNet50 with Contrastive Loss. The training curves are analyzed in terms of convergence speed and stability. The original combination (ResNet50 + NT-Xent) shows poor convergence: the loss remains around 0.5 for a long time and experiences large oscillations. With SE-ResNet50, the training becomes more stable and the accuracy increases steadily. The observed instability of NT-Xent is mainly due to the small batch size, which limits the number of negative samples. By contrast, Contrastive Loss enables stable convergence from the early epochs and maintains a smooth loss curve. The combination of SE-ResNet50 and Contrastive Loss delivers the fastest and most stable convergence.
5.5 Classifier Training Stage Analysis
After pretraining, the classifier is trained on the labeled subset. We evaluate every combination of backbone, loss function, and classifier. The results are summarized below.
| Backbone | Loss Function | Classifier | Accuracy |
|---|---|---|---|
| ResNet50 | NT-Xent | FC | 67.15% |
| ResNet50 | NT-Xent | Self-Attention | 70.39% |
| ResNet50 | Contrastive Loss | FC | 72.41% |
| ResNet50 | Contrastive Loss | Self-Attention | 73.95% |
| SE-ResNet50 | NT-Xent | FC | 70.25% |
| SE-ResNet50 | NT-Xent | Self-Attention | 73.23% |
| SE-ResNet50 | Contrastive Loss | FC | 72.85% |
| SE-ResNet50 | Contrastive Loss | Self-Attention | 74.80% |
These results reveal several important observations. First, the self-attention classifier improved accuracy by about 2% to 3% in every comparable setting. Second, switching to Contrastive Loss provided a larger gain than switching the backbone, especially when only a small batch size is available. Third, SE-ResNet50 yielded a moderate but consistent advantage over the standard ResNet50, demonstrating that channel attention helps capture defect-related patterns.
We also measure the model size and training time for the two classifier variants on the solar panel defect dataset. The model attributes are as follows.
| Backbone | Classifier | Model Size | Training Time |
|---|---|---|---|
| SE-ResNet50 | FC | 99.67 MB | 53 min |
| SE-ResNet50 | Self-Attention | 100.31 MB | 57 min |
The global self-attention mechanism increases the model size by less than 1 MB and the training time by only about 4 minutes (less than 2 seconds per epoch). Such a small overhead is acceptable given the noticeable improvement in classification performance.
6. Conclusion and Outlook
In this paper, we addressed the problem of solar panel defect classification using an improved instance-level contrastive learning framework. We made three main modifications to the original contrastive learning pipeline. First, we used SE-ResNet50 as the backbone network, which applies channel-wise attention to enhance the representational capacity. Second, we replaced the conventional NT-Xent loss with Contrastive Loss to better exploit limited data and stabilize training. Third, we introduced a global self-attention mechanism into the classifier to capture long-range dependencies in the feature map. Experiments on both the STL10 benchmark and a solar panel defect dataset confirmed the effectiveness of each improvement. The best configuration achieved 74.80% accuracy on the solar panel defect dataset with a batch size of only 32, demonstrating the practical feasibility of self-supervised contrastive learning in industrial defect classification.
The challenges and future directions are as follows. Solar panel defects are highly diverse, including variations in shape, size, and severity. Building a model that covers all possible defect types remains difficult. In particular, for rare or novel defects, sufficient positive pairs may not be available for contrastive learning. Another challenge is the construction of hard negative pairs. In instance-level contrastive learning, two different types of cracks on a solar panel are considered negative pairs, even though they share similar visual structures. Such unintended negative pairs may hurt the downstream classification. Future work should investigate more sophisticated sampling strategies to select semantically valid negatives. Although contrastive learning reduces the dependence on annotations, a small labeled subset is still necessary for fine-tuning the classifier. Reducing this dependence further remains an open problem. Finally, deploying the model in real-time solar panel inspection systems requires addressing issues of latency, stability, and hardware integration. The current study is a step toward applying self-supervised methods to industrial scenarios, but more engineering efforts are needed before full practical deployment.
In summary, the improved contrastive learning method presented here offers a practical and effective solution for solar panel defect classification. The combination of SE-ResNet50, Contrastive Loss, and a global self-attention classifier provides a promising direction for leveraging unlabeled industrial data. We hope that this work encourages further research into self-supervised approaches for photovoltaic system maintenance and beyond.