In modern energy systems, the lithium-ion battery has become a cornerstone due to its high energy density, low self-discharge rate, long cycle life, and environmental friendliness. It is widely employed in electric vehicles, aerospace, consumer electronics, and communication systems. However, as the number of charge-discharge cycles increases, the capacity of a lithium-ion battery gradually degrades, often exhibiting complex nonlinear behaviors, including capacity regeneration phenomena. Additionally, data acquisition processes are inevitably contaminated by noise, which compromises data quality. Accurate prediction of the remaining useful life (RUL) of a lithium-ion battery is crucial for battery health management, enabling proactive maintenance, avoiding safety hazards, and optimizing operational strategies. Traditional model-based methods rely on electrochemical mechanisms but suffer from parameter sensitivity and adaptability issues. In contrast, data-driven approaches, leveraging machine learning and deep learning, offer greater flexibility and generalization. Yet, challenges persist, such as handling long-term dependencies, mitigating noise and capacity rebound effects, and managing datasets of varying lengths. To address these, I propose a novel RUL prediction model integrating Pyramid Convolution (PyConv) with Transformer networks, enhanced by a weighted Huber loss function and Dropout techniques. This model effectively extracts multi-scale features from capacity sequences, captures long-range temporal dependencies, and improves robustness against outliers and overfitting.
The performance degradation of a lithium-ion battery is typically monitored through health indicators, with capacity being a direct and reliable metric. The RUL is defined as the remaining number of cycles before the battery capacity falls below a failure threshold. For a starting cycle \(T\) and end-of-life cycle \(T_{\text{EOL}}\), the RUL is calculated as:
$$ T_{\text{RUL}} = T_{\text{EOL}} – T $$
Accurate RUL prediction for lithium-ion batteries requires modeling capacity fade trends while accounting for anomalies like capacity regeneration. Common data-driven models, such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), struggle with long sequences due to gradient vanishing or explosion issues. Transformer networks, with self-attention mechanisms, excel at capturing global dependencies but may overlook local fluctuations. Convolutional Neural Networks (CNNs) are adept at local feature extraction but lack long-range modeling. Thus, combining PyConv for multi-scale feature extraction and Transformer for sequential modeling presents a promising solution for lithium-ion battery RUL prediction.
The PyConv network employs convolutional kernels of varying sizes in a pyramid structure to process input sequences. This allows simultaneous capture of local details and global trends in lithium-ion battery capacity data. For an input sequence \(x_I\), the PyConv output \(x_O\) is computed as:
$$ x_O = \sum_{i=1}^{n} \alpha_i (K_i * x_{I_i} + b_i) $$
where \(K_i\) represents convolutional kernels of different sizes, \(*\) denotes convolution, \(\alpha_i\) are weight parameters, and \(b_i\) are bias terms. The pyramid design uses smaller kernels at lower levels to capture short-term fluctuations and larger kernels at higher levels for long-term degradation patterns, effectively suppressing noise and capacity rebound effects in lithium-ion battery data.
The Transformer network leverages self-attention to model temporal dependencies without recurrent connections. To incorporate positional information, sinusoidal position encoding is applied:
$$ P(t,2i) = \sin\left(\frac{t}{10000^{2i/d}}\right) $$
$$ P(t,2i+1) = \cos\left(\frac{t}{10000^{2i/d}}\right) $$
where \(t\) is the position index, \(i\) is the dimension index, and \(d\) is the embedding dimension. The multi-head attention mechanism enables the model to focus on different parts of the sequence in parallel subspaces. For query \(Q\), key \(K\), and value \(V\) matrices derived from input \(X\):
$$ Q = X W^Q, \quad K = X W^K, \quad V = X W^V $$
the attention function is:
$$ \text{Attention}(Q,K,V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V $$
where \(d_k = d_{\text{model}} / n_{\text{head}}\). Multi-head attention combines multiple heads:
$$ \text{MultiHead}(Q,K,V) = \text{Concat}(\text{head}_1, \ldots, \text{head}_h) W^O $$
$$ \text{head}_i = \text{Attention}(QW_i^Q, KW_i^K, VW_i^V) $$
This allows the Transformer to learn complex temporal patterns in lithium-ion battery capacity sequences. The feedforward neural network (FFN) in each encoder layer applies nonlinear transformations:
$$ H_1 = \text{ReLU}(L W_1 + b_1) $$
$$ H_2 = H_1 W_2 + b_2 $$
$$ H_{\text{output}} = \text{LayerNorm}(L + H_2) $$
where \(L\) is the output from multi-head attention after residual connection and normalization. This enhanced Transformer structure reduces parameters and accelerates training for lithium-ion battery RUL prediction.
To handle imbalanced data and emphasize critical degradation phases in lithium-ion battery cycles, a weighted Huber loss function is adopted. It blends squared and absolute errors to enhance robustness:
$$ L = \frac{1}{N} \sum_{i=1}^{N} w_i \begin{cases} 0.5(y_i – \hat{y}_i)^2 & \text{if } |y_i – \hat{y}_i| \le \delta \\ \delta |y_i – \hat{y}_i| – 0.5\delta^2 & \text{if } |y_i – \hat{y}_i| > \delta \end{cases} $$
where \(y_i\) and \(\hat{y}_i\) are actual and predicted values, \(w_i\) are weights assigned to later cycles to prioritize degradation trends, \(\delta\) is a threshold parameter, and \(N\) is the sample count. This loss function mitigates the impact of outliers and noise common in lithium-ion battery datasets.
Dropout technology is integrated into the FFN layers to prevent overfitting. During training, neurons are randomly deactivated with probability \(p\), updating activation states as:
$$ A_j(t+1) = \begin{cases} 0 & \text{if } p \ge \lambda \\ \frac{x}{1-p} & \text{else} \end{cases} $$
where \(x\) is neuron input and \(\lambda\) is a probability threshold. This regularization improves generalization for lithium-ion battery RUL prediction across diverse conditions.
The overall PyConv-Transformer framework for lithium-ion battery RUL prediction involves several steps: data preprocessing, feature extraction, sequence modeling, and output prediction. Capacity sequences are normalized and segmented via sliding windows. PyConv extracts multi-scale features, which are fed into the Transformer with positional encoding. The model is trained using the weighted Huber loss, with Dropout applied in FFN layers. Hyperparameters are optimized through iterative validation.
Experiments are conducted on public lithium-ion battery datasets: NASA and CALCE. The NASA dataset includes batteries B0005, B0006, B0007, and B0018, cycled at room temperature with specific charge-discharge protocols. The CALCE dataset uses CS2_35 battery samples rated at 1100 mAh, tested under controlled conditions. Failure thresholds are set at 70% of nominal capacity. Data splits for training and testing are detailed in Table 1.
| Dataset | Battery IDs | Nominal Capacity | Failure Threshold | Training Set | Testing Set |
|---|---|---|---|---|---|
| NASA | B0005, B0006, B0007, B0018 | 2.0 Ah | 1.4 Ah (1.45 Ah for B0007) | 3 batteries | 1 battery |
| CALCE | CS2_35 | 1.1 Ah | 0.77 Ah | First 60% cycles | Last 40% cycles |
The experimental environment uses Python 3.9, PyTorch 1.12, NVIDIA RTX4060 GPU, and an i7 processor. Model parameters are tuned: input sequence length \(T\) and hidden size \(h\) are set to 16 and 16 for NASA, 64 and 32 for CALCE; learning rate \(lr = 0.01\); PyConv kernel sizes \(P_l = (3,3), (5,5), (7,7)\); weighted loss threshold \(\delta = 0.5\); Dropout probability \(p = 0.2\). Evaluation metrics include Relative Error (RE), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE):
$$ \text{RE} = \frac{|\hat{y}_t – y_t|}{y_t} $$
$$ \text{MAE} = \frac{1}{n} \sum_{t=1}^{n} |y_t – \hat{y}_t| $$
$$ \text{RMSE} = \sqrt{\frac{1}{n} \sum_{t=1}^{n} (y_t – \hat{y}_t)^2} $$
where \(\hat{y}_t\) and \(y_t\) are predicted and actual RUL values, and \(n\) is the number of samples. Lower values indicate better prediction accuracy for lithium-ion battery RUL.
Results on the NASA dataset demonstrate the effectiveness of the PyConv-Transformer model. Prediction curves for batteries B0005, B0006, B0007, and B0018 show close alignment with actual RUL, successfully mitigating capacity regeneration effects. Comparative analysis with LSTM, RNN, and CNN-Transformer models reveals superior performance, as summarized in Table 2.
| Model | RE | MAE | RMSE |
|---|---|---|---|
| LSTM | 0.0183 | 0.0254 | 0.0285 |
| RNN | 0.0195 | 0.0254 | 0.0332 |
| CNN-Transformer | 0.0160 | 0.0210 | 0.0280 |
| PyConv-Transformer | 0.0086 | 0.0115 | 0.0173 |
The PyConv-Transformer achieves the lowest errors, with RE, MAE, and RMSE reduced by approximately 0.0097, 0.0139, and 0.0107 respectively compared to CNN-Transformer. This underscores its capability to handle short sequences and local fluctuations in lithium-ion battery data.
On the CALCE dataset, which features longer sequences and steeper degradation slopes, the PyConv-Transformer model maintains high accuracy. Prediction curves for CS2_35 battery exhibit stable trends without succumbing to capacity rebound, closely matching true RUL. Performance metrics are presented in Table 3.
| Model | RE | MAE | RMSE |
|---|---|---|---|
| LSTM | 0.0289 | 0.0218 | 0.0285 |
| RNN | 0.0362 | 0.0405 | 0.0364 |
| CNN-Transformer | 0.0339 | 0.0203 | 0.0286 |
| PyConv-Transformer | 0.0193 | 0.0126 | 0.0189 |
The PyConv-Transformer model outperforms others, with RE, MAE, and RMSE lower by 0.0096, 0.0092, and 0.0097 respectively compared to LSTM. This highlights its robustness in managing complex, long-sequence data from lithium-ion batteries.
The superior performance of the PyConv-Transformer model for lithium-ion battery RUL prediction stems from several factors. The PyConv network’s multi-scale convolution kernels effectively extract both local and global features, reducing noise and capacity regeneration interference. The Transformer’s self-attention mechanism captures long-term dependencies without gradient issues, crucial for modeling capacity fade trends. The weighted Huber loss function prioritizes later degradation stages, enhancing sensitivity to critical phases in lithium-ion battery cycles. Dropout regularization prevents overfitting, ensuring generalization across datasets. Compared to standalone models, this integration offers a balanced approach: PyConv addresses local variations, while Transformer handles sequential patterns, making it ideal for lithium-ion battery applications.
However, limitations exist. The model requires careful hyperparameter tuning, and computational complexity may increase with larger kernel sizes in PyConv. Future work could explore adaptive kernel selection, integration of additional health indicators like impedance, and real-time deployment in battery management systems for lithium-ion batteries. Transfer learning across different battery chemistries and operating conditions also warrants investigation.
In conclusion, the PyConv-Transformer model presents a robust data-driven solution for lithium-ion battery remaining useful life prediction. By combining multi-scale feature extraction with advanced sequence modeling, it effectively addresses challenges such as capacity regeneration, noise, and varying data lengths. Experimental validation on NASA and CALCE datasets demonstrates significant improvements in accuracy, with reduced relative error, mean absolute error, and root mean square error compared to existing methods. This advancement contributes to reliable health management of lithium-ion batteries, supporting safer and more efficient energy storage systems. Continued refinement and application of this approach will further enhance predictive maintenance strategies for lithium-ion battery technologies.