Loading...
Mitigating Data Redundancy to Revitalize Transformer-based Long-Term Time Series Forecasting System
Li, Mingjie Liu, Rui Shi, Guangsi Han, Mingfei Li, Changlin Yao, Lina Chang, Xiaojun Chen, Ling
Li, Mingjie
Liu, Rui
Shi, Guangsi
Han, Mingfei
Li, Changlin
Yao, Lina
Chang, Xiaojun
Chen, Ling
Files
Loading...
3735651.pdf
Adobe PDF, 73.83 MB
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
License
http://rightsstatements.org/page/InC/1.0/
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Long-term time series forecasting (LTSF) is fundamental to various real-world applications, where Transformer-based models have become the dominant framework due to their ability to capture long-range dependencies. However, these models often experience overfitting due to data redundancy in rolling forecasting settings, limiting their generalization ability particularly evident in longer sequences with highly similar adjacent data. In this work, we introduce CLMFormer, a novel framework that mitigates redundancy through curriculum learning and a memory-driven decoder. Specifically, we progressively introduce Bernoulli noise to the training samples, which effectively breaks the high similarity between adjacent data points. This curriculum-driven noise introduction aids the memory-driven decoder by supplying more diverse and representative training data, enhancing the decoder’s ability to model seasonal tendencies and dependencies in the time series data. To further enhance forecasting accuracy, we introduce a memory-driven decoder. This component enables the model to capture seasonal tendencies and dependencies in the time series data and leverages temporal relationships to facilitate the forecasting process. Extensive experiments on six real-world LTSF benchmarks show that CLMFormer consistently improves Transformer-based models by up to 30%, demonstrating its effectiveness in long-horizon forecasting.
Citation
M. Li, R. Liu, G. Shi, M. Han, C. Li, L. Yao , et al., "Mitigating Data Redundancy to Revitalize Transformer-based Long-Term Time Series Forecasting System," ACM Transactions on Intelligent Systems and Technology, vol. 17, no. 3, pp. 1-21, 2026, https://doi.org/10.1145/3735651.
Source
ACM Transactions on Intelligent Systems and Technology
Conference
Keywords
46 Information and Computing Sciences, 4611 Machine Learning
Subjects
Source
Publisher
Association for Computing Machinery