Semantics-Aware Patch Encoding and Hierarchical Dependency Modeling for Long-Term Time Series Forecasting
Peng, Sijia Xiong, Yun Zhu, Yangyong Shen, Zhiqiang
Peng, Sijia
Xiong, Yun
Zhu, Yangyong
Shen, Zhiqiang
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Time series forecasting is a vital task with widespread applications. While recent advancements have adopted patching to enrich short-term context, existing encoding methods often struggle to capture the diverse semantics within patches, resulting in semantic information loss and limited model performance. Moreover, most long-term dynamics modeling approaches rely on homogeneous architectures with fixed receptive fields, inevitably sacrificing either performance or efficiency. To address these challenges, we propose Mixture of Universals (MoU), a novel framework designed to prevent semantic loss during patch encoding and efficiently enhance long-term dynamics through a hybrid approach. Specifically, MoU is consist of two novel designs: Mixture of Feature Extractors (MoF) and Mixture of Architectures (MoA). MoF introduces a semantics-aware encoding mechanism that selectively activates the corresponding sub-extractor based on the semantic context of input patches, preserving diverse temporal patterns and mitigating information loss. MoA, on the other hand, hierarchically captures long-term dependency with progressively expanded receptive field, improving model performance while maintaining relatively low computational costs. We conducted extensive experiments on seven real-world datasets, and the results demonstrate the superiority of our model. Our Code is available at https://github.com/lunaaa95/mou/.
Citation
S. Peng, Y. Xiong, Y. Zhu, and Z. Shen, “Semantics-Aware Patch Encoding and Hierarchical Dependency Modeling for Long-Term Time Series Forecasting,” pp. 2269–2280, Aug. 2025, doi: 10.1145/3711896.3737123
Source
Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2
Conference
KDD '25: The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Keywords
Subjects
Source
KDD '25: The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Publisher
Association for Computing Machinery