Structured multi-track accompaniment arrangement via style prior modelling
Zhao, Jingwei Xia, Gus Wang, Ziyu Wang, Ye
Zhao, Jingwei
Xia, Gus
Wang, Ziyu
Wang, Ye
Author
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
2024
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
In the realm of music AI, arranging rich and structured multi-track accompaniments from a simple lead sheet presents significant challenges. Such challenges include maintaining track cohesion, ensuring long-term coherence, and optimizing computational efficiency. In this paper, we introduce a novel system that leverages prior modelling over disentangled style factors to address these challenges. Our method presents a two-stage process: initially, a piano arrangement is derived from the lead sheet by retrieving piano texture styles; subsequently, a multi-track orchestration is generated by infusing orchestral function styles into the piano arrangement. Our key design is the use of vector quantization and a unique multi-stream Transformer to model the long-term flow of the orchestration style, which enables flexible, controllable, and structured music generation. Experiments show that by factorizing the arrangement task into interpretable sub-stages, our approach enhances generative capacity while improving efficiency. Additionally, our system supports a variety of music genres and provides style control at different composition hierarchies. We further show that our system achieves superior coherence, structure, and overall arrangement quality compared to existing baselines.
Citation
J. Zhao, G. Xia, Z. Wang, and Y. Wang, “Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling,” Adv Neural Inf Process Syst, vol. 37, pp. 102197–102228, Dec. 2024, [Online]. Available: https://zhaojw1998.github.io/structured-arrangement/
Source
Advances in Neural Information Processing Systems (NeurIPS 2024)
Conference
Keywords
Music AI?, Multi-track accompaniment?, Style prior modeling?, Vector quantization?, Multi-stream Transformer?, Music generation
Subjects
Source
Publisher
NEURIPS