Pixel-Level Tiling with End-to-End Neural Compression for Tile-Based Video Streaming
Jin, Yili Zhang, Dayou Zhu, Hao Wang, Fangxin Liu, Jiangchuan Liu, Steve Xue
Jin, Yili
Zhang, Dayou
Zhu, Hao
Wang, Fangxin
Liu, Jiangchuan
Liu, Steve Xue
Supervisor
Department
Machine Learning
Embargo End Date
Type
Journal article
Date
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Multimedia data has become the dominant traffic through the mobile network. In recent years, the forms of videos become more immersive and interactive, raising significant challenges for video streaming due to the limited network resources. Since a viewer’s viewport is limited, video frame partition and tile-based streaming stand out as a feasible solution. Existing works all follow the Tiling-on-Frame (ToF) strategy, i.e., partition each frame into tiles before encoding. However, there is a challenging trade-off. Coarse tiling leads to redundant content transmission, while fine tiling reduces compression ratio due to loss of spatial information. ToF inevitably struggles to balance the tiling granularity and compression efficiency. To tackle this challenge, we introduce a pixel-level framework called Tiling-on-Bitstream (ToB). It eliminates the redundant contents and achieves high compression efficiency. The key idea is that, instead of ToF, we first encode the entire video into bitstreams and then select the corresponding bitstream as requested for streaming; and at the receiver side, the bitstream is decoded to retrieve the requested video content. Such ToB framework enables a fine-grained partition at pixel level, avoiding redundancy in boundary. To efficiently implement a prototype of ToB, we design Neural-EGDL, a neural network-based codec algorithm. To demonstrate the power of our design, we also apply it to 360° videos.
Citation
Y. Jin, D. Zhang, H. Zhu, F. Wang, J. Liu, S.X. Liu, "Pixel-Level Tiling with End-to-End Neural Compression for Tile-Based Video Streaming," IEEE Transactions on Circuits and Systems for Video Technology, vol. PP, no. 99, pp. 1-1, 2026, https://doi.org/10.1109/tcsvt.2026.3690578.
Source
IEEE Transactions on Circuits and Systems for Video Technology
Conference
Keywords
40 Engineering, 4008 Electrical Engineering, 46 Information and Computing Sciences, 4603 Computer Vision and Multimedia Computation
Subjects
Source
Publisher
IEEE