CompACT achieves 40× faster planning by compressing images to 8 tokens using frozen semantic encoders.
World models provide a powerful framework for simulating environment dynamics conditioned on actions or instructions, enabling downstream tasks such as action planning or policy learning. Recent approaches leverage world models as learned simulators, but its application to decision-time planning remains computationally prohibitive for real-time control. A key bottleneck lies in latent representations: conventional tokenizers encode each observation into hundreds of tokens, making planning both slow and resource-intensive. To address this, we propose CompACT, a discrete tokenizer that compresses each observation into as few as 8 tokens, drastically reducing computational cost while preserving essential information for planning. An action-conditioned world model that occupies CompACT tokenizer achieves competitive planning performance with orders-of-magnitude faster planning, offering a practical step toward real-world deployment of world models.
CompACT compresses images to just 8 tokens while maintaining planning performance superior to 64-token competitors.
Using frozen pretrained vision encoders preserves semantic information while eliminating unnecessary perceptual details for planning.
Generative decoding synthesizes perceptual details from compact semantic tokens, achieving 40× speedup in planning latency.
Extreme compression creates information bottleneck; approach may not generalize to tasks requiring fine-grained visual details.
Method relies on specific pretrained foundation models; performance may vary with different encoder architectures or domains.
Jintao Zhang, Haoxu Wang +7
Sparse-Linear Attention (SLA) combines sparse and linear attention to accelerate diffusion models and has shown strong performance in video generation. However, (i) SLA relies on a heuristic split tha...
Jintao Zhang, Kai Jiang +6
Many training-free sparse attention methods are effective for accelerating diffusion models. Recently, several works suggest that making sparse attention trainable can further increase sparsity while ...
Yue Ding, Yiyan Ji +13
Omni-modal Large Language Models (Omni-LLMs) have demonstrated strong capabilities in audio-video understanding tasks. However, their reliance on long multimodal token sequences leads to substantial c...
Yongtong Wu, Shaoyuan Chen +11
The performance of multi-turn, agentic LLM inference is increasingly dominated by KV-Cache storage I/O rather than computation. In prevalent disaggregated architectures, loading the massive KV-Cache f...
Haocheng Xi, Shuo Yang +14
Despite rapid progress in autoregressive video diffusion, an emerging system algorithm bottleneck limits both deployability and generation capability: KV cache memory. In autoregressive video generati...
