Overview

This paper introduces Generative Video Compression (GVC), a novel approach to video compression that leverages generative AI models to achieve extreme compression rates approaching 0.01% while maintaining high perceptual quality. Developed by researchers at TeleAI (China Telecom’s AI research institute), GVC represents a fundamental departure from traditional video compression paradigms by trading computational resources for bandwidth efficiency.

Figure: The GVC framework processes input video through a neural encoder to generate compressed tokens, which are then reconstructed by a generative video decoder, enabling both high-quality perception reconstruction and downstream task completion.

Traditional Compression vs. Generative Approaches

Traditional video compression methods like HEVC operate under Shannon’s Level A communication paradigm, focusing on accurate pixel-level reconstruction of the original video signal. These approaches optimize for signal fidelity but can be inefficient when the end goal is human perception or machine understanding rather than perfect reconstruction.

GVC shifts to Level C communication - effectiveness-oriented transmission where the focus is on enabling the receiver to achieve a desired perceptual outcome or complete a specific task. This approach recognizes that in many modern applications, particularly those involving AI at the edge or in bandwidth-constrained environments, the utility of the video matters more than its exact reconstruction.

The key insight is that powerful generative models can synthesize high-quality, realistic video content from minimal inputs by leveraging learned priors about natural video statistics. This enables a dramatic reduction in the amount of information that needs to be transmitted.

Core Methodology

Framework Architecture

GVC consists of two primary components working in tandem:

Neural Encoder: The encoder processes input video sequences through pre-trained neural networks to extract extremely compact "compressed tokens." These tokens contain both discrete and continuous representations, including compressed keyframes, high-level video descriptors, and low-level features that capture essential semantic information and motion dynamics. The tokens are further compressed into minimal bitstreams using techniques like residual coding.

Generative Video Decoder: At the receiver, a pre-trained diffusion-based generative model reconstructs the full video sequence from the minimal compressed tokens. This reconstruction operates as a conditional video generation task, where the model uses its learned priors to synthesize visually faithful content guided by the received tokens.

Computation-for-Compression Trade-off

The fundamental innovation lies in GVC’s "computation-for-compression" trade-off. Instead of preserving every pixel detail during transmission, the encoder intelligently selects only the most task-relevant information. The heavy computational burden of video reconstruction shifts to the decoder, which leverages powerful generative models to synthesize complete video sequences from these minimal cues.

This approach can be understood mathematically as optimizing a different objective function. Where traditional compression minimizes:

Ltraditional=Dsignal(x,x^)+λRL_{traditional} = D_{signal}(x, \hat{x}) + \lambda R

where DsignalD_{signal} measures pixel-level distortion, GVC optimizes:

LGVC=Dperceptual(x,x^)+Dtask(x,x^)+λR+γCL_{GVC} = D_{perceptual}(x, \hat{x}) + D_{task}(x, \hat{x}) + \lambda R + \gamma C

where DperceptualD_{perceptual} captures perceptual quality, DtaskD_{task} measures task performance preservation, RR is the transmission rate, and CC represents computational cost at the receiver.

Balancing Practicality

While extreme compression is achievable through sophisticated generative models, real-world deployment requires managing computational demands. GVC introduces a secondary "compression-for-practicality" trade-off involving:

• Model Miniaturization: Applying compression techniques to reduce the size and complexity of decoder components
• Inference Acceleration: Using distillation and sampling acceleration methods to reduce decoding latency
• Adaptive Representation Richness: Transmitting slightly richer latent representations when computational resources are limited
• Quality Compensation: When model simplification affects quality, this is compensated by transmitting more informative features

Experimental Results and Performance

Compression and Perceptual Quality

The experimental validation demonstrates GVC’s effectiveness across multiple metrics. On the MCL-JCV benchmark dataset, GVC achieves compression rates as low as 0.02% (0.005 bits per pixel) while significantly outperforming traditional codecs in perceptual quality metrics.

At an average bitrate of 0.008 bpp, GVC achieves a Learned Perceptual Image Patch Similarity (LPIPS) score of 0.214, compared to HEVC’s 0.271 at the same rate. Lower LPIPS scores indicate better perceptual quality, demonstrating GVC’s superior performance in perception-oriented scenarios.

Visual comparisons show that GVC can require approximately six times less bandwidth than traditional methods to achieve equivalent perceptual reconstruction quality, highlighting its efficiency for perception-driven applications.

Downstream Task Performance

To validate semantic information preservation, the authors evaluated reconstructed videos on video object segmentation (VOS) using the DAVIS2017 dataset with the XMEM model. At 0.01 bpp, GVC substantially outperforms HEVC across all metrics:

• J&F (Average Jaccard & Contour Accuracy): 75.22% vs. 57.68%
• J (Jaccard Index): 71.17% vs. 56.84%
• F (Contour Accuracy): 79.28% vs. 58.51%
• F-Recall: 91.87% vs. 67.44%

These results confirm that GVC effectively preserves semantic content crucial for machine understanding tasks, even under severe bandwidth constraints.

Computational Efficiency

The practical deployment aspect was validated through latency measurements on consumer-grade hardware (NVIDIA GeForce RTX 4090). For generating a Group of Pictures (GOP) of 29 frames:

• 480p: ~2.3 seconds total (0.95s encoder + 1.35s decoder)
• 720p: ~7.55 seconds total (1.15s encoder + 6.4s decoder)
• 1080p: ~23 seconds total (1.59s encoder + 21.5s decoder)

The authors note these latencies are comparable to large language model response times, suggesting practical viability for many real-world scenarios, particularly for 480p and 720p resolutions.

Technical Innovations and Contributions

Paradigm Shift

GVC’s primary contribution is demonstrating how generative AI can fundamentally redefine video compression. By embracing task-oriented communication principles, it moves beyond pixel-perfect reconstruction to focus on preserving perceptual and semantic utility.

Extreme Compression Rates

The achievement of compression rates approaching 0.01% represents a significant advance over conventional methods. This is accomplished not through incremental improvements to existing algorithms, but through a fundamentally different approach that leverages the representational power of generative models.

Practical Deployment Framework

The research addresses the critical gap between theoretical compression capabilities and real-world deployment constraints. The "compression-for-practicality" trade-off provides a concrete framework for balancing compression efficiency with computational feasibility.

Integration with AI Infrastructure

As part of TeleAI’s broader AI Flow framework, GVC is designed to enable ubiquitous AI-powered services by providing ultra-efficient visual data transmission. This positions it not just as a compression technique but as an enabling technology for distributed AI systems.

Implications and Future Directions

GVC represents a strategic technological leap that could transform video communication across numerous domains. Its ability to maintain high perceptual quality and semantic preservation at ultra-low bitrates makes it particularly valuable for bandwidth-constrained environments such as emergency communications, maritime operations, remote surveillance, and mobile edge computing.

The framework’s emphasis on preserving information relevant to downstream AI tasks aligns with the growing importance of machine understanding in communication systems. This could enable new applications where video transmission is optimized not just for human viewing but for automated analysis and decision-making.

The demonstrated practicality on consumer hardware suggests potential for widespread adoption, while the modular design allows for continued optimization as generative models and hardware capabilities advance. Future research directions include optimizing encoder-decoder designs for specific applications, developing more efficient generative models tailored for compression, and exploring theoretical limits of perception-oriented communication.

The work establishes a foundation for reimagining communication systems in the era of generative AI, where the focus shifts from faithful data reproduction to intelligent content delivery that enables desired outcomes at the receiver.

Relevant Citations