Pegasus logo
Aman Sachan papers • projects • research index
Research paper

KVQuant: Adaptive Long-Context KV-Cache Compression for Memory-Bounded LLM Inference

The most complete version: rate-distortion framing, sink-aware allocation, head/layer coupling, benchmark plan, failure analysis, and a full appendixed derivation.

Open PDF Source file All papers Home

Technical synopsis

The paper is built around a precise cache-compression story with explicit maths and implementation detail.

The most complete version: rate-distortion framing, sink-aware allocation, head/layer coupling, benchmark plan, failure analysis, and a full appendixed derivation.

  • Adaptive mixed-precision cache compression
  • Recency-aware bit allocation
  • Sink-aware reservation for stable anchors
  • Head/layer coupling for finer precision control
  • Explicit metadata for invertibility and auditing
  • Forward-pass wrapping for drop-in use

Math + algorithm focus

Why the paper is not just a narrative, but a system design with structure.

Core equation

The cache footprint grows as MKV = 2 · B · L · H · T · Dh · b. The paper uses this to motivate a precision allocator rather than naive uniform compression.

Allocation logic

The relaxed allocator gives more precision to more important tokens and smaller budgets to tokens that are older, less sensitive, or less structurally important.

Implementation contract

The runtime wrapper decompresses incoming cache state, runs the model, then compresses the returned cache with metadata preserved for exact reconstruction.

Preview

Open the PDF inline without leaving the page.

Related papers

Alternate manuscript versions of the same project.

Paper research draft

research draft

technical report + public demo

Open page PDF
Paper v2 — extended draft

v2 — extended draft

research manuscript

Open page PDF
Paper deep draft

deep draft

technical report

Open page PDF