A shorter, crisper manuscript if you want the idea in a more compact form.
The paper is built around a precise cache-compression story with explicit maths and implementation detail.
A shorter, crisper manuscript if you want the idea in a more compact form.
Why the paper is not just a narrative, but a system design with structure.
The cache footprint grows as MKV = 2 · B · L · H · T · Dh · b. The paper uses this to motivate a precision allocator rather than naive uniform compression.
The relaxed allocator gives more precision to more important tokens and smaller budgets to tokens that are older, less sensitive, or less structurally important.
The runtime wrapper decompresses incoming cache state, runs the model, then compresses the returned cache with metadata preserved for exact reconstruction.
Open the PDF inline without leaving the page.
Alternate manuscript versions of the same project.