Google Research just published a new compression algorithm called TurboQuant, created by research scientist Amir Zandieh and VP and Google Fellow Vahab Mirrokni, that reduces the key-value cache memory of large language models by at least 6x, delivers up to 8x speed improvements, and does all of this without any loss in accuracy whatsoever.
If you’ve spent any time following AI news lately, you already know that one of the biggest problems with large language models isn’t just how smart they are, it’s how expensive and slow they can get when handling long, complex tasks.
Every time a model processes text, it stores previously computed data in something called a key-value cache, a high-speed memory layer that lets the model retrieve information quickly instead of recalculating everything from scratch.
It sounds efficient, and it is, until the model gets big enough that the cache itself becomes the bottleneck. At that point, memory costs skyrocket and performance takes a hit regardless of how powerful your hardware is.
Introducing TurboQuant: Our new compression algorithm that reduces LLM key-value cache memory by at least 6x and delivers up to 8x speedup, all with zero accuracy loss, redefining AI efficiency. Read the blog to learn how it achieves these results: https://t.co/CDSQ8HpZoc pic.twitter.com/9SJeMqCMlN
— Google Research (@GoogleResearch) March 24, 2026
That’s the exact problem TurboQuant was designed to solve. The algorithm uses a technique called vector quantization, which compresses data by replacing large, high-precision numbers with smaller, more compact ones. Vectors are the fundamental building blocks AI models use to understand information, they encode everything from the meaning of a sentence to the features of an image.
The problem with most existing compression methods is that they introduce their own memory overhead in the process, needing extra bits just to keep track of how the compression was done in the first place, which partially cancels out the savings. TurboQuant eliminates that overhead completely.
A two-step process that changes the compression game
TurboQuant doesn’t work alone. Under the hood it relies on two other algorithms that Google developed as part of the same research effort: PolarQuant and QJL, short for Quantized Johnson-Lindenstrauss.
PolarQuant handles the heavy lifting. It starts by randomly rotating the data vectors to simplify their geometry, then converts them from standard coordinates into polar coordinates, a shift that maps the data onto a fixed, predictable circular grid where the boundaries are already known.
Because of that predictability, the model no longer needs to run expensive normalization calculations, and PolarQuant can compress the data at high quality while carrying zero memory overhead. Most of the available bits go toward capturing the core meaning and strength of the original vector, nothing wasted.
Then QJL steps in to clean up what’s left. It takes the small residual error remaining after PolarQuant finishes and compresses it down to a single sign bit per value, either +1 or -1. One bit. Zero additional memory cost. It acts as a mathematical error-corrector that keeps the model’s attention scores accurate, which is how the model decides which parts of its input actually matter when generating a response.
Put both stages together and TurboQuant compresses the key-value cache down to just 3 bits per value, with no training required, no fine-tuning, and no accuracy penalty. According to Google, it also runs faster than the original uncompressed versions of the models it was tested on.
The tests were tough, and TurboQuant passed all of them
Google ran TurboQuant through five demanding long-context benchmarks: LongBench, Needle In A Haystack, ZeroSCROLLS, RULER, and L-Eval, using open-source models Gemma and Mistral. The tasks covered question answering, code generation, and summarization. The Needle In A Haystack test is particularly brutal, it asks the model to find one specific piece of information buried inside a massive block of text, the kind of test where compressed models usually start falling apart.
TurboQuant passed all of them with perfect downstream results, with the key-value cache memory reduced by at least 6x across the board. PolarQuant also performed nearly lossless on these same tasks independently, which says a lot about the quality of both components on their own.
The speed results are just as impressive. Running 4-bit TurboQuant on NVIDIA H100 GPU accelerators, attention computation ran up to 8x faster compared to the standard 32-bit unquantized baseline. No hardware upgrade, no architectural change, just the algorithm doing its job.
TurboQuant is also data-oblivious, meaning it doesn’t need to be retrained or adjusted for specific models or datasets. It works out of the box, which is a massive practical advantage when deploying AI at scale.
Google also put TurboQuant to work on high-dimensional vector search, the technology behind semantic search engines that understand the meaning of a query rather than just matching keywords. Against established baselines like PQ and RabbiQ, TurboQuant achieved superior recall accuracy without relying on large codebooks or dataset-specific tuning that those competing methods depend on. For a company running search across billions of vectors, that combination of efficiency and precision is a very big deal.
What separates TurboQuant, PolarQuant, and QJL from just being smart engineering tricks is that all three are backed by rigorous mathematical proofs and operate near theoretical lower bounds, meaning they’re provably close to the best results that math says are even possible.
The research was carried out in collaboration with Praneeth Kacham, Lars Gottesbüren, and Rajesh Jayaram from Google, Majid Hadian from Google DeepMind, Insu Han from KAIST, and Majid Daliri from NYU. TurboQuant is set to be presented at ICLR 2026 and PolarQuant at AISTATS 2026.
Google’s researchers point directly to Gemini as one of the main models that stands to benefit from solving the key-value cache bottleneck. And as AI keeps growing and gets embedded into more products, from assistants to real-time applications to search at massive scale, the pressure on memory and compute is only going to increase.
TurboQuant is exactly the kind of foundational work that makes the next generation of AI not just possible, but actually affordable and fast enough to matter.
What do you think about TurboQuant, is this the kind of breakthrough AI infrastructure has been waiting for? Drop your take in the comments!