Running a large language model is expensive, and a surprising amount of that cost comes down to memory, not computation. Every time a model like Gemini or GPT-4 processes a long document or sustains a ...