DDN added new capabilities to the Lustre platform it manages with Google Cloud, including means to share key-value (KV) cache to boost AI inference workloads. Unveiled at Google’s annual Next event, ...
VerTQ is an accelerator chip that implements Google's TurboQuant algorithm which reduces KV cache memory usage of Large ...
A new technical paper titled “Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System” was published by researchers at Rensselaer Polytechnic Institute and IBM. “Large ...
SwiftKV optimizations developed and integrated into vLLM can improve LLM inference throughput by up to 50%, the company said. Cloud-based data warehouse company Snowflake has open-sourced a new ...
Google researchers have published a new quantization technique called TurboQuant that compresses the key-value (KV) cache in large language models to 3.5 bits per channel, cutting memory consumption ...
Remote-First-Company | NEW YORK CITY, Jan. 05, 2026 (GLOBE NEWSWIRE) -- VAST Data, the AI Operating System company, today announced a new inference architecture that enables the NVIDIA Inference ...
Lightbits Labs Ltd. today is introducing a new architecture aimed at addressing one of the most stubborn bottlenecks in large-scale artificial intelligence inference: the growing mismatch between the ...
FREMONT, Calif.--(BUSINESS WIRE)--Penguin Solutions, Inc. (Nasdaq: PENG), the AI factory platform company, today announced the industry's first production-ready KV cache server that utilizes CXL ...
Generative AI is arguably the most complex application that humankind has ever created, and the math behind it is incredibly complex even if the results are simple enough to understand. GenAI also it ...
With the AI infrastructure push reaching staggering proportions, there’s more pressure than ever to squeeze as much inference as possible out of the GPUs they have. And for researchers with expertise ...
Some results have been hidden because they may be inaccessible to you
Show inaccessible results