Tencent's Hy Memory Shrinks Local Footprint from 60GB to 18GB with llama.cpp

Key Info

Tencent announced a smaller local deployment option for its open-source Hy Memory stack, cutting the footprint from 60GB to 18GB by moving from vLLM to a single llama.cpp binary while keeping the memory bank on the user's own disk.

Highlights

• Local size reduced from 60GB to 18GB.
• Runtime stack simplified from vLLM to one llama.cpp binary.
• Quality on Tencent's LoCoMo eval: 58.8% for Q6_K and 56.5% for Q4_K_M.
• Memory data remains stored locally on the user's disk.
• Model weights are available on Hugging Face.