腾讯 Hy4-preview（770B MoE，每层 256 个路由专家）通过 MIX-STQ1_0 混合精度量化方案，从 1.5TB 压缩至约 200GiB GGUF。该方法不是统一使用低比特，而是由校准数据逐层分配位宽，相对 BF16 精度几乎不变。
RT @jenzhuscott: The interesting bit isn’t “1.5TB → ~200GiB.” It’s that they refused uniform low-bit & treated bit-width as an allocation problem. Hy4-preview is a 770B MoE with 256 routed experts per layer. Almost all of the mass sits in the routed gate/up projections. MIX-STQ1_0 spends 1.3125 bpw (STQ1_0: ternary {-d, 0, +d} with forced 3:4 sparsity) on 29 of those layers and 2.06 bpw IQ2_XXS on the rest, then raises precision on down experts because they write straight into the residual. Calibration/imatrix decides the split. Same budget, lower error. Result vs BF16: •MCP Atlas 83.7 → 83.2 •SWE-Bench multi 82.9 → 81.3 •MRCR 81.3 → 81.1 •IFBench 73.5 → 72.5 That’s a ~7× weight cut on a frontier 49B-active / 1M-context model w almost no collapse on agent & coding evals. The next bottleneck is no longer “can we store it,” it’s whether 1.31-bit experts still hold recall past a few hundred thousand tokens. 🤯🤯
We compressed Hy4-preview from 1.5TB to ～200GiB GGUF and it still works well ! Meet MIX-STQ1_0.The trick isn’t just going low, it’s deciding where: calibration data picks each layer’s bit-width, some down to 1.31-bit STQ1_0, some up to 2.06-bit IQ2_XXS. Same budget, lower error. Accuracy barely moves vs BF16 📊 MCP Atlas 83.7→83.2 📊 SWE-Bench multi 82.9→81.3 📊 MRCR 81.3→81.1 📊 IFBench 73.5→72.5 See the details on HF : AngelSlim/Hy4-preview-GGUF Weights & low-bit GGUFs 👇 https://t.co/9uM9NT9Wem #LLM #Quantization #llamacpp #Hy
