腾讯混元Hy4-preview从发布首日起即可在vLLM中运行，并已在NVIDIA GPU上验证。该模型采用770B总参数、49B激活参数的MoE架构，支持1M上下文，但每次查询仅需关注2048个token。
Hy4 preview runs in vLLM
@TencentHunyuan's Hy4-preview runs in vLLM from day 0, verified on NVIDIA GPUs. 🎉 - 770B total, 49B active, 256 routed experts plus one shared - 1M context, but each query attends to just 2048 tokens - Only 21 of the 78 layers compute their own sparse index, the other 57 reuse one - A 10B MTP layer ships inside the checkpoint, 0.7B of it active, draft depth 3 Tencent's HPC-Ops attention and MoE kernels have been in vLLM main since Hy3. VLLM_ENABLE_HPC_OPS=1 vllm serve tencent/Hy4-preview-FP8 -tp 8 Thanks @TencentHunyuan for the preview weights! 🙌 🔗 https://t.co/REAxUUfyZb
