Unofficial guide
Hy4 Flash requirements
Last updated
“Can I run it?” depends on which file you mean. The answers below are the hardware lines the vLLM recipe and AngelSlim actually publish. Where they do not give a number, this page does not fill one in.
Disk
| File | Disk |
|---|---|
| tencent/Hy4-flash (BF16 instruct) | 440.53 GB |
| tencent/Hy4-flash-FP8 | 287.09 GB |
| tencent/Hy4-flash-base | 438.84 GB |
| GGUF Q4_K_M pair | 156.79 GiB |
| GGUF Q3_K pair | 144.70 GiB |
| GGUF IQ2-AH pair | 125.73 GiB |
| SPARK-4Bit / SPARK-4Bit-Fast | 151.35 GiB / 148.65 GiB |
vLLM: GPUs and the VRAM hint
The recipe lists Hopper and Blackwell data-center GPUs: H100, H200, B200, GB200, B300, and GB300. The per-GPU memory figures on that page are the cards' capacities (80 GB, 141 GB, 180 GB, 192 GB, 268 GB, and 288 GB on the NVIDIA side). AMD Instinct is listed in the page chrome and then ruled out in the text: no ROCm image ships Hy4 Flash.
The same page shows variant hints of 529 GB for BF16 and 345 GB for FP8. The prose says those
are the checkpoint download sizes, 440.5 GB and 287.1 GB, times 1.2, and that they are not
measured minimums. Engram storage is offloaded to the CPU by default, which lowers the GPU
footprint and means you also need host RAM. The recipe does not say how many gigabytes of host
RAM. The published Docker examples set --tensor-parallel-size 8.
Context is a separate budget. The weights advertise 1,048,576 positions. The recipe's starting
command uses 8,192 tokens, four sequences, and one image, with video left at zero. Raising
--max-model-len is how you spend the rest of the memory, not something the sample
command does for you.
GGUF: the machines with published results
AngelSlim publishes speeds for Q4_K_M and Q3_K on a Mac M5 Max with 128 GB, and for the two SPARK-4Bit files on an RTX Spark with 128 GB. Their Mac README is more specific about memory, and it is about that patch, not a generic laptop:
- They set
iogpu.wired_limit_mb=114688, which is 112 GiB, and say the weights are about 97.4 GiB. The Q4_K_M trunk file in the repo is 97.78 GiB. - They say to keep the GGUF on a local SSD because the ~59 GiB Engram table is lazily mapped. Every split quant's second shard is 59.01 GiB.
- f16 KV cache plus the index pool is 5.7 GiB per 256k tokens. The compute buffer is 2.4 GiB at micro-batch 512 and 4.5 GiB at 1,024, for a 256k pool. MTP draft context is 0.9 GiB at their draft micro-batch of 256. The vision projector is about 1.0 GiB.
- Four slots, a 512k pool, and micro-batch 1,024 totals 113.3 GiB and fails with an out-of-memory error. Four slots sharing a 256k pool at micro-batch 512 totals 105.5 GiB and is fine. The same pool at micro-batch 1,024 is about 107.6 GiB and is fine.
IQ2-AH is the smallest pair at 125.73 GiB, and its trunk shard is 66.72 GiB. AngelSlim does not publish a speed or a memory budget for it. A GPU whose memory is 24 GB or 48 GB is below every file in the official GGUF repo. This page will not invent a quant that fits there.
What “12 billion active” does not change
Active parameters describe the experts evaluated for one token: 11.3 billion for text, plus the 0.43 billion vision encoder when an image is attached. The file you download still holds the full mixture, including the 63.5 billion Engram parameters. A small active count is why the model can be cheaper to serve than a dense model of the same total size. It is not a claim that the checkpoint fits in laptop memory.
Common questions
Can I run Hy4 Flash locally?
Local in the published sense means either the vLLM Docker image on Hopper or Blackwell data-center GPUs, or an AngelSlim GGUF on a machine in the class they measured: a Mac M5 Max with 128 GB or an RTX Spark with 128 GB. The BF16 download is about 440.5 GB. This guide has no sourced recipe for a 16 GB, 24 GB, or 48 GB GPU.
How much VRAM does Hy4 Flash need?
The vLLM recipe prints 529 GB for BF16 and 345 GB for FP8 and says those numbers are the download sizes, 440.5 GB and 287.1 GB, multiplied by 1.2. It calls them conservative aggregate estimates, not measured GPU minima, because Engram tables are offloaded to host RAM. It does not publish the host-RAM figure.
Does the 1M context fit in the starting command?
The checkpoint advertises 1,048,576 tokens. The vLLM recipe starts at 8,192 tokens, four sequences, and one image. AngelSlim's Mac notes budget 5.7 GiB of f16 KV cache and index pool per 256k tokens, and they say a 512k pool with four slots runs out of memory on their 128 GB setup.
Sources
- vLLM recipe for tencent/Hy4-flash
- tencent/Hy4-flash on Hugging Face Model card, architecture table, deployment commands, and license pointer.
- AngelSlim/Hy4-flash-GGUF Official GGUF builds linked from the model card.
- AngelSlim, Apple silicon notes