Field guide Hy4 Flash

Checked

Unofficial guide

Hy4 Flash requirements

Last updated

“Can I run it?” depends on which file you mean. The answers below are the hardware lines the vLLM recipe and AngelSlim actually publish. Where they do not give a number, this page does not fill one in.

Disk

You need the disk before you need the GPU. Tree sums were checked 21 November 2026. GGUF sizes are AngelSlim's.
File Disk
tencent/Hy4-flash (BF16 instruct) 440.53 GB
tencent/Hy4-flash-FP8 287.09 GB
tencent/Hy4-flash-base 438.84 GB
GGUF Q4_K_M pair 156.79 GiB
GGUF Q3_K pair 144.70 GiB
GGUF IQ2-AH pair 125.73 GiB
SPARK-4Bit / SPARK-4Bit-Fast 151.35 GiB / 148.65 GiB

vLLM: GPUs and the VRAM hint

The recipe lists Hopper and Blackwell data-center GPUs: H100, H200, B200, GB200, B300, and GB300. The per-GPU memory figures on that page are the cards' capacities (80 GB, 141 GB, 180 GB, 192 GB, 268 GB, and 288 GB on the NVIDIA side). AMD Instinct is listed in the page chrome and then ruled out in the text: no ROCm image ships Hy4 Flash.

The same page shows variant hints of 529 GB for BF16 and 345 GB for FP8. The prose says those are the checkpoint download sizes, 440.5 GB and 287.1 GB, times 1.2, and that they are not measured minimums. Engram storage is offloaded to the CPU by default, which lowers the GPU footprint and means you also need host RAM. The recipe does not say how many gigabytes of host RAM. The published Docker examples set --tensor-parallel-size 8.

Context is a separate budget. The weights advertise 1,048,576 positions. The recipe's starting command uses 8,192 tokens, four sequences, and one image, with video left at zero. Raising --max-model-len is how you spend the rest of the memory, not something the sample command does for you.

GGUF: the machines with published results

AngelSlim publishes speeds for Q4_K_M and Q3_K on a Mac M5 Max with 128 GB, and for the two SPARK-4Bit files on an RTX Spark with 128 GB. Their Mac README is more specific about memory, and it is about that patch, not a generic laptop:

IQ2-AH is the smallest pair at 125.73 GiB, and its trunk shard is 66.72 GiB. AngelSlim does not publish a speed or a memory budget for it. A GPU whose memory is 24 GB or 48 GB is below every file in the official GGUF repo. This page will not invent a quant that fits there.

What “12 billion active” does not change

Active parameters describe the experts evaluated for one token: 11.3 billion for text, plus the 0.43 billion vision encoder when an image is attached. The file you download still holds the full mixture, including the 63.5 billion Engram parameters. A small active count is why the model can be cheaper to serve than a dense model of the same total size. It is not a claim that the checkpoint fits in laptop memory.

Common questions

Can I run Hy4 Flash locally?

Local in the published sense means either the vLLM Docker image on Hopper or Blackwell data-center GPUs, or an AngelSlim GGUF on a machine in the class they measured: a Mac M5 Max with 128 GB or an RTX Spark with 128 GB. The BF16 download is about 440.5 GB. This guide has no sourced recipe for a 16 GB, 24 GB, or 48 GB GPU.

How much VRAM does Hy4 Flash need?

The vLLM recipe prints 529 GB for BF16 and 345 GB for FP8 and says those numbers are the download sizes, 440.5 GB and 287.1 GB, multiplied by 1.2. It calls them conservative aggregate estimates, not measured GPU minima, because Engram tables are offloaded to host RAM. It does not publish the host-RAM figure.

Does the 1M context fit in the starting command?

The checkpoint advertises 1,048,576 tokens. The vLLM recipe starts at 8,192 tokens, four sequences, and one image. AngelSlim's Mac notes budget 5.7 GiB of f16 KV cache and index pool per 256k tokens, and they say a 512k pool with four slots runs out of memory on their 128 GB setup.

Sources

  1. vLLM recipe for tencent/Hy4-flash
  2. tencent/Hy4-flash on Hugging Face Model card, architecture table, deployment commands, and license pointer.
  3. AngelSlim/Hy4-flash-GGUF Official GGUF builds linked from the model card.
  4. AngelSlim, Apple silicon notes