Unofficial guide
Run Hy4 Flash on a rented GPU
Last updated
Some links are affiliate links; we may earn a commission at no extra cost to you.
The published vLLM and SGLang examples use eight GPUs and a checkpoint of a few hundred gigabytes. AngelSlim's GGUF speeds are from a 128 GB Mac M5 Max and a 128 GB RTX Spark. If that is not hardware you own, renting a GPU for the length of the job is the way to run the same files. The TokenHub API is the alternative that does not need a GPU at all.
What to rent
- For the vLLM command, the recipe's GPU list is H100, H200, B200, GB200, B300, and GB300. The sample uses tensor parallel 8. AMD Instinct is not supported by that recipe. Budget disk for 440.5 GB (BF16) or 287.1 GB (FP8), plus host RAM for Engram. The recipe does not print the host-RAM amount.
- For GGUF, the machines with published numbers have 128 GB. Q4_K_M is 156.79 GiB on disk, with a 97.78 GiB trunk and a 59.01 GiB second shard that AngelSlim's Mac notes say is lazily mapped.
- Start from the recipe's 8,192-token context. The 1,048,576-token window is a property of the checkpoint, not a promise that the first command will allocate it.
Turn the instance off when the download or the eval is done. A forgotten multi-GPU box costs more than the experiment. Neither link below is an official Tencent host, and neither is required to follow this guide.
Rent a GPU
- Vast.ai Affiliate link Marketplace of rented GPUs. You pick a listing, run the instance, and stop it when you want the rental to end.
- Novita AI Affiliate link GPU instances and an API. Use an instance unless their console lists this model by name.
Requirements has the size table to set against a listing's disk and GPU memory. Download has the repository ids.
Common questions
When is a rented GPU the right way to run Hy4 Flash?
When you want the official vLLM or SGLang command and you do not have eight Hopper or Blackwell data-center GPUs, or when you want an AngelSlim GGUF and you do not have a 128 GB workstation of the kind they measured. The BF16 checkpoint is about 440.5 GB on disk and the FP8 checkpoint is about 287.1 GB.
Do Vast.ai and Novita AI host Hy4 Flash by name?
This guide does not claim that either console has a one-click Hy4 Flash template. They are GPU rentals. Pick a machine whose GPU class and disk match the checkpoint on the requirements page, install the runtime yourself, and stop the instance when you are finished. Prices are on their sites and change.
Are the cloud links advertisements?
Yes. Some links are affiliate links; we may earn a commission at no extra cost to you. They appear only on this page. The rest of the guide links here without those URLs.
Sources
- vLLM recipe for tencent/Hy4-flash
- AngelSlim/Hy4-flash-GGUF Official GGUF builds linked from the model card.