Field guide Hy4 Flash

Checked

Unofficial guide

Hy4 Flash on SGLang

Last updated

The Hy4 Flash model card recommends vLLM or SGLang for production serving and prints an SGLang command. That command is the official quick start this page repeats. A standalone SGLang recipe page for Hy4 Flash was not available when this guide was checked.

Command from the model card

Model card · SGLang · FP8 · 8 GPUs
docker pull lmsysorg/sglang:hy4-flash

docker run --gpus all \
  -p 8000:8000 \
  --ipc=host \
  lmsysorg/sglang:hy4-flash \
  sglang serve \
    --trust-remote-code \
    --model-path tencent/Hy4-flash-FP8 \
    --tp 8 \
    --cuda-graph-backend-prefill disabled \
    --speculative-algorithm DFLY \
    --speculative-dfly-block-size 6 \
    --host 127.0.0.1 \
    --port 8000

The image name is lmsysorg/sglang:hy4-flash. The card says it is multi-arch, for x86 and Arm. The process inside the container is sglang serve with --model-path tencent/Hy4-flash-FP8 and --tp 8. Prefill CUDA graphs are disabled with --cuda-graph-backend-prefill disabled. Speculative decoding uses DFLY and a block size of 6. The server listens on 127.0.0.1:8000 inside the container, and Docker publishes that port on the host.

The card does not print a BF16 SGLang command, a smaller tensor-parallel size, or a measured memory figure for this image. For the hardware list, the 8,192-token starting context, and the note that Engram is offloaded to host RAM, use the vLLM recipe, which is the page that states those limits. Disk sizes for the checkpoint are on download.

Once the server is up, the OpenAI-compatible client on the model card can target it. Match the model field to whatever name this server exposes. The model card's sample client uses hy4-flash against the vLLM command that sets --served-model-name hy4-flash. This SGLang command does not set a served name, so check /v1/models before assuming the same id.

Common questions

Does SGLang officially support Hy4 Flash?

Yes. The Hy4 Flash model card says production serving should use vLLM or SGLang, and it prints a Docker command for the image lmsysorg/sglang:hy4-flash. This guide did not find a separate SGLang recipe page for the model.

Which checkpoint does the SGLang example load?

The command on the model card sets --model-path tencent/Hy4-flash-FP8 and --tp 8. Speculative decoding is --speculative-algorithm DFLY with --speculative-dfly-block-size 6.

Is the SGLang image multi-architecture?

The model card describes lmsysorg/sglang:hy4-flash as a multi-arch image for x86 and Arm.

Sources

  1. tencent/Hy4-flash on Hugging Face Model card, architecture table, deployment commands, and license pointer.