Local ModelsMedium importance
NVIDIA shows how to serve GLM-5.3, with 753 billion parameters, on two networked DGX Stations, with vLLM
The NVFP4 version of GLM-5.3, Z.ai's mixture-of-experts model with 753 billion parameters and 40 billion active per token, needs about 464 GB of GPU memory, more than the 252 GB of HBM3e on a GB300 GPU. NVIDIA's guide links two DGX Stations over ConnectX-8 with RDMA and splits the model between the two GPUs (tensor parallelism 2), serving everything through a single OpenAI-API-compatible endpoint, with multi-token prediction, FP8 KV cache and prefix caching. In the test with the SPEED-Bench coding dataset, peak throughput was 607.8 tokens per second with 16 concurrent requests and fell to 385 tokens per second with 32, with an average latency of 453.5 seconds.
Published in edition no. 28 of Noticiário do Joãozinho ·
Translated by Joãozinho AI from the original Brazilian Portuguese edition. Facts, numbers and sources are the same as in the original; quotes, video titles and source names stay in their original language. Read the original in Portuguese
Original source
NVIDIA Technical BlogNVIDIA: how to connect two DGX Stations
https://developer.nvidia.com/blog/when-one-gpu-isnt-enough-serving-glm-5-3-across-two-nvidia-dgx-stations-with-vllm/