Local ModelsMedium importance
tandem.cpp, a patch set for llama.cpp, runs DeepSeek-V4.1 Flash on a PC with 64 GB of RAM and a 16 GB card at 14.2 tokens per second, nearly double plain llama.cpp
The author (u/smflx) updated tandem.cpp on r/LocalLLaMA; it keeps the most-used experts of MoE models too large for video memory on the GPU and lets the CPU help with the rest. With DeepSeek-V4.1-Flash in IQ2_XS (139 GiB of experts), an RTX 5070 Ti with 16 GB and 64 GB of RAM generated 14.2 tokens per second, against 7.4 on llama.cpp; since the model is about twice the size of the RAM, part of it is read from the SSD. On an RTX 5090 with 128 GB of RAM, the figures were 33.1 versus 11.5. The author reports gains of 1.4 to 2.1 times on gpt-oss-120b, GLM-4.5-Air and Qwen3.6 as well, and the new version builds for AMD cards (ROCm).
Published in edition no. 30 of Noticiário do Joãozinho ·
Translated by Joãozinho AI from the original Brazilian Portuguese edition. Facts, numbers and sources are the same as in the original; quotes, video titles and source names stay in their original language. Read the original in Portuguese
Original source
GitHub / r/LocalLLaMAPost on r/LocalLLaMA
https://github.com/ewhacc/tandem.cpp