Local ModelsLow importance
Three retired V100 GPUs run Qwen 3.6 35B at around 100 tokens per second on a $1,042 home-built machine
An enthusiast tested nine builds with one to three 16 GB K80, P100, or V100 cards, retired server GPUs, on the same Supermicro motherboard with dual Xeons, across more than 130 tests in llama.cpp, according to Tom's Hardware. With three V100s, Qwen 3.6 35B reached around 100 tokens per second on a machine that cost $1,042; the dense Qwen 3.6 27B achieved around 24 tokens per second. A $5,399 Mac Studio M5 Max is slightly faster with short contexts, but the V100s take the lead starting from 64,000 tokens of context. The V100s consume about half the power of the same number of P100s for the same workload.
Published in edition no. 40 of Noticiário do Joãozinho ·
Translated by Joãozinho AI from the original Brazilian Portuguese edition. Facts, numbers and sources are the same as in the original; quotes, video titles and source names stay in their original language. Read the original in Portuguese
Original source
https://www.tomshardware.com/tech-industry/artificial-intelligence/three-e-waste-nvidia-v100s-hit-100-tokens-per-second-on-qwen-3-6-35b-in-a-usd1-042-homelab-rig-decommissioned-gpus-outpace-a-usd5-399-m5-max-mac-studio-at-long-contexts