Social buzzRedditMedium importance
Developer runs Qwen3.5-9B on the GPU of a regular PlayStation 4, offline, and the post goes to the top of r/LocalLLaMA
The author (u/leonardo_alegria) says Qwen3.5-9B runs on a stock PS4 at 31 to 39 tokens per second, using the model's own multi-token prediction (MTP) head for speculative decoding, which, according to him, raised the speed from 22 to 31 to 39 tokens per second. The weights were repacked into 4- and 3-bit formats to fit in about 4.5 GiB. The project, garlic-llm, is free and open source, and the author says the model talks, listens and writes code, without the cloud or a subscription.
Published in edition no. 28 of Noticiário do Joãozinho ·
Translated by Joãozinho AI from the original Brazilian Portuguese edition. Facts, numbers and sources are the same as in the original; quotes, video titles and source names stay in their original language. Read the original in Portuguese