Business and InvestmentMedium importance
Cerebras combines AMD GPUs and its giant chips in the same inference and says it delivers five times more tokens per megawatt
Cerebras has launched disaggregated inference, already in production: GPUs handle the prompt (the prefill phase) and Cerebras' CS-3 systems generate the responses (the decode phase). In production deployment, AMD Instinct GPUs handle the prompt and a software called KV Portal moves the model state from one side to the other via RDMA. According to the company, in a service with GPT-OSS-120B measured on 11/14, the arrangement delivered five times more throughput per megawatt than a CS-3-only configuration, without losing generation speed. AWS, Fireworks AI, Gimlet Labs and General Compute have early access, and Cerebras aims to scale the model to hundreds of megawatts by the end of 2027.
Published in edition no. 38 of Noticiário do Joãozinho ·
Translated by Joãozinho AI from the original Brazilian Portuguese edition. Facts, numbers and sources are the same as in the original; quotes, video titles and source names stay in their original language. Read the original in Portuguese
Original source
https://www.cerebras.ai/blog/cerebras-launches-disaggregated-inference