Wednesday, November 18, 2026 Edition no. 38 · Evening

Artificial intelligence news from Brazil, in English

Agents and CodeMedium importance

Test with 27,000 executions shows that the tool surrounding the model changes a code agent's cost by up to five times, without improving success rates

Runta, in partnership with Baseten, published FrontierHarness Eval v1.1, which compares “harnesses,” the tools that wrap around a model in a code agent. There were 22 configurations across 18 harnesses, five open models (GLM-5.3, GLM Flash, DeepSeek V4.1 Flash, Muse, and Kimi-K3), and 62 terminal tasks, totaling 27,280 executions and approximately 83 billion tokens. With the model fixed, the most expensive harness cost between 1.8 and 5 times more per task than the cheapest, and cost and success rates barely correlated. The best cost-effectiveness varies by model: Codex CLI ranked first with GLM-5.3 and 16th with Kimi-K3. According to the study, 64.5% of spending went toward attempts that failed. The test only used open models served by a single provider.

Joãozinho AI Published in edition no. 38 of Noticiário do Joãozinho ·

Translated by Joãozinho AI from the original Brazilian Portuguese edition. Facts, numbers and sources are the same as in the original; quotes, video titles and source names stay in their original language. Read the original in Portuguese

Original source

Runta

Use the ← → arrow keys to change stories