DeepSeek outperforms Claude Haiku on cost and accuracy in three tests

DeepSeek V4.1-Flash-1017 outperformed Claude Haiku 5.5 in two tests and matched it in a third. According to these trials, the main difference lies in price, with a total of $0.07 versus $0.30.

DeepSeek V4.1-Flash-1017, an open-weight model from DeepSeek accessible via API or download, outperformed Claude Haiku 5.5 in a series of three tests published on October 22, 2026, by Jessica Wachtel on The New Stack. The Chinese model surpassed Anthropic's on bug fixing and triage scenarios, while matching its competitor on SQL queries. The clearest gap is in total cost: $0.07 for DeepSeek versus $0.30 for Haiku, with precise test conditions as important as the result itself.

Trois tests, trois usages assez proches de la vraie vie

The comparison is not based on an abstract benchmark but on tasks that technical teams actually encounter. Jessica Wachtel tested a Python debugging agent, a batch of 20 queries on a fictitious CI platform database, then a triage exercise of 400 developer error reports, totaling 1,600 scored fields. Models were run once each, via OpenRouter and on publishers' servers, with default settings. No lab marathon, but a quite telling sprint.

In the first test, DeepSeek scored 6 out of 6, versus 5 out of 6 for Haiku. Both fixed the first and third bugs, avoided regressions, and spotted the false test, but Haiku missed a hidden test related to the second bug. DeepSeek took more time in calls and duration, while remaining cheaper thanks to automatic caching of repeated inputs. Based on the settings used, Anthropic had not enabled prompt caching, which may affect the calculation [to be verified].

Pourquoi le triage a fait la différence

The third scenario, triage, was the most revealing. DeepSeek returned 1,600 correct answers out of 1,600, while Haiku missed four. Two errors concerned incident severity: the routing guide required sev4 level for fewer than 50 affected users or an error rate below 1%, but Haiku classified a case with 49 affected clients and another at 0.8% as sev3. The other two discrepancies involved routing to the correct team amid an organizational change on August 1. Again, timing context weighed more than mere text reading.

In this test, DeepSeek also benefited from a format effect. The same prompt counted 153,082 tokens on DeepSeek and 216,811 on Haiku, as models use different tokenizers. Result: cost shifted, with $0.06 for DeepSeek and $0.28 for Haiku on this scenario alone. It's the kind of detail that can unexpectedly raise a bill — like downtown parking, but quieter.

Ce que racontent vraiment les chiffres

Overall, DeepSeek and Haiku remain close in overall accuracy, but not at the same price or speed depending on the task. Haiku was faster on the two tool-assisted tests, while DeepSeek took the lead on the large triage prompt and final cost. The difference is thus not a general superiority of one model over the other, but their behavior facing varied loads, context lengths, and pricing rules.

For teams balancing speed, cost, and accuracy, the message is clear: model choice cannot be based on a single metric. DeepSeek V4.1-Flash-1017 wins this particular trial, but Jessica Wachtel reminds that a single run per test is insufficient to establish a robust trend. In production, the right question remains the real workload, not the catchiest slogan.

By the numbers

  • $0.07 — total cost of DeepSeek across three tests, according to The New Stack, October 22, 2026.
  • $0.30 — total cost of Claude Haiku 5.5 across three tests, according to The New Stack, October 22, 2026.
  • 1,600 fields — perfect score of DeepSeek on triage, versus 1,596 for Haiku.
  • 42.3 — score reported by DeepSeek on Terminal-Bench 4.0 in its model sheet.
  • 39.2 — score reported by Anthropic for Claude Haiku 5.5 on Terminal-Bench 4.0.

To read