Valory brings its forecasting model closer to GPT-5.6
Valory presents Olas-Predict-Qwen3.8-27B, an open-weight model dedicated to forecasting. On 6,824 markets, it achieves 67.3% balanced accuracy, compared to 68.5% for its comparator GPT-5.6.
Valory has introduced Olas-Predict-Qwen3.8-27B, a 27-billion-parameter open weight forecasting model designed to estimate probabilities from provided indices, rather than to independently search the web. According to the company, it reaches 1.2 percentage points of its comparator GPT-5.6 on a real predictive markets test. The challenge goes beyond raw performance: it shows that a specialized model can approach a very large system when the evidence is well prepared. And, in this kind of exercise, the data weather matters almost as much as the model itself.
What the comparison really measures
The comparison covers 6,824 "researchable" predictive markets, with identical prompts, zero temperature, and a date filter linked to the knowledge cutoff announced by GPT-5.6 on February 16, 2026. In other words, Valory here evaluates how models reason about already provided elements, not their ability to fetch information on their own.
This framing limits the conclusions. These are results published by the company on a selected set, not proof of general equivalence between the two models. The announcement is also distinct from a previous experiment on 10,916 samples and an earlier study around a 14-billion-parameter model.
For measurement, Valory uses two indicators. The balanced accuracy gives equal weight to both result classes, so that a very frequent event does not skew the reading. The Brier score evaluates the quality of predicted probabilities and penalizes overconfident errors: the lower it is, the better.
The generalist base already does much of the work
The most visible gain does not come solely from fine-tuning. The Qwen3.8-27B model untrained for this use case already reaches 66.4% balanced accuracy and a Brier score of 0.2235, which explains a good part of the final result. Specialization then brings an additional precision boost, but not from scratch.
In the final version, Olas-Predict-Qwen3.8-27B reaches 67.3% balanced accuracy and 0.2124 Brier score. Valory also reports about a 12.3% decrease in calibration error, an indicator summarizing the gap between announced confidence and observed results.
Compared to the former Olas-Predict-R1-14B, the progress is clear in this comparison: about 22% reduction in Brier error and 8.4 points more in balanced accuracy. But the most useful reading is elsewhere: the starting base weighs heavily, and specialized training mainly refines how the model expresses its probabilities. It’s less spectacular than a sci-fi leap, but often more useful in production.
An architecture designed to separate data and decision
Valory describes a two-step system. A separate tool gathers market intelligence and evidence elements, then the model transforms these signals into probabilistic estimates. This separation matters, because good logic applied to bad evidence remains a bad forecast.
The Olas project also relies on a continuous feedback loop. Agents create markets, trade, provide forecasts, and finalize results via the Predict network and the Mech Marketplace. Once events are resolved, forecasts can be compared to actual outcomes, which feeds future training.
Valory recalls having used more than 200,000 agent forecast records to specialize Olas-Predict-R1-14B. A later evaluation on 2,628 markets reported 75.8% accuracy, but this data is not directly comparable with the 67.3% balanced accuracy of the new announcement [to be verified].
Why open weight changes the game
An open weight model allows developers to host and adapt it themselves, rather than relying on a remote proprietary service. For a company building a forecasting flow, this can facilitate control over updates, infrastructure, and how evidence is injected into the system.
The hardware question remains. A 27-billion-parameter model requires appropriate resources, with needs varying depending on precision used, context length, and concurrency level. Valory does not publish deployment cost or operational benchmarks here, so the openness gain alone says nothing about the real usage cost.
Finally, forecasting and trading are not the same. Returns also depend on market price, transaction costs, and execution. In other words, a good forecast is not yet a good financial result, and that’s where the most credible models are sometimes less glamorous than a launch press release.
Key points
- 6,824 markets tested in Valory’s announcement.
- 67.3% balanced accuracy for Olas-Predict-Qwen3.8-27B.
- 68.5% for the GPT-5.6 comparator.
- Olas-Predict-R1-14B serves as a previous baseline reference.
- Valory says it uses more than 200,000 agent records.
- 2,628 markets were included in a later cited evaluation.
By the numbers
- 27 billion — parameters of the Olas-Predict-Qwen3.8-27B model, according to Valory.
- 6,824 — predictive markets evaluated in the announcement.
- 67.3% — balanced accuracy of the final model on the reported subset.
- 68.5% — balanced accuracy of the GPT-5.6 comparator on the same test.
- 0.2124 — Brier score of the final model, lower than the cited references.