AI Agents Lost Money Predicting the Future. The Research Was Still Useful.

Six frontier AI models received $10,000 each and access to live prediction markets. They researched, chose contracts and traded autonomously for 57 days.

On Kalshi, every model in the main cohort lost money. Returns ranged from -16.0% to -30.8%. [1]

That sounds like a failed experiment only if profit was the sole thing worth measuring.

A benchmark with consequences

Most AI tests ask questions with known answers. Models can sometimes benefit from memorised patterns or benchmarks that leak into training data.

Prediction Arena used live markets and real capital. Outcomes had not happened yet. The agents had to decide whether to trade, how much, when to exit and which platform to use.

This tests a different kind of intelligence: acting under uncertainty when being articulate is not enough.

The losses exposed useful weaknesses

The models could gather large amounts of information and still make poor decisions. The researchers reported no clear relationship between research volume and returns.

More searching can become a performance of diligence. The agent accumulates facts, writes a convincing thesis and then overpays for a probability the market already understands.

The models also differed in execution. A correct forecast may fail to generate profit if the entry price is bad, the position is too large or the agent exits poorly.

Platform design changed the outcome

The same cohort averaged about -22.6% on Kalshi but only around -1.1% on Polymarket. One preliminary next-generation model made no Kalshi trades yet returned positively on Polymarket over a short period.

That contrast warns against declaring one model a universal forecasting genius. Market selection, liquidity, fees and available contracts can dominate performance.

A bettor behaves differently at an exchange, sportsbook and prediction app because the environment changes what is possible.

Refusing to trade is intelligence too

AI demonstrations are rewarded for action. A trading agent that says “no edge” for three weeks looks inactive.

In real markets, inactivity can be the best decision. Every contract does not require a position. The benchmark found meaningful differences in how models translated confidence into trades.

This matters beyond gambling. Automated systems deployed in finance, procurement or business strategy need the ability to recognise when their information is insufficient.

Why the public will misread the leaderboard

People will focus on which model finished first. The deeper question is why.

Did it forecast more accurately? Take less risk? Choose a friendlier platform? Benefit from one large position? Avoid bad markets? A return number without decomposition can turn an experiment into brand marketing.

The World Cup research reached a similar conclusion: models with comparable predictions behaved differently as decision-makers, and market prices remained difficult baselines to beat. [2]

What the research does not prove

It does not prove AI can never trade profitably. Models, tools and markets will change. The sample lasted 57 days and covered particular platforms.

It also does not prove that humans would have done better. Many humans lose for the same reasons: overconfidence, poor execution and confusing information with edge.

My view

The most valuable result was not that AI lost. It was that researchers could inspect the gap between prediction, confidence and action.

A model may know many facts and still fail to price uncertainty. That is a more realistic measure of intelligence than answering another exam question.

The agents paid tuition to the market. The paper kept the receipt.