CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · 
CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT · CONFIDENT IS NOT CORRECT ·

MONTHLY TEST·OCTOBER 2026·630 ANSWERS

How often AI models get trading math right

Name: LLM trading-math benchmark
Creator: TradingCalc

Every month we give current language models a fresh set of trading-math questions, web search off, and score each answer against an exact reference. Results by model and by task, with the confidence each model claimed.

97%

of 630 answers within 1% of the exact value

12 of 17

wrong answers came with a stated confidence of 80% or more

99% / 93%

correct with conventions written out / phrased the way a trader asks

models: a flagship and a smaller model from each of OpenAI, Anthropic and Google

01RESULTS BY MODEL

October 2026: 105 answers per model

claude-opus-5-5Anthropic, flagship

105/105 correct (100%) · 0 wrong · 0 confidently wrong

$1.68 for 105 answers · $0.016 per correct answer

gpt-6.1-solOpenAI, flagship

104/105 correct (99%) · 1 wrong · 1 confidently wrong

$0.527 for 105 answers · $0.00507 per correct answer

gemini-3.8-flashGoogle, smaller

104/105 correct (99%) · 1 wrong · 1 confidently wrong

$0.434 for 105 answers · $0.00418 per correct answer

gpt-5.4-miniOpenAI, smaller

103/105 correct (98%) · 2 wrong · 2 confidently wrong

$0.477 for 105 answers · $0.00463 per correct answer

gemini-4-argonGoogle, flagship

99/105 correct (94%) · 6 wrong · 6 confidently wrong

$0.700 for 105 answers · $0.00707 per correct answer

claude-haiku-5-5Anthropic, smaller

98/105 correct (93%) · 7 wrong · 2 confidently wrong

$0.095 for 105 answers · $0.00097 per correct answer

Confidently wrong means more than 1% off with a stated confidence of 80 or more. This is one run: a gap of one or two answers between models is within what a second run could change, so read the order as indicative until several months agree.

What it cost: the whole run was charged $3.92 for 630 answers, as charged by the API the models are called through, not the vendors' list prices. Cost per correct answer is everything a model’s replies cost, wrong ones included, divided by its correct answers.

WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · 
WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP · WHERE MODELS SLIP ·

02WHERE THE ERRORS ARE

All 17 wrong answers fall into 7 task types

Statistics over a series of returns and the direction of inverse (coin-margined) contracts account for most of them. Each row counts answers from all 6 models.

VaR from 40 daily returnsREALISTIC

6 of 30 wrong, 5 confidently

Annualized Sharpe from 30 daily returnsREALISTIC

3 of 30 wrong, 3 confidently

Call delta at the implied volatility of a quoted priceREALISTIC

2 of 30 wrong, 1 confidently

Bybit inverse PnL with taker feesREALISTIC

2 of 30 wrong

Inverse-contract liquidation priceSPEC

2 of 30 wrong, 2 confidently

Annualized Sharpe from 12 monthly returnsSPEC

1 of 30 wrong, 1 confidently

Inverse-contract PnL with feesSPEC

1 of 30 wrong

No errors from any model: Black-Scholes price, delta, gamma and vega; implied volatility; forex lot sizing, including a USD account on JPY, CHF and CAD pairs; parametric VaR and CVaR from a given mean and deviation; Kelly fraction for a binary contract; inverse average entry; linear PnL with fees; Deribit coin-settled straddle breakeven.

Why the answers were wrong

Every wrong reply was read in full. None came from an unclear question: all 17 wrong answers fall into the 3 causes below.

Slip in hand arithmetic, right formulaA wrong mean or sum of squares over 30 to 40 returns, or one wrong logarithm.

11 of 17

Wrong side or sign, right ideaA losing short reported as a profit; a short liquidation priced with the long formula.

5 of 17

Numerical solve stopped earlyImplied volatility 0.2136 instead of 0.2100.

1 of 17

The reference answers of the 7 task types with wrong answers were recomputed independently, in plain Python and without TradingCalc code: 35 of 35 matched. The other 14 types had no wrong answers. The script runs on the raw data file.

03WHAT IT MEANS

Mostly right, and the wrong answers look the same

On a single, clearly stated question, current models are almost always right. The misses read exactly like the hits, often with the same stated confidence. An agent has no way to spot them from the reply alone.

That is why every TradingCalc MCP result is signed, and every formula and test vector is published.

Why AI agents need exact calculations they can verify What deterministic AI means for trading math Live verification of our own math

04HISTORY

Results by month

October 20266 models · 630 answers · 97% correct · 12 of 17 wrong answers confident · $3.92 run cost · raw data

A new run is added every month with new inputs, so the tasks are never the same twice. Run cost is what that month’s run was charged by the API, not list prices.

05METHOD

How the test works

21 task types across options, forex, risk statistics, prediction markets and perpetual futures, in two tiers: one with every convention written into the question, one phrased the way a trader asks. Inputs are drawn fresh for every run, so no task can come from a public test set. Reference answers come from the same functions our tools call. Each model answers on its own, with web search off, and states its confidence.

Full method, scoring and limits Every task, reference and reply (October 2026)