Research log

Tennis probability recalibration: deferred to 1 November, and here is why

Our settled tennis picks with well-established ratings run about 5 points overconfident (stated 64%, actual 59%). We tested the obvious fix — Platt-scaling the stated probabilities, fitted on the first half of the record and judged on the second. It passed that test narrowly. But the mirror test failed: fitted on the second half and judged on the first, the same correction made things worse, and the fitted coefficients swung by a factor of three between halves. The reason is simple: those 449 picks span nineteen days. A correction whose size depends on which three-week window you fit it on is not a correction yet — it is variance wearing a number.

So nothing changes today. The question is registered for 1 November 2026 with the rest of that day's pre-committed reviews, when the sample will be several times larger, under a stricter gate declared now: the recalibration ships only if it improves Brier and log-loss on both chronological splits. The overconfidence itself is real and persists out-of-sample; what we lack is a stable estimate of the fix.

Goals-model calibration study: two corrections shipped, two refused — and a lesson about measuring your own record

Following the side-asymmetry finding below, we tested whether our goals models systematically over-predict goals, on 6,534 held-out matches per market that the deployed models never trained on. The clean answer: mostly no. Over/under 2.5 and both-teams-to-score are unbiased out of sample (+0.6 and +0.1 points) — the larger gaps we measured on our own live board were a selection effect: conditioning on the model choosing a side inflates apparent overconfidence. Two real flaws did survive scrutiny: the over-3.5 tail is genuinely fat (+1.8 points, +4.1 in the big leagues), and the over-1.5 probabilities were too timid. Both now carry fitted corrections that improved held-out accuracy in both test halves; both-teams-to-score and over/under 2.5 ship nothing, because fixing an artifact would have damaged real probabilities. Stated probabilities on the affected markets change by roughly half a point on a typical fixture.

We went looking for our edge in three places and did not find it

Having established that our match-result threshold was testing for a disagreement our own pipeline had removed, the obvious question was whether the disagreement could be restored. We ran three studies. All three said no, by different routes, and we are publishing all three.

First: turn down the market blending. Our served probability is blended 85% into a second model, and the intuition — ours and our founder's — was that this made us too close to the market. It does widen disagreement: turn it off and the spread of our differences from the price nearly doubles. But the extra disagreement is noise. Measured on the matches each setting would actually select, the model gets worse the more we de-anchor it, and at the lowest settings it is significantly worse than simply devigging the bookmaker's price. It also produces fewer qualifying picks, not more. A related dial inside our correct-score model gave the same answer again, independently.

We also found that the 85% figure was never chosen. The search that set it was capped at 85%, and every market pinned to that ceiling; unconstrained, the measurement wanted 100%. And the correct-score blend it feeds, documented in our code as fitted against exchange data, cannot be reproduced — the fitting script does not exist, the data is not on disk, and on the prices we actually serve against, the claimed improvement is a tenth of the stated size and points the wrong way. We have been running a number that was presented as measured and was not.

Second: closing-line value. If our probabilities match where prices end up, and we publish early, we would be capturing value without ever needing to beat the bookmaker. Tested across 7,000 selections, the effect is real and far too small to matter: about 0.2% of stake, against a price handicap of roughly 6%. Worse, we discovered our own measuring instrument is broken — the closing prices we record fail to register around six in ten genuine market moves, so a chunk of what we have been calling closing-line value was never measurable in the first place.

Third, and hardest: our flagship. On the matches where bookmakers price correct-score markets, their prices predict the actual scoreline better than our correct-score model does. Not by much, but consistently, and correct score is the model we are proudest of.

The honest summary is that our football models are competent — they match the market's accuracy, which most published models do not — but we cannot presently show that they beat it. “We find value the bookmakers missed” is not a claim our own evidence supports today.

Three caveats we are not hiding behind, but which are true. These samples are small: the selections that matter run to about a hundred matches each, where the differences at stake need a thousand. Twenty-nine days of season is not a season. And two of the three studies were partly measuring instruments we now know are faulty. Absence of demonstrated edge is not demonstrated absence of edge — but it is where the evidence sits, and we would rather say so than wait for a month that flatters us.

What changes, today. The affected channels stop pretending to be value products: they will publish the model's most confident calls as free analysis, with the probability shown and no claim attached, alongside both the calibration record (are our percentages honest?) and the profit-and-loss record (what would backing them blindly have cost?). We expect the first to look good and the second to look bad, and showing them together is the most useful thing we can publish, because the gap between them is the entire lesson. The 15 September review may conclude that no sound threshold exists for these markets. We are also fixing the measurement, because another sixty days of unmeasurable data would waste the wait.

We found this in August, with six subscribers and nothing sold, because we track every game rather than the ones we back. That is the whole reason the record exists.

Correction: we misdiagnosed our own silent channel, and the real cause is worse

Earlier today we published the entry below, blaming our quiet match-result and over/under channels on an early-season adjustment we shipped on 9 August. That was right for over/under. For match result it was wrong, and we found out within hours of publishing it. The correct account:

The match-result threshold has never produced a single backed selection — not once in 1,209 tracked selections, and not before our 9 August change either. It was never going to. The study that chose that threshold measured it firing on about 3% of matches; live it has fired on none of 628. Those two numbers describe different worlds, and the reason is ours: we chose the threshold using one version of our model's probability and then served a different one. The study used the raw model, which never sees a bookmaker's price. What our channels actually send is that number calibrated, then blended 85% into a second model whose inputs include the market's own prices, then nudged a further 25% toward the market on early-season fixtures. Our published number now tracks the market's number at a correlation of 0.95.

That matters because of what the threshold asks. It only backs a selection when our probability is better than the price implies — a disagreement test. We had quietly rebuilt our probability out of the market's own prices, so there was almost nothing left to disagree with. The two conditions became not merely rare but arithmetically incompatible: on our current board, the highest probability among fixtures priced loosely enough to qualify is 62%, and the threshold needs 65%.

There is a second, independent error. The study's test period was December to May. The channel has only ever run in July and August — and in that same study, even on raw probabilities, the threshold fired zero times in June and zero times in July. A summer of nothing was visible in our own data and we averaged it away.

We are not loosening the threshold in response. We have instead extended what the 15 September review must do: generate its candidates through the pipeline that actually serves you rather than a simplified stand-in, report results month by month so a threshold that cannot fire in the months it operates in cannot pass, and apply the same sanity limits to prices on both sides. That review is explicitly permitted to conclude that no sound threshold exists for these markets and that they should carry analysis rather than backed selections — and if that is the answer, we will publish it and act on it.

The uncomfortable part we are not going to bury: this doubt does not stop at one market. Every threshold set in that July study was chosen the same way, including ones that are currently firing. Firing is not the same as being right. We will know in September, and you will get the answer whichever way it falls. Subscribers to the affected channel have been told this directly rather than left with an explanation that flattered us.

Two of our football gates have gone quiet — and we caused half of it

Corrected the same day — see the entry above. The diagnosis below is accurate for over/under 2.5 but wrong for match result, whose threshold never fired at all and for a different reason. Left standing unedited, as everything here is.

Since the season started, our match-result channel has published no backed picks at all, and over/under 2.5 none since 5 August. Both-teams-to-score still fires normally. We checked whether the record was failing to update: it isn't — every market has settled every day, and the site matches our ledger row for row. The picks simply aren't clearing their thresholds.

Two things explain it. First, the gates are shaped differently than they appear: match-result and over/under only back a selection when our model prices it better than the market does, while the both-teams-to-score gate requires no such disagreement — which is why one keeps passing and the others starve. Second, and this part is ours: on 9 August we shipped a genuine improvement that pulls early-season probabilities toward market prices where teams have little form data. On 23 August it was touching 46 of 60 over/under fixtures and cutting model-market disagreement by roughly 28%. We improved the model's honesty and, with the same change, removed the raw material two of its own gates feed on.

The uncomfortable consequence, stated plainly: thresholds we froze on 26 July are being applied to probabilities that no longer exist in the form they were tuned on.

So we are moving our threshold review forward, from 1 November to 15 September, and we want to be precise about why — because “our thresholds stopped producing picks, so we changed them” is exactly the behaviour a freeze exists to prevent. That is not this. We changed the model, which invalidated the basis on which those gates were tuned; correcting a mismatch we introduced is a different act from loosening a rule we simply dislike the output of. Three commitments make the difference checkable rather than assertable:

First, the rules were written down today, before the review runs, and they are the July rules unchanged: a threshold is adopted only if it is profitable in both halves of the record with at least thirty bets in each, stability preferred over peak performance, and any market without a robust setting stays tracking-only rather than being given a gate that flatters it. The only additions are the two we registered this morning — regenerate probabilities under the current model, and test the probability and odds requirements together instead of holding one fixed.

Second, the review is a historical exercise across roughly 43,000 matches, as it was in July. Our live sample since the model changed is two backed selections; it could not justify a decision and is not an input. The live weeks to 15 September serve only as an out-of-sample sanity check on what the historical review proposes, and we will read them loosely, because early season is the window our own adjustment distorts most.

Third, whatever is adopted on 15 September is frozen until 1 February 2027. A revision date that moves once is a correction; a revision date that moves twice is a habit, and we would rather you hold us to that sentence than to our good intentions. Until September the gates are untouched and the quiet channels explain why they are quiet.

Trading: a two-day cost audit rebuilt how our FX book executes

We asked one question of every trading book: how much edge do execution costs eat, and can we pay less without changing what the strategies believe? Six studies later, the honest answers: our rebalancing books lose ~2.4% a year to spread but most of that churn carries real signal (slowing it down loses more than it saves); our breakout engines' edge lives in their exits (holding longer guts them); and our main FX book had three genuine leaks — impatient exits on two strategies, one pair (EUR/GBP) where every strategy loses because the spread is huge relative to how little it moves, and a once-a-day entry slot at 9pm UK where spreads run five times normal.

Two attractive ideas died on the way, and we're publishing them alongside the wins: a margin-of-victory upgrade for the snooker model improved probability quality but not picks (failed its pre-registered gate), and one strategy's apparent gain from passive entries evaporated under a stricter fill assumption — the trades the limit order missed were precisely the best ones.

The surviving changes — resting limit orders for four strategies (with per-pair exceptions the data demanded), minimum holding periods for two, EUR/GBP removed from execution — were validated as one combined book against 15 years of data, required to beat the current book in both the early and recent halves and with costs doubled. It passed everything: Sharpe 0.77 → 1.03 overall, 1.24 → 1.61 in the modern era, 0.42 → 0.80 under doubled costs, with 11% fewer trades.

The uncomfortable part, disclosed plainly: our backtests had always charged a flat average spread. Re-scoring with real hour-by-hour spreads shows the current live construct — a third of whose entries land in the expensive rollover hour — was flattered by that assumption; much of its paper edge shrinks under honest pricing. The revision is three times less exposed, because resting orders don't pay the entry spread at all. The live bot cuts over only after a verification period confirms real fills match the simulated construct; until then it trades the old way, and the demo record continues unbroken either way.

Pre-declared question for the 1 November threshold review: side-specific goals gates

Observed on 10 August 2026, looking at the record because a reader-level question (“why so many unders?”) sent us there: across four independent goals markets, our models are consistently better on the no-goals side. Under 2.5 (+11.6% ROI, positive in both chronological halves, model underconfident by 4.7pts), Under 3.5 (+8.8%, both halves positive) and BTTS-No (calibrated within 0.2pts) all outperform their opposites: Over 2.5 (−14.7%), Over 3.5 (−27.7%, both halves negative), BTTS-Yes (−14.5%, 4.1pts overconfident) and the Under 1.5 longshot (−37.5%, the worst product in the book).

This is a post-hoc observation being converted into a pre-declared test, and we are saying so plainly: registering it today, before the season's data arrives, means the 1 November review answers it with evidence that had no chance to shape the question. The question: should the goals-market thresholds be side-specific (or side-restricted)? Until then nothing changes — the frozen gates keep backing both sides, and the losing sides keep settling in public as their own control group.

Two snooker and tennis model upgrades tested — both failed their gates

Snooker margin-of-victory Elo. We tested whether scaling rating updates by frame margin (a 9–1 win moves ratings more than a 9–8) improves the snooker model, on 20,977 scored matches with the same parameters the live model uses. Probability quality improved by a sliver in both test halves (Brier 0.2239 vs 0.2246 and 0.2056 vs 0.2067) but pick accuracy got marginally worse in both, and our pre-registered gate required both directions to improve. Not shipped.

Tennis sparse-rating damper. After seeing a match where two nearly-unrated players drew a 97% model probability against a 91% market, we proposed capping stated probabilities at the market's own when either player has fewer than five rated matches. Validation on our settled record refuted the premise: the sparse-rating class (207 picks) is well calibrated — stated 65.7%, actual 66.2% — and outperforms the market's 64.4%. The overconfidence actually lives in the well-rated class (+4.8pts), which is a calibration question, not a data-poverty one. Not shipped. Finding one alarming example and fixing the class it belongs to would have destroyed real edge; the check that stopped us is the same one we run before every change.

Football: season-opener shrinkage extended to the totals and BTTS models

With a new season starting, teams have empty form windows and the models flatten toward their priors — flat probabilities against a confident market print absurd value that is really the model saying “I don't know.” Since 1 August the match-result model has handled this by blending 25% toward the de-margined market whenever either team has played fewer than three matches this season (a mechanism we validated on held-out data before shipping). On 9 August we extended the same mechanism, unchanged, to the over/under 1.5, 2.5, 3.5 and both-teams-to-score models, which were still going into opening weekends unprotected — and which our own settled record shows run 7–9 points overconfident. The correct-score model is deliberately unchanged: it already anchors half its weight to the market at all times.

What this changes publicly: stated probabilities on early-season fixtures will sit closer to the market's view until teams have three matches of form. Every affected prediction carries an opener_shrunk flag in our data. The betting thresholds themselves are frozen until 1 November 2026 and are not touched by this change.

Trading: live entries lagged the backtested fill by 30 minutes to 18 hours

Our simulations fill at the next bar's open, seconds after the signal candle closes. An execution audit on 7 August 2026 found every live bot filling later than that: about 30 minutes late on the FX book, 55 on the explorer book, 90 on indices, and up to 18 hours on commodities. The schedules had been chosen for operational convenience when each bot went live and were never checked against the simulated fill assumption. That is our failure, found by our own audit.

Measured damage across the live history: the average entry was only about 1.5 basis points of price worse than assumed on the 4-hour books, but the dispersion was large, with individual entries up to 70 basis points adrift there and up to 500 on daily commodities. In plain terms, the live bots were trading a noisier version of the strategy than the one we validated. Direct profit impact at current scale: small. Edge dilution: real.

All bots now fill within roughly two minutes of the signal candle closing, with a freshness guard that retries when the broker has not yet published the candle. Live trading results before 7 August 2026 carry the old execution drag. Nothing was restarted and nothing deleted; the drag is disclosed here instead.

FX: the first filter to survive the full gauntlet

We spent a week stress-testing our own research with tools designed to kill ideas: shuffled-signal controls, chronological half-splits, and walk-forwards that only let a rule use information available at the time. Three plausible ideas died that way, including two of our own favourites. One survived: requiring our volatility-breakout entries on FX to trade with the 200-bar trend rather than against it.

The evidence, in the order it was gathered: better results in both halves of eight years of data while the same filter on time-scrambled signals showed nothing; better in seven of nine individual years; and when we replayed history year by year, only ever allowing the decision to use prior data, the filter would have been adopted almost immediately and kept every year but one, roughly doubling the edge per trade on fewer, better trades. A sibling variant using momentum alignment looked equally good in the pooled numbers and failed the year-by-year test, which is exactly the kind of look-alike this process exists to catch. The trend gate went live on the FX book today; its effect will show up in the live record, or it will not, and either way we will report it.

Rugby union: half-by-half scores do not improve the match model

Our rugby data source publishes each game's first and second half scores separately, and we had been discarding them. The reasoning for collecting them was sound: a full time margin is distorted by game state, since blowouts pile up against tiring or fourteen-man sides, while a first half margin is a closer to even contest and a second half margin should carry fitness and bench depth. We backfilled 30,904 games, reaching 96.5% coverage of the competitions the model actually rates.

Then we tested it properly: rolling half-margin features added to the production model, trained and judged on the identical set of 13,777 qualifying games, with the 2024 onward out-of-sample period cut into two chronological halves. Both versions came out slightly WORSE on both halves, by 0.0003 and 0.0020 of log-loss, with accuracy a shade lower too. The reason is arithmetic once you see it: the two half margins add up to the full time margin the model already tracks, so the split carries no new information and only spends parameters. This is the second time this month a promising decomposition has died at the gate, after basketball pace features. The data stays collected, because it opens half-time markets we do not currently model, but it changes nothing about who we think wins.

Football: the 1X2 value avenue is switched off

Our July 2026 sweep tested every sensible value threshold on the match-result market. All of them made money on the first half of the held-out data and lost it on the second. The most tempting one, a 12% modelled edge at odds of 1.70 or better, returned +42.8% across 61 bets and then -20.2% across the next 45. That is the signature of noise, not edge.

We published that finding on 26 July 2026 but left the old bar running, which meant the system could still back a bet under a rule our own test had refuted. As of 29 July 2026 it cannot: those selections are still generated, published and settled so the whole funnel stays visible, but they are never backed. The probability avenue on the same market is untouched and is the one that passed, at 65% or better with odds of 1.55 or better, returning +20.4% and +10.2% across the two halves. The accumulator builder is unaffected, because its legs are drawn from the raw prediction file rather than from the value flag. The Big Odds four-fold keeps the small edge floor it was backtested with: we removed that floor briefly on the same day and restored it, since the acca shapes were chosen by a walk-forward test that included it. No accumulator was published while it was off.

Basketball: box-score pace features for totals, killed by the serving gate

The idea was sound: pace and shooting efficiency should predict total points, and an exploratory study agreed, beating a simple baseline by 1.3% with the improvement holding in every resample. Then it faced the real test: added to our full production totals model and judged on both chronological halves of 2024–26 held-out data.

It failed both halves. The production model's existing features (rolling scoring rates, league context, ratings) already contain what pace was adding, so the extra features made predictions slightly worse, not better. The same thing happened when box scores were tried for match-winner in July. Conclusion twice over: box-score features are redundant with our feature set, and the daily harvesting they would have required is cancelled. A promising idea that dies at the gate is the system working. Published here so it stays dead.

All sports: can the model beat the opening price? (rules fixed before the data)

Registered 27 July 2026, before results exist. Since today our exchange recorders sample every market two-hourly under a plausibility gate (a book only counts with sane two-sided prices and matched volume), so openers and closes are now captured cleanly. The question: when our model disagrees with the opening price, does the market move our way by the close?

The rules, fixed now: measure closing-line value on the side the model favours at opener time, per sport. A sport counts as a REAL edge only with 200+ settled two-sided captures AND positive CLV in both chronological halves of the sample. Anything less is reported as unproven. First read: September 2026, and the numbers will be published here either way — a negative answer ends the idea in public, same as always.

Basketball & handball: totals and handicap models gated, now recording

Pre-registered study (features, model parameters and split fixed before any results were seen): predict each game's total points and winning margin, gate on both chronological out-of-sample halves of 2024–26. All eight cells passed. The margin models beat a rolling-league baseline by 19% (basketball, 79k held-out games) and 28% (handball, 32k) on mean error; totals by 6–8%; and 1σ coverage sat at 0.66–0.69 in every half, so the probabilities derived from these predictions are honest about their own uncertainty.

The usual caveat, stated up front: beating a naive baseline is not beating the market. Both models now record bookmaker opening and closing Over/Under and handicap lines silently, captured from the same odds calls the winner models already make. The market verdict — closing-line value — accrues from the autumn restarts, and nothing joins the public record before the betting rules are pre-registered.

Football: pre-season threshold freeze, swept and confirmed on both halves

Before the 2026-27 season we rebuilt the gate-sweep harness (verified by reproducing the July study near byte-for-byte), re-swept every market on data through July, and froze the gates that passed a strict rule: positive on both chronological halves, thirty-plus bets in each. BTTS was the headline, with a stable value plateau the old gate was missing. Three honest negatives are published alongside: no profitable 1X2 value gate exists, no profitable Double Chance gate exists, and Over/Under 1.5's sweep data is contaminated by mis-lined historical quotes, so its gates stay put rather than being tuned on bad numbers. One pre-committed revision date: 1 November. Rules first, record second.

Esports: a dedicated LoL rating model, second attempt with better data, same answer

Dedicated per-title models halved our gap to the market in Dota 2 and CS2, so LoL got the identical test on 45,000 new match histories. Result: probability quality improved slightly on both halves of the held-out sample, but winner accuracy fell on one half, and our rule is that an upgrade must win on both measures, in both halves, or it doesn't ship. LoL stays on the shared rating pool, which turns out to be genuinely decent at LoL. Two clean refutations means we stop here unless materially new information (side/patch context) becomes available. Re-running the same test until it passes is not research.

Esports: deriving map-score lines (2-0 vs 2-1) from the rating model

Pre-registered before analysis, tested on 86,000 completed best-of-3 series across CS2 and Dota 2. First finding stands on its own: maps within a series are far from independent: the team that takes map one wins map two 22–30 points more often than independence predicts. Even after fitting the single momentum correction the registration allowed, predicted score probabilities missed observed rates by up to 9 points in parts of the range. A number that far off doesn't go on a card, so no map-score lines ship. The momentum finding itself is banked for future, separately registered work.

Snooker: 45 years of historical ratings as a warm start

We licensed access to snooker.org's full match archive (1980→) and tested whether warming the rating engine with four decades of history improves the live model. Under the exact configuration we serve, it didn't: better on one test half, worse on the other, twice. The current model stays. The archive is retained for future, pre-registered experiments.

Football: accumulator construction chosen by backtest

Our daily accumulators were originally built by sensible-but-arbitrary rules. We replayed four seasons of walk-forward model predictions through dozens of candidate construction rules (leg counts, odds bands, market pools, selection scores) with a choose-half/confirm-half split. The winning rules now build the daily accas, including shrinking the combo doubles, because the longer versions almost never landed.

Tennis: surface ratings and world rankings

Surface-specific Elo and official rankings are the features every tennis guide recommends. In ablation on 100k+ ATP and WTA matches, neither reliably improved a model that already blends a 20-year rating with the market's opening price. We don't serve them, and we say so on the tennis page.

All sports: opening-vs-closing price study

Every market we model now has its opening price and near-off closing price recorded. In a few weeks this answers a question we refuse to guess at: does the market move toward our published probabilities? Results will be published here either way.

Five sports: the rating + market blend

A gradient-boosted calibration of two inputs (our rating difference and the market's own price) was validated sport by sport against held-out data, and now serves darts, snooker, rugby league, esports and tennis. In every sport it beat our older, more complicated models.

Darts: rolling-form features from per-match scoring data

We harvested five years of per-match scoring averages for 350 players and built rolling form features. The full model looked better, until ablation showed the gain came entirely from better calibration, not the features. Rating + market alone won on both test halves. The features were dropped; the harvest is kept for future format studies.

Football: correct-score ensemble as a blend

Blending our correct-score model's implied probabilities into the match-result, totals and BTTS models looked like it improved everything, until we traced the gain to a contaminated refit. With clean holdout artifacts, the blend was worse on all three markets. It now serves as an agreement filter only, where the value survived honest testing.

Handball: 1,326 games relabelled for overtime

Our harvested handball scores silently included overtime in some finals, inflating totals. We re-harvested history with half-by-half scores, relabelled exactly the affected games to regulation scores, and retrained. Small, unglamorous, and exactly the kind of data bug that quietly poisons models when nobody audits.

Basketball & handball: opener-capture seasons

Both models are validated offline but their betting layers are deliberately not public yet: thresholds will be pre-registered from forward-captured opening odds before the seasons resume: rules first, record second, never the reverse.

Trading: performance-picked strategy portfolios

250 strategy cells that won era A were re-tested on era B: the portfolio went from excellent to sharply negative. Performance selection does not transfer. Our live book is chosen by mechanism, validated per-cell, and new candidates must earn slots with forward trades only, in a register with frozen verdict dates, evaluated in October 2026.