How it unfolds, step by step
The setup
On this chart a rule's running hit rate is tracked over three hundred observations. The dashed line is the base rate. That is what the same measurement gives with no rule applied at all.
What happens next
The rule's line settles around 60%, comfortably above 50%, and would read as a strong result on its own. The base rate line sits at 55%.
What it means — and does not
The gap is five points, not ten and not sixty. Whether five points is worth anything then depends entirely on the arithmetic from the costs track: what a trade costs, and how often the rule fires. Stated without its base rate, the same 60% was a headline. Stated with it, it is a small effect that has to clear a known hurdle.
- Rule's running hit rate (%)
- Base rate: what happened anyway · 55
- A coin · 50
The rule settles near 60% and looks strong until the base rate is drawn on the same axes. The finding is the gap.
Show the underlying data
| Observations | Rule's running hit rate (%) |
|---|---|
| 10 | 70 |
| 20 | 65 |
| 40 | 67.5 |
| 60 | 63.3 |
| 80 | 62.5 |
| 100 | 61 |
| 130 | 60.8 |
| 160 | 59.4 |
| 190 | 60.5 |
| 220 | 59.5 |
| 250 | 60.4 |
| 300 | 60 |
What this does not establish
The reading fails if the base rate is measured on a different population from the rule. A rule that only fires on high-volatility days must be compared against the base rate ON high-volatility days, not against all days. Otherwise the 'edge' is just the difference between two kinds of day. The level that settles it is whether the comparison set is the set the rule could have fired on.
Things to check
4What did the same measurement give with no rule applied?
Is that base rate measured on the same population the rule fires on?
How large is the gap, and does it clear the cost hurdle?
Compare the rule with the market over the same dates.
Common mistakes
2Quoting a hit rate with no benchmark, which makes any number above 50% sound like a finding.
Comparing a rule that fires on unusual days against the base rate for all days.
A rule is right 60% of the time. The same measurement with no rule applied gives 55%. What has been found?
- A 60% edge
- Nothing, because 60% is close to a coin
- Five percentage points, which then has to clear the cost of trading it
Show the answer
C. Five percentage points, which then has to clear the cost of trading it
A rule can only be credited with the difference between its rate and the base rate for the same population. Five points is a real effect and a small one. Whether it is worth anything is the break-even arithmetic, not a matter of the headline number.
Frequently asked
3Where do I find the base rate?
By running the same measurement with the rule switched off, over the same period and the same population. If that is not possible, the honest statement is that the result has no benchmark.
Is a five-point edge worth anything?
It depends on the cost per trade and how often the rule fires. The break-even hit rate arithmetic in the costs track is exactly this calculation.
Why does this site publish base rates as its own section?
Because a figure without its benchmark is not interpretable. The /base-rates section publishes the unconditional numbers with a methodology and a pre-registration log, so a claim can be checked against them.