Appearance

Personalize your experience

Mode

Accent Color

Direction

Evaluation Performance

Coding

Why a backtest is slow and what actually fixes it: condition ordering, indicator reuse, incremental computation, and the difference between an O(n) and an O(n·p) indicator.

Where the time goes

A backtest is a loop over bars, and inside it a tree of conditions each resolving one or more indicators. The cost is therefore roughly bars × conditions × cost per indicator, and the last term varies by two orders of magnitude between a price comparison and a multi-line system like Ichimoku.

Operation	Type	Description
Price field access	O(1)	Reading Close or Volume. Effectively free.
Time variable	O(1)	, DayOfWeek. A timestamp field read, essentially free.
Recursive indicator	O(1) per bar	and its derivatives. Constant work regardless of — an EMA(900) costs the same as an EMA(9).
Windowed indicator	O(p) per bar	A naive SMA, or Max sums its on every bar, so an SMA(200) does 200 additions per bar rather than one.
Multi-line system	O(p) × lines	Ichimoku computes five series; computes a mean and a deviation. Several windowed passes per bar.
Correlation over N	O(p) per bar	Two windows plus their covariance. One of the most expensive things you can put in a condition.

ℹ

Note:The practical consequence: a using (200) is dramatically cheaper than one using SMA(200), because the form does one multiply-add per bar where the windowed form does two hundred. Where the choice is genuinely arbitrary, the recursive one is free money.

Order conditions by cost

AND groups short-circuit on the first false. A cheap condition that rejects most bars therefore prevents an expensive condition from ever being evaluated on them — and on a typical intraday a time alone rejects more than half.

the same strategy, two orderings

# slow — Ichimoku is computed on every bar in the dataset
AND {
  Ichimoku(9,26,52)  ==  "bullish"
  Correlation(Close, Volume, 50)  >  0.4
  HourOfDay  >=  10
  HourOfDay  <   15
}

# fast — most bars are rejected before anything expensive runs
AND {
  HourOfDay  >=  10                       # free, rejects ~60%
  HourOfDay  <   15                       # free
  Ichimoku(9,26,52)  ==  "bullish"        # expensive, now on 40% of bars
  Correlation(Close, Volume, 50)  >  0.4  # expensive, on far fewer still
}

Time filters and checks first — free, and highly selective on intraday strategies.
Price comparisons next — Close > Open costs nothing.
Single indicators after that — , MACD.
Windowed indicators next — SMA, , Max over a long .
Multi-line systems and correlations last, where they are reached least often.

★

Pro tip:Ordering changes nothing about the result — AND is commutative — so this is a pure win. It is also the only optimisation available entirely from inside the , with no code.

Reuse identical indicator calls

RSI(14) referenced in four conditions is one series, computed once and cached, provided the parameters match exactly. RSI(14) and RSI(13) are two separate series with almost identical values and twice the cost.

one series or three

# three separate RSI computations for no analytical gain
AND {
  RSI(14)  >  30
  RSI(13)  <  70
  RSI(15)  >  RSI(15)[1]
}

# one series, cached, and a clearer statement of intent
AND {
  RSI(14)  >  30
  RSI(14)  <  70
  RSI(14)  >  RSI(14)[1]
}

Caution:Near-duplicate periods are a symptom rather than just an inefficiency. If a needs (13) in one condition and RSI(15) in another, those numbers came from tuning rather than from reasoning — and parameters that came from tuning are the ones that will not survive out of sample.

Incremental computation

A windowed indicator does not have to be O(p). A rolling sum SMA into constant work per bar by adding the entering and subtracting the leaving one, which is the difference between 200 operations per bar and two.

naive against rolling

1// O(n · p) — re-sums the whole window on every bar2const smaNaive = (source: number[], period: number) =>3  source.map((_, i) => {4    if (i < period - 1) return NaN;5    let sum = 0;6    for (let k = i - period + 1; k <= i; k++) sum += source[k];7    return sum / period;8  });910// O(n) — maintain the sum, adjusting at both ends11const smaRolling = (source: number[], period: number) => {12  const out: number[] = [];13  let sum = 0;1415  for (let i = 0; i < source.length; i++) {16    sum += source[i];17    if (i >= period) sum -= source[i - period];   // drop the value leaving18    out.push(i < period - 1 ? NaN : sum / period);19  }2021  return out;22};

Caution: sums accumulate floating-point over long series, because the running total is never recomputed from scratch. Over a million bars that drift becomes visible in the low decimals. If exactness matters — reconciling against a reference implementation, for instance — recompute the periodically, or accept the naive form and its cost.

Rolling extremes need more than a sum

Max(High, N) cannot use the add-subtract trick: when the maximum leaves the there is no way to recover the new maximum without looking. The standard solution is a monotonic deque, which keeps the whole series O(n).

rolling maximum in O(n)

1function rollingMax(source: number[], period: number): number[] {2  const out: number[] = [];3  const deque: number[] = [];   // indices, values decreasing front to back45  for (let i = 0; i < source.length; i++) {6    // drop indices that have fallen out of the window7    while (deque.length && deque[0] <= i - period) deque.shift();89    // anything smaller than the incoming value can never be the max again10    while (deque.length && source[deque[deque.length - 1]] <= source[i]) deque.pop();1112    deque.push(i);13    out.push(i < period - 1 ? NaN : source[deque[0]]);14  }1516  return out;17}

Reducing the work rather than speeding it up

The largest wins are usually not micro-optimisations. They are decisions about how much evaluation is needed at all.

Lever	Description
Timeframe	A year of 1-minute bars is roughly 100,000 candles; the same year at 15 minutes is 6,500. Fifteen times less work, and often a better for unrelated reasons.
Date range while iterating	Develop on one month, validate on two years. Fast while the logic is still changing matters more than significance you are not yet ready to read.
Condition count	Fewer conditions is faster and generalises better. The two goals the same way, which is unusual and worth exploiting.
Long-period windowed indicators	Prefer (200) to SMA(200) where the choice is arbitrary. Constant work per bar instead of proportional.
Symbol count	Cost is linear in instruments. Validate the logic on one, then expand — across forty symbols at once is slower in every sense.

★

Pro tip:Reach for the first. Moving from 1-minute to 15-minute bars cuts the work by more than any code change will, and if the stops working at 15 minutes that is information worth having early — a signal that only exists on 1-minute bars is usually noise plus commission.

A note on premature optimisation

evaluation is rarely the bottleneck worth attacking first. Before optimising anything, check that the slowness is real and where you think it is: a backtest that takes ninety seconds because it loads a million bars over a network is not by a faster SMA.

ℹ

Note:And bear the priorities in mind. A that runs twice as fast and is overfitted is worth less than a slow one that works. Condition ordering and indicator reuse are free and worth doing habitually; anything beyond that should follow a measurement.

Appearance

Evaluation Performance

Where the time goes

Order conditions by cost

Reuse identical indicator calls

Incremental computation

Rolling extremes need more than a sum

Reducing the work rather than speeding it up

A note on premature optimisation

See also

Related articles

Appearance

Where the time goes

Order conditions by cost

Reuse identical indicator calls

Incremental computation

Rolling extremes need more than a sum

Reducing the work rather than speeding it up

A note on premature optimisation

See also

Related articles

Related to Evaluation Performance