Backtesting

Backtesting

Backtests let you validate the strategy offline with deterministic inputs and configurable friction. The engine lives in src/trading_system/backtest/__init__.py and is invoked through the CLI.

Running a backtest

poetry run ts backtest run \
  --config configs/sample-config.yml \
  --start 2018-01-01 \
  --end 2024-09-23 \
  --output reports/backtests/baseline \
  --label baseline

Outputs:

• metrics.json – strategy-level statistics.
• equity_curve.parquet – daily NAV, drawdowns, returns.
• trades.parquet – fills with timestamps, direction, size, slippage, commission.
• chart.html – optional Plotly equity curve if backtest.include_chart is true.

Compare two runs with:

poetry run ts backtest compare \
  --baseline reports/backtests/baseline \
  --candidate reports/backtests/new-signals

Metrics dictionary

metrics.json is stable and human-readable. Key fields:

• total_return, cagr – compound growth over the requested window.
• volatility, sharpe, sortino – risk-adjusted performance using annualisation from backtest.trading_days_per_year and backtest.annual_risk_free_rate.
• max_drawdown, hit_rate, turnover_total, turnover_average.
• rebalance_events, trades_executed, slippage_cost.

Use ts observability manifest --run reports/backtests/<run> to confirm the manifest recorded each artifact and hash.

IS/OOS protocol

To avoid overfitting:

1. Split history into in-sample (70–80%) and out-of-sample (20–30%) segments.
2. Iterate on strategy parameters only within the in-sample period.
3. Lock parameters, run a single OOS backtest, and accept/reject based on pre-defined success criteria (Sharpe ≥ target, drawdown ≤ guardrail, etc.).
4. If the OOS run fails, reset to hypothesis stage—do not tweak on OOS data.

Walk-forward and rebalancing cadence

• Perform rolling walk-forward tests (e.g., 2-year IS, 6-month OOS) once the base strategy is stable. The preprocessor and risk modules are deterministic, so you can automate these loops in notebooks.
• Align backtest rebalances with rebalance.cadence to reflect operational behaviour.

Friction modelling

• backtest.slippage_pct (default 0.1%) applies per trade by shifting execution prices.
• backtest.commission_per_trade applies a fixed commission per order.
• backtest.initial_cash sets the starting portfolio value; adjust for realistic brokerage balances.

Don’t overfit banner

⚠️ Caution: Higher Sharpe in backtests does not guarantee real-world performance. Bias creeps in via survivorship errors, data snooping, and parameter hunts. Lock down your data provenance (see Data-and-Provenance), keep hypothesis notes, and require live/paper validation before trading capital.

Further reading

The project’s learning plan (docs/trading guid book/Quantitative Trading Learning & Implementation Plan.pdf) condenses risk-adjusted metrics, validation checklists, and monitoring practices. Use it alongside this page when designing new strategies.