Can formulas for 'stats' be provided

[Zirafnik]

Enhancement description

I was wondering whether the formulas and algorithms used in calculation of stats could be documented. It would not only add transparency, but also aid in understanding what the assumptions are, and what you are actually calculating.

For example, I was confused why running the tests with different parameters on the same data, would result in different "Buy & Hold Return [%]" returns...? If I buy at the beginning of the period, and hold until the end, my differing indicator parameters should not affect this result for the same time period.

I presume the "Buy & Hold Return [%]" assumes buying when our algorithm makes the first purchase and then holding until the end of the period? Or perhaps the buy is executed when the lookback period of the most laggy indicator ends?

I see there is a disclaimer in the docs for bt.run(), that the trading simulation begins at different points in the time period, based on the provided indicator parameters, which seems appropriate for some values, but not logical for others (e.g. Buy & Hold)?

Code sample

Additional info, images

No response

[kernc]

See #327. Comments and improvements welcome.

[kernc]

calculation of stats could be documented

As far as code goes, I'm leaning to most stats ideally consisting of self-documenting single lines, e.g.:

backtesting.py/backtesting/_stats.py

Line 116 in cd4bc6c

s.loc['Buy & Hold Return [%]'] = (c[-1] - c[first_trading_bar]) / c[first_trading_bar] * 100 # long-only return

But I'm not against adding a few words about each stat, particularly for the exceptions, somewhere in the existing API docs?

[Zirafnik]

Usually I would agree with software being as self-documenting as possible, however, since this is a financial app I think being explicit with assumptions and explanations is important. After all, people are investing real money based on the results. Additionally, it is easier to understand clear documentation, than reading other people's code (no matter how clean it is).

This way you can also explain the assumptions and decision-making which lead to them. For instance in #327 you changed the 'Buy & Hold' to start after the warm-up period. To me that did not make sense, since I considered the warm-up time for indicators to be their opportunity cost, the time during which you cannot place trades due to lacking information.

This would make sense from a live deployment perspective, if you just turned the algorithm on you would have to wait for it to accumulate enough data before it can start trading, whereas with long-only 'Buy & Hold' you could immediately make an investment. However, this might be flawed thinking as we have access to historical data with which we can "back-calculate" the indicators when turning the algorithm on, so it can start making trades immediately. In this latter case, the way 'Buy & Hold' is currently calculated makes more sense.

All this, I pretty much had to figure out and piece together from multiple issues, docs, and your code snippets. Instead the above should be explicitly described somewhere in the docs. The only real downside is that you have to update the docs when something is changed. (But that should be done anyhow, so more people - other than just those present in the corresponding issue discussion - get eyes on and comment on any flaws in thinking or logic implementation.)