I do this list of tips so I review before I do some back-testing analysis.

main questions (from John Bogle)

• What is the rationale or hypothesis?
• What is the empirical evidence?
• What are the implementation results?

Tips to remind

1. Do a test of your system with 2008.
2. If you look for correlation, you probably will find it, it does not mean that it will work in future.
3. Test in not the best conditions and compare with other systems of
4. Check volatility, tests on periods with low and high volatility is useful to get conclusions.
5. Track the results in a word document with screenshots of the work done, the variables you touched and the specific results.
6. Prepare set of 5 to 10 stocks for the back-test, track the results of all these stocks.

Typical pitfalls

1. Survivor bias: you are not adding the companies that have not survived.
2. Look-ahead Bias: you have all the data in front of your eyes and can make trades according to that data, this provokes you get a false understanding about the behavior of your system.
3. In sample bias: you use a sample data that is always the same or similar, so then your system adjust to this sample and lose effective.