Backtesting Reliability - Are Simulations in Forex Automated Trading Reliable?

When you start to get into the world of forexa fixed value and uses this value through all the
automated trading and expert advisors you willtest assuming an even separation between the
find that most commercial expert advisors beingAsk and Bid prices. This in turn leads us into an
sold are hyped in virtue of simulation results. Manyunderestimation of the spread costs and a further
of you may have found out that most of theseoverestimation of the profitability of trading
results - showing 100 USD turning into 10 million instrategies with small TP values in which spread
a few months - are not realistic something whichwidening is absolutely critical. We also have the
has sparked a complete lack of trust in backfact that there are no re quotes, off-quotes or
testing results. However in order to understandslippage in back testing something which may also
why these errors are possible and how they canprove to be critical for strategies that attempt to
be avoided it is important to understand theenter and exit the market rapidly.
nature of simulations within the meta traderIn the end you will find out that the simulations of
platform and how these can actually give reliablesystems with average Profit per trade values of
results if adequate precautions are taken. Withinless than 10 times the spread will give significant
this article I will try to explain why the simulationsover estimations of profitability as well as any
can be manipulated to give profitable results andsystem traded on or below the 30 minute time
what particular precautions need to be taken toframe. The lesser time frame where interpolation
avoid this from happening.errors cease to be of any significance is the one
The first thing we need to understand is how thehour time frame where enough reference points
Meta trader back tester works. Since theexist to make the randomness due to
MetaTrader platform does not store any tick datainterpolation negligible.
(or actual Bid and Ask data), it become impossibleThere are other issues concerning data quality on
to do 100% accurate simulations. The softwareseveral pairs and the fact that pairs such as EUR
uses the tick volume information and the OpenCHF and EUR/GBP seem to have inaccurate data
Close/High/Low information of one minute bars tobefore 2006 when downloaded from the Meta
perform the simulations. Since actual tick data istrader history center. The 4 digit history data
not available, the platform "makes up" the ticksshould also be avoided because it introduces
within the one minute bars generating a "randomrandom Sunday candles and 1 pip bars on holidays
walk" over the four aforementioned points on thewhich could strongly affect trading strategies,
one minute bars.making the simulations useless. The best quality
This in turn generates a problem which isdata from Meta quotes is downloaded from a 5
exploitable by certain trading strategies. Fordigit broker so you may want to use a
example, if the value of the Take Profit is belownon-Sunday candle 5 digit broker like Alpari to
10 times the spread it is very likely that theperform your back testing analysis. Also bear in
expert will get to a profitable position in backmind that all back testing data from the history
testing when it would have not in live trading. Itcenter is downloaded from meta quotes so
also leads to a problem when low time framesbeyond the 4-5 digit data sets (from which the 4
are used since indicator signals get triggered athas many data errors) it makes no sense to back
points where they wouldn't have been triggered intest of different brokers.
real life owing to the fact that the "random" walkSo if you want to make sure that your back
given by the back tester account for antests are as reliable as possible please make sure
important percentage of the bars. This effect isyour system trades on the one hour time frame,
very important for almost all time frames belowhas an average profit above 10 times the spread
30 minutes and critical for the 5 minute timeand also make sure you use a non-Sunday 5 digit
frame where the achievement of realistic resultsbroker for the simulations. With that in mind you'll
based on either price action or indicators is simplyhave simulations which will have a high like hood of
not possible due to this problem.being consistent with live testing. Any deviation
Then we also have the problem that livefrom the above parameters will most likely
execution variables are ignored and that there isintroduce problems that will be reflected as over
no separate Bid/Ask feed. Since no tick data isestimations of profitability within the back tester
available the tester "assumes" that the spread hasresults.