Validating R-Backtesting frameworks against Metatrader 4 with 99% tick accuracy

The primary objective here is to make sure the framework I use in R to backtest strategies delivers similar results to that of MT4, most specifically to 99% tick accuracy MT4 backtesting. Why the 99% tick accuracy …. because it’s reproducible. A secondary outcome of this validation is to determine if backtesting could be carry on only in R, to benefit from enhanced statistical optimization, but in a time-efficient way when compared to MT4. Assuming positive outcomes, this will be followed by a post detailing the process of transferring a backtested strategy in R to production for MT4 demo-trading, prior live release.

MT4 Strategy used for validation: the SMA

I will investigate a Simple Moving Average (SMA) strategy, derived from this universalMACrossEA.mq4 MT4 script. The strategy is as follows: When FastMA (period=20) cross-up Slow MA (period=50), send a Buy and Close Sell, When FastMA cross-down SlowMA, Close Buy  and Open Sell. EA Inputs are as follows (Refer to the figure below for a complete setup of strategy inputs):  [...] FastMAPeriod=20, FastMAType=0 [...] SlowMAPeriod=50, SlowMAType=0 [...] MinCrossDistance=0 [...] StopAndReverse=false, PureSAR=true, ExitOnCross=true [...] ; If I’m getting this thing right, it doesn’t matter what the other inputs are to obtain the desired behavior.

There follows the illustration of this SMA strategy backtested in MT4 with 99% tick accuracy, on GBPUSD Hourly (H1). Ah ! the 99% tick-by-tick backtesting in MT4, a whole story in itself but hopefully there are great tools out there to automate the process now. I personally use TickStory. The final drop to easily breath through that process is to download an old version of Metatrader that is compatible with TickStory ; I use an old Alpari NZ version 4.00 build 451 for that purpose w/o any account to prevent auto-updating. You may google mtjp4setup-451.exe to find proper builds. The complete MT4 Backtesting Strategy Report is available here.

Strategy Tester Report with 99% tick-accuracy for SMA strategy

Strategy Tester Report with 99% tick-accuracy for SMA strategy

… discussing the interest of backtesting on tick-by-tick data for a strategy based on hourly stats is out of the present context – yet I do think this is not necessary but it looks good on the report !

R Backtesting Results on Equivalent SMA Strategy

Initial Test on hourly quotes not originating from the tick data

My first objective was to quickly determine whether or not there were supporting evidence that R & MT4 could provide the same results prior getting into precise testing. More specifically, as I’m interested in using the great framework designed by the Systematic Investor Toolbox for R (SIT), I’ve elected to reuse the Forex Intraday backtest example mixed with a later blog to implement the SMA strategy – The R-script is presented below (Thx to R syntax highlighting in wp.com) and you can see how simple it is , and thus how great the R SIT library is. It is worth noting that this design uses hourly historical Forex data from FXHISTORICALDATA.COM, that features hourly Low-High 4-digits quotes only, that are of no match to the 5-digits tick data quality provided by TickStory. Nonetheless, I felt this might give sufficient evidence, if lucky, to motivate further work.

con = gzcon(url('http://www.systematicportfolio.com/sit.gz', 'rb'))
source(con)
close(con)

#*****************************************************************
# Load historical data
#******************************************************************
load.packages('quantmod')

New_MA <- function (){
tickers = spl('GBPUSD')

capital = 10000

data <- new.env()
getSymbols.fxhistoricaldata(tickers, 'hour', data, download=TRUE)

#  bt.prep function merges and aligns all symbols in the data environment
bt.prep(data, align='remove.na', dates='2009::2010')

prices = data$prices
# n = len(tickers)
models = list()


#*****************************************************************
# Code Strategies
#******************************************************************
data$weight[] = NA
data$weight[] = 1
models$buy.hold = bt.run.share(data, clean.signal=TRUE
#,capital=capital
)

#*****************************************************************
# Code Strategies : SMA Fast/Slow cross over
#******************************************************************
sma.fast = SMA(prices, 20)
sma.slow = SMA(prices, 50)

# data$weight matrix holds weights (signals) to open/close positions.
data$weight[] = NA
data$weight[] = iif(cross.up(sma.fast, sma.slow), 1, iif(cross.dn(sma.fast, sma.slow), -1, NA))

# bt.run computes the equity curve of strategy specified by data$weight matrix
models$ma.cross = bt.run.share(data, clean.signal=TRUE, trade.summary = TRUE
, do.lag = 2
#, commission=0.0002
#, capital=capital
)

#*****************************************************************
# Code Strategies : MA Cross Over
#******************************************************************
#   sma = bt.apply(data, function(x) { SMA(Cl(x), 200) } )
#   data$weight[] = NA
#   data$weight[] = iif(prices >= sma, 1, 0)
#   sma.cross = bt.run(data, trade.summary=T)

#*****************************************************************
# Create Report
#******************************************************************

# plotbt.custom.report function creates the customized report, which can be fined tuned by the user
plotbt.custom.report(models$ma.cross, models$buy.hold)
plotbt.custom.report(models$ma.cross, models$buy.hold, trade.summary=TRUE)
strategy.performance.snapshoot(models, T)
models$ma.cross$trade.summary
}

Considering the modest quality of the Forex data used in R for that first test, a first step was to compare the Date & Time at which orders were past in both R and MT4 backtests. Thereafter are presented parts of the tables summarizing the results. Green marks indicate an Order Type & Time match, Orange marks indicate Order Type & Time (+/- 1Hour) match, and Red marks indicate Orders with time mismatches. The proportion of Green and Orange marks indicates a good correlation between the 2 backtests and support further investigation. I have briefly looked at the origin of the Red marks to found that, for example, MT4 data started on the Feb.22nd 2009, a Sunday, whereas the R data started on the 23rd at midnight. I had the feeling this was Broker-GMT related and didn’t feel like this should require further details.

Comparing Hourly backtested orders from R with MT4 tick-by-tick orders

Comparing Hourly backtested orders from R with MT4 tick-by-tick orders

… I couldn’t help myself but kept pushing my luck into investigating the magnitude of the Loss/Profits for each of R/MT4 orders, to quickly realize that the discrepancies between the two Forex data were to great for a more precise study. At this point we’ve been lucky enough and I need to load the 5-digits tick data into R for a comparable backtest and perform a thorough evaluation against the MT4 backtests.

Running a Backtest in R on Hourly Data derived from the Tick Data

In the SIT framework, the data variable returning the hourly-formatted tick data is as followed:

<br />&gt; ls(data)<br />[1] "dates" "execution.price" "GBPUSD_hour" "prices" "symbolnames" "weight"<br />&gt; ls.str(data)<br />dates : POSIXct[1:6171], format: "2009-01-11 21:00:00" "2009-01-11 22:00:00" "2009-01-11 23:00:00" "2009-01-12 00:00:00" "2009-01-12 01:00:00" ...<br />execution.price : An ‘xts’ object on 2009-01-11 21:00:00/2010-01-01 21:00:00 containing:<br />Data: num [1:6171, 1] NA NA NA NA NA NA NA NA NA NA ...<br />- attr(*, "dimnames")=List of 2<br />..$ : NULL<br />..$ : chr "GBPUSD_hour"<br />Indexed by objects of class: [POSIXct,POSIXt] TZ: GMT<br />xts Attributes:<br />NULL<br />GBPUSD_hour : An ‘xts’ object on 2009-01-11 21:00:00/2010-01-01 21:00:00 containing:<br />Data: num [1:6171, 1:4] 1.51 1.51 1.51 1.51 1.51 ...<br />- attr(*, "dimnames")=List of 2<br />..$ : NULL<br />..$ : chr [1:4] "Open" "High" "Low" "Close"<br />Indexed by objects of class: [POSIXlt,POSIXt] TZ: GMT<br />xts Attributes:<br />NULL<br />prices : An ‘xts’ object on 2009-01-11 21:00:00/2010-01-01 21:00:00 containing:<br />Data: num [1:6171, 1] 1.51 1.51 1.51 1.51 1.51 ...<br />- attr(*, "dimnames")=List of 2<br />..$ : NULL<br />..$ : chr "GBPUSD_hour"<br />Indexed by objects of class: [POSIXct,POSIXt] TZ: GMT<br />xts Attributes:<br />NULL<br />symbolnames : chr "GBPUSD_hour"<br />weight : An ‘xts’ object on 2009-01-11 21:00:00/2010-01-01 21:00:00 containing:<br />Data: num [1:6171, 1] NA NA NA NA NA NA NA NA NA NA ...<br />- attr(*, "dimnames")=List of 2<br />..$ : NULL<br />..$ : chr "GBPUSD_hour"<br />Indexed by objects of class: [POSIXct,POSIXt] TZ: GMT<br />xts Attributes:<br />NULL<br />

Now that I know the input format for SIT to work as-is, I move on to TickStory and look at the possible export format hoping I will not need to write an R-script to hourly-format the tick data myself. Luckily, I found that TickStory can output the following ‘Generic Bar Format’ = {BarBeginTime:yyyyMMdd},{BarBeginTime:HH:mm:ss},{Open},{High},{Low},{Close},{Volume} (told you it was a great piece of freeware); and that is the case for bar from 1 second to 1 week. Great ! The R function to read-in TickStory Generic Bar Format (hourly) into something SIT can digest, based on sourcing the SIT function getSymbols.fxhistoricaldata(…) is presented thereafter. For testing purposes, the output from TickStory for hourly bars from 2009.01.01 to 2010.01.01 is available here (396KB). One last thing I need to mention is that I have loaded information from my broker for TickStory to use a Spread of 20 on 5-digits, that is 2pips.

Note that this script is sensitive to the ‘Generic Bar Format’ – for now I’ll stick to what comes out of the box and we’ll see in the future how we’ll do things, specifically as you might have seen that the ‘Generic Tick Format’ differs from the ‘..Bar..’ one in that Dates and Times are 2 different columns for ‘..bar..’ and concatenated as a single column for ‘..tick..’  !

TickStory2SIT <- function (Symbols="GBPUSD", type="hour", env = .GlobalEnv,
                           file="Forex//TestData/GBPUSD_GenericBarFormat2009-2010.csv" , download = FALSE){

  # test file at https://drive.google.com/file/d/0BxMQxUqd263pTFFYV29IcmlHem8/edit?usp=sharing
  temp <- read.csv(file, sep = ",")
  out = xts(temp[3:6],
                 strptime(paste(temp$Date, temp$Timestamp), format='%Y%m%d %H:%M:%S'))
  assign(paste(gsub('\\^', '', Symbols), type, sep='_'), out, env)
  return(env)
}

Now, in the ‘New_MA <- function ()’ created previously replace the following lines to input the hourly tick-derived data we’ve just created:

  # tickers = spl('GBPUSD')
  capital = 10000
  data <- new.env()
  # getSymbols.fxhistoricaldata(tickers, 'hour', data, download=TRUE)
  TickStory2SIT("GBPUSD", "hour", data, file="./MT4ReportStatementAnalysis/TestData/GBPUSD_GenericBarFormat2009-2010.csv")

One final touch is to write a R function to load the MT4 Strategy Reports and further quantify potential differences between R and MT4 backtests :

MT4_ParseStrategyReport = function(SRname)
{
x = readHTMLTable(SRname)
#   > head(SR_Parsed)
#     1 2009.01.02 06:01   buy 1 0.02 1.38696 0.00000 1.38926            NA
#   2 2 2009-01-02 06:40   buy 2 0.04 1.38486 0.00000 1.38716          <NA>
#   3 3 2009-01-02 07:10   t/p 2 0.04 1.38716 0.00000 1.38716 9.20 10009.20
#   4 4 2009-01-02 07:10 close 1 0.02 1.38717 0.00000 1.38926 0.42 10009.62
perf <- NULL
perf$time <- as.character(x[[2]][,2])
perf$OrderType <- as.character(x[[2]][,3])
perf$OrderID <- as.double(as.character(x[[2]][,4]))
perf$LotSize <- as.double(as.character(x[[2]][,5]))
perf$Price <- as.double(as.character(x[[2]][,6]))
perf$SL <- as.double(as.character(x[[2]][,7]))
perf$TP <- as.double(as.character(x[[2]][,8]))
perf$Profit <- as.double(as.character(x[[2]][,9]))  ; perf$Profit[is.na(perf$Profit)] =0
perf$Balance <- as.double(as.character(x[[2]][,10])); perf$Balance[is.na(perf$Balance)] =0

table = getNodeSet(htmlParse(SRname),"//table") [[1]]
mypattern = '<td>([^<]*)</td>'
xt <- readHTMLTable(table,
header = c("Content"), colClasses = c("character"),
trim = TRUE, stringsAsFactors = FALSE
)
perf$Symbol <- unlist(strsplit(xt[[2]][1]," "))[[1]]

# desired structure
# symbol | weight | entry.date | exit.date | entry.price | exit.price | return
entry = which(!duplicated(perf$OrderID), arr.ind=TRUE)
ret <- NULL
for (i in 1:length(entry) ){
exit = which(perf$OrderID==perf$OrderID[entry[i]], arr.ind=TRUE)[2]
ret$symbol <- c( ret$symbol, perf$Symbol)

if (perf$OrderType[entry[i]]=="buy") {ret$weight <- c(ret$weight, perf$LotSize[entry[i]])}
else {ret$weight <- c(ret$weight, -perf$LotSize[entry[i]])}

ret$entry.date <- c(ret$entry.date, as.character(perf$time[entry[i]]) )
ret$exit.date <- c(ret$exit.date, as.character(perf$time[exit]) )
ret$entry.price <- c(ret$entry.price, perf$Price[entry[i]])
ret$exit.price <- c(ret$exit.price, perf$Price[exit])
ret$profit <- c(ret$profit, perf$Profit[exit])
} # VALIDATION: cumsum(ret$profit)+10000 #InitialDeposit=10000
return(ret)
}

NOTE: As I was validating this R function by comparing the cumulative returns obtained with R with that of MT4 Build 225 from the Strategy Test Report on INDRAFXSCALPING_V4.2, I realized that MT4 was off in calculating the ‘Balance=Equity at orders’ closes’ – refer to OrderID 45 to 49 ; kind of disappointing MT4 cannot even out put correct cumulative sums but I guess that’s what retail platform stands for (but hey, they do provide means to make $$ at a retail price – so I’m fine with it). Yet, I have to check if this was corrected in latter builds (we’ll see about that). Although I don’t have the same MT4 strategy to test, what i can say is that, with MT4 Build 451 from the Strategy Test Report on GBPUSD_2009_SMAcross (our real interest for that post), R cumulative sum of profits correlate with that of MT4.

Comparing the Results of Both Backtests on hourly data derived from tick data

Apple to Apple … that’s always the issue. Looking closely at the Hourly data derived from Tick data through TickStory I realized that Hourly {Open,Close,Low,High} are bid prices only (after comparing with the complete bid/ask quote on tick data). That’s the first thing to remember – THE SET-UP IS SUBOPTIMAL WHEN COMPARED TO LIVE TRADING CONSIDERING ONLY BID PRICES ARE USED (during live trading, you buy @ ask, and sell @ bid). However, looking at MT4 Strategy Report, I could not explain the execution prices from the tick data (weird right!). Anyways, I’ll come to this later – here is a quick analysis of the SIT Vs MT4.

First you will need to re-source this function to update the SIT trade.summary function and enable 5-digits precision in the report (instead of the rounding 2-digits).

bt.trade.summary <- function
(
  b,
  bt, Rounding=5
)
{

  if( bt$type == 'weight') weight = bt$weight else weight = bt$share
  out = NULL
  weight1 = mlag(weight, -1)
  tstart = weight != weight1 & weight1 != 0
  tend = weight != 0 & weight != weight1
  tstart[1, weight[1,] != 0] = T
  trade = ifna(tstart | tend, FALSE)
  prices = b$prices[bt$dates.index,,drop=F]
  if( sum(trade) > 0 ) {
    execution.price = coredata(b$execution.price[bt$dates.index,,drop=F])
    prices1 = coredata(b$prices[bt$dates.index,,drop=F])
    prices1[trade] = iif( is.na(execution.price[trade]), prices1[trade], execution.price[trade] )
    prices1[is.na(prices1)] = ifna(mlag(prices1), NA)[is.na(prices1)]
    prices[] = prices1
    weight = bt$weight
    symbolnames = b$symbolnames
    nsymbols = len(symbolnames)
    trades = c()
    for( i in 1:nsymbols ) {
      tstarti = which(tstart[,i])
      tendi = which(tend[,i])
      if( len(tstarti) > 0 ) {
        if( len(tendi) < len(tstarti) ) tendi = c(tendi, nrow(weight))
        trades = rbind(trades,
                       cbind(i, weight[(tstarti+1), i],
                             tstarti, tendi,
                             as.vector(prices[tstarti, i]), as.vector(prices[tendi,i])
                       )
        )
      }
    }
    colnames(trades) = spl('symbol,weight,entry.date,exit.date,entry.price,exit.price')
    out = list()
    out$stats = cbind(
      bt.trade.summary.helper(trades),
      bt.trade.summary.helper(trades[trades[, 'weight'] >= 0, ]),
      bt.trade.summary.helper(trades[trades[, 'weight'] <0, ])
    )
    colnames(out$stats) = spl('All,Long,Short')
    temp.x = index.xts(weight)
    trades = data.frame(coredata(trades))
    trades$symbol = symbolnames[trades$symbol]
    trades$entry.date = temp.x[trades$entry.date]
    trades$exit.date = temp.x[trades$exit.date]
    trades$return = round(100*(trades$weight) * (trades$exit.price/trades$entry.price - 1),Rounding)
    trades$entry.price = round(trades$entry.price, Rounding)
    trades$exit.price = round(trades$exit.price, Rounding)
    trades$weight = round(100*(trades$weight),Rounding)
    out$trades = as.matrix(trades)
  }
  return(out)
}

The you may use this to quickly compare Priece Entry differences and Profits Differences:

###########################
# Entry Price Comparison
##########################
SIT <- as.data.frame(models$ma.cross$trade.summary[2])
l = length(ret$entry.date)

DiffInSeconds = abs(as.POSIXct(strptime(ret$entry.date[2:l], "%Y.%m.%d %H:%M")) -
as.POSIXct(strptime(SIT$trades.entry.date, "%Y-%m-%d %H:%M")) )

SIT$trades.entry.date[which(DiffInSeconds>0)]
ret$entry.date[which(DiffInSeconds>0)+1]

############################
# Entry Price Comparison
############################
DiffInEntryPrice = abs(as.double(paste(ret$entry.price[2:l])) - as.double(paste(SIT$trades.entry.price)))

###############################
# Profits Comparison
##############################

MT4Prof = (as.double(paste(ret$entry.price[2:l])) - as.double(paste(ret$exit.price[2:l]))) * (-ret$weight[2:l])
RProf = (as.double(paste(SIT$trades.entry.price)) - as.double(paste(SIT$trades.exit.price))) * (-as.double(paste(SIT$trades.weight))/100)

formatC(100*abs(MT4Prof - RProf) / abs(MT4Prof), format="fg")

###########################
# Order Type Comparison
###########################
max(cumsum(ret$weight[2:l] - as.double(paste(SIT$trades.weight))/100))

The results will be as follows:

<br /><br />&gt; ###########################<br />&gt; # Entry Price Comparison<br />&gt; ##########################<br />&gt; SIT  l = length(ret$entry.date)<br />&gt;<br />&gt; DiffInSeconds = abs(as.POSIXct(strptime(ret$entry.date[2:l], "%Y.%m.%d %H:%M")) -<br />+                       as.POSIXct(strptime(SIT$trades.entry.date, "%Y-%m-%d %H:%M")) )<br />&gt; DiffInSeconds<br />Time differences in secs<br />  [1]    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0 3600    0    0    0    0    0<br /> [28]    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0 3600    0    0    0    0    0    0<br /> [55]    0    0    0   60    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0<br /> [82]    0    0    0    0 3600    0    0    0    0    0 3600    0    0    0    0    0    0 3600 3600    0    0    0    0    0    0 3600    0<br />[109]    0    0    0    0    0    0    0    0 3600 3600    0 3600    0    0    0    0    0    0    0    0    0    0    0<br />attr(,"tzone")<br />[1] "GMT"<br /><br />&gt; DiffInEntryPrice = abs(as.double(paste(ret$entry.price[2:l])) - as.double(paste(SIT$trades.entry.price)))<br />&gt; DiffInEntryPrice<br />  [1] 0.00110 0.00590 0.00685 0.00335 0.00155 0.00790 0.00240 0.00295 0.00240 0.00905 0.00150 0.00020 0.00025 0.00180 0.00135 0.00290<br /> [17] 0.00156 0.00805 0.00815 0.00150 0.00120 0.00075 0.00115 0.00095 0.00460 0.00195 0.00415 0.00210 0.00550 0.00185 0.00275 0.00065<br /> [33] 0.00085 0.00265 0.00170 0.00075 0.00005 0.00280 0.00225 0.00090 0.00012 0.00065 0.00210 0.00065 0.00047 0.00030 0.00200 0.00295<br /> [49] 0.00000 0.00455 0.00225 0.00218 0.00500 0.00007 0.00203 0.00370 0.00078 0.00010 0.00015 0.00545 0.00055 0.00425 0.00175 0.00175<br /> [65] 0.00140 0.00210 0.00140 0.00095 0.00060 0.00170 0.00055 0.00460 0.00010 0.00000 0.00185 0.00140 0.00145 0.00005 0.00250 0.00180<br /> [81] 0.00215 0.00320 0.00075 0.00030 0.00000 0.00215 0.00240 0.00185 0.00320 0.00000 0.00060 0.00405 0.00200 0.00080 0.00030 0.00360<br /> [97] 0.00105 0.00150 0.00710 0.00015 0.00105 0.00260 0.00090 0.00485 0.00010 0.00520 0.00475 0.00370 0.00310 0.00235 0.00200 0.00130<br />[113] 0.00150 0.00085 0.00420 0.00005 0.00355 0.00085 0.00050 0.00010 0.00160 0.00120 0.00075 0.00046 0.00085 0.00135 0.00055 0.00126<br />[129] 0.00015 0.00015 0.00015<br /><br />&gt; formatC(100*abs(MT4Prof - RProf) / abs(MT4Prof), format="fg")<br />  [1] "30.28"             "19.66"             "523.1"             "20.45"             "21.65"             "69.13"<br />  [7] "37.93"             "23.62"             "27.88"             "47.63"             "5.457"             "3.448"<br /> [13] "18.9"              "4.39"              "48.44"             "14.66"             "307.6"             "102.2"<br /> [19] "140"               "5.825"             "26.47"             "7.143"             "1.19"              "21.6"<br /> [25] "10.25"             "10.86"             "5.647"             "73.91"             "73.87"             "20"<br /> [31] "9.722"             "6.148"             "10.91"             "4.308"             "5.864"             "6.829"<br /> [37] "18.46"             "54.01"             "25.1"              "3.982"             "6.476"             "16.13"<br /> [43] "11.51"             "2.287"             "2.081"             "9.524"             "10.8"              "16.76"<br /> [49] "107.1"             "10.98"             "23.75"             "23.81"             "12.11"             "155.6"<br /> [55] "46.59"             "57.94"             "30.77"             "2.024"             "35.45"             "36.04"<br /> [61] "800"               "23.12"             "0"                 "53.39"             "102.9"             "3.271"<br /> [67] "51.09"             "16.28"             "34.85"             "26.32"             "55.38"             "54.65"<br /> [73] "0.3552"            "7.66"              "60.75"             "95"                "466.7"             "51"<br /> [79] "8.046"             "7.865"             "35"                "158.1"             "5.202"             "2.344"<br /> [85] "37.07"             "40.81"             "2.813"             "36.33"             "47.76"             "2.797"<br /> [91] "202.2"             "96.03"             "7.818"             "37.93"             "29.07"             "17.85"<br /> [97] "318.8"             "162.3"             "64.95"             "8.491"             "13.78"             "55.74"<br />[103] "12.92"             "89.62"             "196.3"             "265.3"             "20.39"             "30.77"<br />[109] "2.693"             "12.07"             "36.07"             "16.87"             "29.55"             "54.47"<br />[115] "27.85"             "47.37"             "31.88"             "6.995"             "6.63"              "6.883"<br />[121] "37.33"             "5.028"             "6.971"             "3.858"             "9.462"             "6.4"<br />[127] "20.94"             "178.5"             "0"                 "0.000000000001283" "1.641"<br />&gt; max(cumsum(ret$weight[2:l] - as.double(paste(SIT$trades.weight))/100))<br />[1] 0<br />&gt; ###########################<br />&gt; # Order Type Comparison<br />&gt; ###########################<br />&gt; max(cumsum(ret$weight[2:l] - as.double(paste(SIT$trades.weight))/100))<br />[1] 0<br /><br />

Conclusion

  1. MT4 execution prices could not be explained from the Tick Data, even when accounting for the fixed spread (2pips), although such execution price is close to Bar-Open (+/- spread).
  2. SIT uses Bar-Close as the execution price for both Entry Price and Exit Price.
  3. MT4 and SIT Profits do not match in this testing due to (1) and (2).
  4. MT4 and R Order Sequence does match perfectly
  5. MT4 and R Entry Time are really close, at most separate by a Bar (a hour in our case).
  6. Based on (4) and (5) I would suggest that R “has the potential” to serve as an accurate backtesting framework, although SIT and Quantmod, used as-is, are not optimal.
  7. THE QUALITY OF THIS BACKTESTING SET-UP IS SUB-OPTIMAL – our primary objective was off as it assumed MT4 99% tick data backtesting was accurate, but no evidence of such assumption was found.

Running a Backtest in R on Raw Tick Data

As you may think, I was really not happy with this situation. So I decided not to leverage a high-level R framework such as SIT, but to directly get to R’s Quantmod framework.

I wrote a simple R function to load tick data into R xts objects. For that purpose I have created with TickStory a one day tick data file with the following format: {Timestamp:yyyyMMdd} {Timestamp:HH:mm:ss:fff},{BidPrice},{AskPrice},{BidVolume},{AskVolume}

loadDukaTick <- function(file = "Forex//TestData/GBPUSD_GenericTickCommaDelimited.csv") {
  tick <- read.csv(file, sep = ",")

  # TEST DATA: https://drive.google.com/file/d/0BxMQxUqd263pTXpTb1plN1AtTm8/edit?usp=sharing

  # Tick Data Format - verify with TickStory when exporting
  # Timestamp Bid.price Ask.price Bid.volume Ask.volume
  # 1 20090111 21:00:17:245   1.51295   1.51345        4.2        1.2

  # set options to format the R Session with milliseconds - https://stat.ethz.ch/pipermail/r-help/2007-June/134006.html
  options("digits.secs"=6)
  # Sys.time()

  # Answer 2: http://stackoverflow.com/questions/13613655/colon-in-date-format-between-seconds-and-milliseconds-how-to-parse-in-r
  # To emphasize that a bit, %OS represents the seconds including fractional seconds ---
  # not just the fractional part of the seconds: if the seconds value is 44.234, %OS or %OS3 represents 44.234, not .234
  # So the solution is indeed to substitute a . for that final :
  ## gsub(":", ".", tick[,1])

  # Create zoo objects and append in series - http://shemz.wordpress.com/2013/02/15/download-fx-tick-data-using-r/
  # tick[, 2:3] <-> Bid and Ask, discarding the Volume
  # heavily inspired from http://shemz.wordpress.com/2013/02/15/download-fx-tick-data-using-r/
  dates <- as.POSIXct(strptime(gsub(":", ".", tick[,1]), "%Y%m%d %H.%M.%OS"))

  TickData <- zoo(tick[, 2:3], dates)
  tick <- NULL
  tick$bid = as.xts(TickData[, colnames(TickData) != "Bid.price"])
  tick$ask = as.xts(TickData[, colnames(TickData) != "Ask.price"])

  return (tick)
}

Then I wrote a very simple Quantmod backtest on these.

BasicQuantmodBacktest <- function (price){
  load.packages('quantmod')
  # http://blog.fosstrading.com/2011/03/how-to-backtest-strategy-in-r.html
  ## dvi <- DVI(price)
  sma.fast = SMA(price, 20)
  sma.slow = SMA(price, 50)

  # create signal: (long (short) if DVI is below (above) 0.5)
  # lag so yesterday's signal is applied to today's returns
  ## sig <- Lag(ifelse(dvi$DVI < 0.5, 1, -1))
  sig <- Lag(iif(cross.up(sma.fast, sma.slow), 1, iif(cross.dn(sma.fast, sma.slow), -1, NA)))

  FirstOrder <- which(!is.na(sig))[1]
  # Order <- which(!is.na(sig))

  for (i in 1:(FirstOrder-1) ){
    if (is.na(sig[i])) sig[i] <- -sig[FirstOrder]
  }

  for (i in (FirstOrder+1):length(sig)){
    if (is.na(sig[i])) sig[i] <- sig[i-1]
  }

  # calculate signal-based returns
  ret <- ROC(price)*sig #roc <- x/lag(x, n, na.pad = na.pad) - 1
  # FOR TESTING: subset returns to match data in Excel file
  # ret <- ret['2009-01-11/2009-01-11']
  eq <- exp(cumsum(na.exclude(ret)))
  plot(eq)

  # use the PerformanceAnalytics package
  # install.packages("PerformanceAnalytics")
  require(PerformanceAnalytics)
  # create table showing drawdown statistics
  table.Drawdowns(ret, top=10)
  # create table of downside risk estimates
  # table.DownsideRisk(ret)
  # chart equity curve, daily performance, and drawdowns
  charts.PerformanceSummary(ret)
}

Finally, those scripts run with

data = loadDukaTick();
BasicQuantmodBacktest(data$ask)
#####
# OR
#####
data=new.env();
TickStory2SIT("GBPUSD", "hour", data, file="./MT4ReportStatementAnalysis/TestData/GBPUSD_GenericBarFormat2009-2010.csv")
bt.prep(data, align='remove.na', dates='2009::2010')
prices = data$prices
BasicQuantmodBacktest(prices);

GBPUSD_2009-SMACross which we compare to this in MT4 GBPUSD_2009_SMAcross

That’s it for today. Definitely not perfect, but as I’ve learnt through research, sometimes it’s good to publish something that did not work properly for the community to learn from undesirable methodologies, inefficient testing and, in our case, improper tools used for testing. From there, I will spend a little more time on defining the optimal behavior of a backtest engine with these tick data and such simple strategy. In truth, I’ve already looked at different python framework as my little finger’s telling me that R+python might be a good option.

HTH – keep me posted on your thoughts folks.

PS: As I was looking for importing the Dukascopy data into R, I’ve stumbled upon a R-parser for Gain Capital Forex data – If time is on my side, I’ll have a look as well.

MT4 -> Multi-R sessions for tick-analysis

The Shared-Memory between multiple R sessions mentioned in my previous post got me thinking … quite some potential indeed. As a result, I investigated further using (calling) multiple R sessions from the same MT4 script. Specifically, I wanted to have a clearer understanding of the time required to performed lightning fast & dead slow processing, while being in the same MT4 script and without preventing real-time tick analysis (that is HFT potential at least on the receiving+processing side).

I drafted the following scenario to compare performances:

  1. Single R session called at every tick-update, displaying Bid/Ask @tick-update and using ChartSeries to plot either 500 bars or 10,000 bars – Bars are generated in the Init() function.
  2. 2 R sessions called sequentially, first R session displaying the Bid/Ask @tick-update, the second using ChartSeries to plot either 500 bars or 10,000 as previously
  3. 2 R sessions, but where the second R session is executed asynchronously, the first session plotting the Bid/Ask in @tick-update, the second using ChartSeries to plot 500bars/10,000bars.

The 10,000bars chart is used to further amplify potential anticipated observations made with the 500bars (that is a delay, or even a loss, of ticks due to the time required to “sequentially” render the chart). The two major area of concerns I had were the cooperation of 2 R sessions within the same script and the second the time required to perform the asynchronous call.

RESULTS are presented thereafter; y-axis is the number of ticks (GbpUsd) and x-axis is the time required to perform a complete MT4 Start-loop in [ms].

CONCLUSION: I have been very surprised by the behavior of the Asynchronous mode of MT4R. The cost of checking if an Asynchronous call is pending of finished prior re-issuing another Asynch call is about 1ms. This, added to the 1-2ms required to display Bid/Ask @tick-update depicted an extremely fast data processing system.

Although I’m not too happy about the set-up as I think a c++ multithreaded (or mpi) would have been better (through Rcpp or else), I can’t really discard the obvious that based on these simple tests, the R Asynch multisession set-up seems perfectly feasible for a Live Trading System. I even think I will push the limits to testing HFT trading, for which I will probably have to recompile Blas and Atlas anyways.

Finally, upcoming strategies should be design so as to separate real-time processing from asynch processing. As well, I did not yet evaluate the time required to send trade signals from R -> MT4, neither did I evaluate the Broker time to accept such signal (what’s the point of processing data at 2-5ms if trade signals are executed at 250ms)

1 with 500bars: .                   1 with 10,000bars:

————————————————————————————————————————————————————————————————————————————–

————————————————————————————————————————————————————————————————————————————–

2 with 500bars: 3 with 10,000bars:

Multithreading in R (or other types of non-sequencial programming)

Considering forwarding tick data from MT4 to R requires less than 2ms, but that charting 4 different time-frame (1min, 15min, 30min and 1hour) at each tick-update may require more than 250ms (depending on the number of bars in history), I think it is fundamental to investigate further in different types of non sequential programming and R, specifically multithreading or parallel programming.

Indeed, although charting is perfectly useless for our purpose of algorithmic trading, it is a valuable example to demonstrate a main loop that still receives tick updates at a rate of few milliseconds, while another “thread” would chart different time-frame every 1/2 seconds, without impacting the real-time gathering of ticks.

Along with the evaluation of MT4->R data transfer investigation, this is another major area to clarify prior stepping into the system design world (the fun part), considering our goal is to achieve a high-quality trading platform with Open Source Softwares.

Currently, I’m investigating:

  • Parallel computing in R through Rmpi, Snow. I found that particular example applied to optimizing an algorithmic system on backtested data at Revolutionalytics. This comforts me in saying that I’m probably following the right path. A list of High performance R computing can be found here.
  • Multi-threaded strategies through Rcpp
  • Shared-Memory between multiple R sessions.  The author mentioned the motivation of his example to be the developement of a High Frequency Trading systems. For that purpose, R nws package can be further investigated.
  • Again, one may also use socket connections through RServer.

On the Shared-Memory between multiple R sessions, I wanted to add that this strategy may support the development of multicurrency trading systems. Each currency would have its own Metatrader EA that at least reports the Bid/Ask continuously for that given PAIR, in a Shared-Matrix space …. actually quite a lot to think about but i shall not yet go that way ;-)

Transfering MT4 quotes in R

The more I get into this, the more I adjust my objectives. Currently, I’m willing to make the most of the Quantmod and Blotter R-packages, while eventually building a MT4 equivalent of the InteractiveBroker-R_API. As a first step, I was willing to Chart ticks from MT4 in R using the Quantmod’s chartSeries()

There are different options to export ticks from MT4:

  1. Using the DDE – the major drawback, if I’m not mistaken, is this is an export-ONLY mechanism. Considering I’m willing to use R to generate the trade signals, the DDE is not an option.
  2. Using sockets on 127.0.0.0 – the major issue is the extra-work required to design the communication protocol. However, there are some interesting sockets ideas. I believe this is a similar, apparently efficient, COM/DCOM server design.
  3. Through a database interface such as MySQL.dll for MT4 – on the bright side, it allows for the easy management of tick data (using timestamps as main key), but I believe this would bring quite some extra-complexity and speed issue. Here is an example.
  4. Using an EA/Indicator with the R.dll
  5. Using an EA/Indicator with any [custom].dll

I have decided to follow the 5th alternative refered to as the MT4R interface library, specifically that proposed by 7bits on ForexFactory. 7bits’s interface starts Rterm.exe as a background process and communicate with its command line via stdin/stdout. Although I’m a little concerned by real-time efficiency (I read the following: For 1 minute bar trading, with limited computational overhead, R is a perfectly workable solution. Inside of that time-frame, things may get difficult), this seems a straight forward first-move, that has lots of benefits and may allow live-trading through R (providing one is not HFT or scalping).

Thereafter is a screenshot of my R-Chart from MT4 tick, along with a video of the overall performance regarding R-tick “real-time” performance (although my code is not yet designed for speed).

I have uploaded a video of both MT4 next to R-chart so as to have a sense of the R-charting delay: Megaupload link ( .swf playable in any browser). As you may notice, at some specific time R is lagging behind MT4. However, one should keep in mind that, at every tick, I pass the last 500 OHLC+Volume quotes from MT4 to R, whereas a more logical approach would simply append that last new tick. Nonetheless, this gives some insight in R’s performance once we’ll start performing real processing through R, to eventually lead to a trade signal.

From that first test, I was relatively concern by the ability to quickly send MT4 ticks to R in real-time. I decided to write the simplest transfer logger in that I simply send Bid/Ask prices @everytick from MT4 to R (assigning 2 double [variables] in R) while measuring execution time. Surprisingly I found out only 1-4 ms are required. Here is my logged-video Megaupload link. I quickly added a line to pass a Matrix[1000] that I initialized in the Init() that resulted in an average increase of 2ms (Megaupload link). I will further investigate different processing scenario to have a clear picture of MT4->R->MT4 live-trading set-ups.

I will be putting my simple code examples @ xmph.forex-googlecode.

Enhancing R Graphic packages

This will be the last thing I’ll post prior starting to connect R with Metatrader. Indeed, as you may suspect already I use this blog as my lab-book. The following might be quite general, but it fits in my overall ambition to build a personal quant-trading architecture as professional as I can.

Graphic visualization is a major part of a trading system design. To me, nice & clean graphs are as important as their content. One needs to appreciate looking at the information to understand it, or at least to spend some time trying !

Here are the few framework I found that could interact with R:

Finally, few web resources for graph examples:

Algorithmic Trading in R: Available Information

After digging through different blogs and info on the web I found an amazingly strong and developing community supporting R for specific applications in algorithmic trading. Thereafter I present some of the major information that I believe are essential to the novice R algorithmic trader, hoping this will reduce the learning curves of the few following that path:

  1. QuantMod: development, testing, and deployment,
  2. TTR: technical trading rules,
  3. PerformanceAnalytics: econometric functions for performance and risk analysis of financial instruments or portfolios,
  4. XTS: extensible time-series,
  5. Blotter/QuantStrat: Transaction-oriented infrastructure for defining instruments, transactions, portfolios and accounts for trading systems and simulation – build, and back-test strategies
  6. R-QuantLib: R interface to the QuantLib library
  7. CRAN Finance & Econometrics packages

A nice summary can be found here.

Now I’m early in my project, which aims at using R to provide trading-signals to a Metatrader platform, but there is a specific information that I’m not yet able to find: Is R efficient enough (fast enough if you will) to allow live trading on a per-tick basis, per-second basis or per-? basis ? What is the ideal architecture to perform live trades: asynchrone R calculation for major analysis, combined to c++ functions and/or Metatrader functions to closely watch market opportunities? I was not able to find anyone detailing his/her Live trading set-up involving R which is a pity as a lack of design would result in basically restarting from scratch or living with inefficient set-ups. I guess I will have to experience the trial & error investigation.

“R” scripts for Metatrader ….. The first steps of many for proper trading

I have designed and developed Automated Trading Systems (ATS) for few years now. Since trading live, I have been fairly profitable (that is, I achieved my goals). However, I had the unsatisfactory feeling that I was missing some information to increase the level of trust in my systems … I was missing the proper tools to closely examine, and follow-up on the Real Equity Curve of an ATS.

I define the Real Equity Curve (REC) as the accrual of the

In-Sample backtesting Period + Out-Sample backtesting Period + Paper Traded Period + Live Trading Period.

There are quite a bit of information on the web for such but I did not find any Free Open Source tools to perform such assessment, specifically for Metatrader. Indeed, I do use MT4 for trading my strategies, and it’s fine with me; no issue nor any concerns. You might think otherwise but I will not go there … that’s far, far from my current objective.

Moving from one thread to another, I found this particular one: Synthetic hedges, cointegration, mean reversion and similar stuff (related to: MQL4->R-project – Interface Library).     Well, this was more than I needed to finally get a kick-start attempting to fill my needs for REC analysis.

I have started a GoogleCode SVN which contains my R scripts along with example files. My objectives are to design and implement the following:

  • Analysis of multiple MT4 StrategyTester [.htm] files … as I realized most, if not all, the software out-there are commercial solutions
  • Analysis and Follow-up of ATS through Real Equity Curve evaluation
  • Provide a detection for ATS performance Break-Down … to determine as soon as possible when things get ugly

In a second session, and because I stumble across few interesting information here and there, I will be looking at the following:

  • Strategy Optimization of MT4 ATS through R
  • Enhanced the analysis of ATS performance, through System Consistency evaluation for example
  • Enhanced the REC evaluation with Monte-Carlo analysis and Bayesian Statistics
  • and eventually combine the above to investigate Computer Generated ATS (CGATS).

As a first start, thereafter I present 2 examples of my programming kick-start: (1) The aggregation of multiple StrategyTester reports and, (2) the visual follow-up of an optimized MT4 Moving Average from backtesting to paper-trading. Every information and material needed can be found at http://code.google.com/p/rscripts4metatrader/

(1) Using: MT4_StratRep_Analyzer()

(2) Using: MT4_RealEquityCurve_Analysis ()

Metatrader Optimization on the Moving Average EA for the period 2008.01 -> 2009.01

3    2408.02    694    1.27    3.47    784.52    18.86%    MovingPeriod=60    Lots=0.1     MaximumRisk=0.02     DecreaseFactor=3     MovingShift=6