Accuracy of Replay vs. Actual Data

[jtfancher]

Attached is a brief file example of a big problem that I have with Market Replay data generating significantly different trades than the actual trading itself. The discrepancies are far beyond any slippage issues.

I am running two strategies on SIM - one on GC, CL, and TF. And another on 6e,6a and ZB.

The file shows trades from about 7:45 - 10:30 last night which was intentionally chosen as a slow period. During the more normal, higher volume hours the problem is greatly exaggerated with a Market Replay showing far greater profitability that actual.

My actual data is Zenfire/Mirus and the Market Replay data is what I get from File-Utilities-Download Replay Data.

Thank you.

[NinjaTrader_Bertrand]

jtfancher, unfortunately I could not open the file you attached for us here : could you please recheck the attachment? What simulator settings are you using then under Tools > Options > Simulator and with which speed did you replay?

Please also check this link here for overwiew of common discrepancies seen :

http://www.ninjatrader.com/support/h...ime_vs_bac.htm

[jtfancher]

Originally Posted by NinjaTrader_Bertrand

jtfancher, unfortunately I could not open the file you attached for us here : could you please recheck the attachment? What simulator settings are you using then under Tools > Options > Simulator and with which speed did you replay?

Please also check this link here for overwiew of common discrepancies seen :

http://www.ninjatrader.com/support/h...ime_vs_bac.htm

Bertrand - the file I was sending is Excel which doesn't seem to be supported directly so I zipped it. Here is a picture of my SIM settings. I have this problem no matter what speed I run.

[NinjaTrader_Bertrand]

jtfancher, yes, but if I open your zip there's only a link / shortcut incluced and not the actual underlying file.

The simulator settings are looking ok, are you trading any large size with this strategy? What order types do you use / simulate?

[jtfancher]

Bertrand - I am not sure why the file zips like that. I am going to send you an unzipped version to the support@nt address. I am only trading 1 contract each. Both use only limit and stop orders.

[NinjaTrader_Bertrand]

Ok, sounds good - I will check into it shortly.

[dojidubbs]

Any update on this issue that might help us folk looking at market replay in general?

[NinjaTrader_Matthew]

dojidubbs,

Bertrand will be following up on Monday.

[NinjaTrader_Bertrand]

dojidubbs, sorry I don't recall the specifics at this time - what issue are you running into?

[dojidubbs]

No specific issue. I was just looking at market replay in general, and saw this item. Perhaps if jffancher is still watching the thread, he can give us an update on why Market Replay was generating different trades than the actual trading.

Thx

[jtfancher]

Market replay gives me vastly different results versus trading when the markets are open. Ninja apparently makes assumptions about how fills take place. Often in replay, stop orders are filled at favorable prices versus my order level which virtually never occurs in real trading.

[temerson]

I am having the same issue with market replay on CL. It shows more positive slippage than negative slippage. I use a fixed stop loss of $40. Only long entries are used so all stops are "sell stops".

Avg Loss - $37.69 Note: If there was negative slippage, I would expect avg loss to be > $40
# of Trades - 860

Looking through trade data, most losing trades are at -$40. There is a bias toward positive slippage (ie -$30) as opposed to negative slippage (-$50). Is it realistic for me to expect positive slippage when I start trading this live?

[NinjaTrader_Austin]

temerson, please try running the strategy live on a simulation account and let us know what sort of slippage you run into - this will give a more accurate representation of trading live than market replay.

[temerson]

I ran the strategy 2 ways for 10-17-2011 with CL Nov contract:
1. simulation mode with interactive brokers as data feed
2. market replay with data downloaded through ninjatrader

There are huge discrepancies with slippage. Using sim mode with IB data feed shows an overall negative slippage, whereas market replay shows overall positive slippage. This results in a net loss of $350 in sim mode with IB and a profit of $260 using market replay. I'm not sure why market replay is giving overall positive slippage in all my tests; this seems unreasonable.

I have included the order grid with combined orders from market replay and sim mode in .csv format.

Any help will be much appreciated.

[NinjaTrader_RyanM]

temerson,

I show a lot of the same values in this example so overall looking pretty good. The simulator engine on Sim 101 is identical to the simulator engine on market replay.

The data is not the same on your live feed compared to market replay downloadable files. It's a different source for data, and the potential for latency or any upstream issues make it extremely challenging to match up identically.

If you wanted to playback the same data live in market replay, record on your local machine. Click Tools > Options > Data > Check Record for market replay. That way you're working with the same data that you previously streamed live to your computer.