Replay Data not accurate

[MJT]

I have been running Market Replay Recorder for a few days and notice different values when replaying the data. For example:
ER2 on 5-minute chart. Session time 9:30AM to 4:05PM (Eastern Time which is time zone for computer). Using IB data feed.
May 31 0935 bar:-
Replay data; O=849.4 H=850.2 L=849.2 C=849.7
Historical Data; O=846.3 H=847.0 L=845.7 C=846.1
June 1 0935 bar:-
Replay data; O=855.9 H=856.4 L=854.9 C=855.7
Historical Data; O=851.8 H=854.4 L=850.5 C=853.3

The difference is not just on the first bar of the day.
For replays I am connecting to relevant replay data, then openning a new chart with Days back set at 1. The historical data above matches the realtime chart on each of those days as best as I can tell.

Any ideas on what I may be doing wrong?

[NinjaTrader_Dierk]

I suppose you are comparing
a) replay data recorded from a IB live account
b) historical data pulled from IB

Since we do not know how IB records their data I suggest eliminating that variable. Please:
- record replay data from your IB live account and leave e.g. a 1 minute chart running while recording
- take a screenshot of your IB 1 minute chart
- replay the same period with the recorded replay data with a 1 minute chart opened
- compare replay chart with your screenshot

Should be the same

Also: Please make sure you are on latest NR6R2 (6.0.1000.2).

[NinjaTrader_Dierk]

Also: IB's realtime data is snapshot data and is not all trades/bid/ask which occured. Their historical data likely is built from a different data source (since e.g. they would need to see all trades for proper volume). This for sure would result in what you observed.

[zoltran]

Well ... There's a problem here I think.

Doesn't NT reload the days data when you open charts or MA's ? So there's a real good chance you'll get mixed IB historical and RT/Collected data as you open/close charts.

At least that's the way it seems to work for me.

[NinjaTrader_Dierk]

Nope, there is no mixed data on the day which is replayed. NT will not load from a historical data provider for the replay date but only "build bars" from replay data.

[MJT]

As suggested I did the following:
>> Since we do not know how IB records their data I suggest eliminating that variable. Please:
>> - record replay data from your IB live account and leave e.g. a 1 minute chart running while recording
>> - take a screenshot of your IB 1 minute chart
>> - replay the same period with the recorded replay data with a 1 minute chart opened
>> - compare replay chart with your screenshot

I have attached my results over 2 days - June 5th and 7th (on 6th IB had data feed problems). June 5th is as expected with only small differences in open/close which could be due to IB not sending every tick.
However on June 7th it appears approx 16 minutes missing, then not well syncronized thereafter.

There was a difference in my operations between these days.
On June 7th (as normal) I was recording data (DAX) before US market open, but there was a BasicEntry window open for ER2. Just after 0900 eastern time I close down everything, reboot computer then sync computer clock before reopenning all programs and reconnect to IB.
On June 5th, I had not been connected to IB and was not recording data before the reboot routine, so there was no break in the recorded data.

Could this break and computer reboot be the cause of the replay data problem I am seeing.

thanks,

[NinjaTrader_Dierk]

- you only can compare as your PC keeps going without reconnect/restart, since this would pull data from IB's historical data server adn screw up the test -> 5th is fine, 7th is worthless
- replay timestamps do not match live timestamps down to the millisecond, since the granularity on recording is 1/4 of a second -> results in slight differences you'll see as you compare charts from 5th

[MJT]

Not sure I understand Dierk.

Are you saying my test of recorded data on 7th is not valid, or confirming that anytime there is a break in recording with logoff/on to IB the recorded data is no good?

The reboot was before the session start time of the charts I posted, so no break in recording occurred during the charted time period. I can't see how IB's historical data interferes with recorded data, and even on 5th there would be historical data come in when I connected to IB.

[NinjaTrader_Dierk]

The problem is: as you connect, NT will pull data for the current session from IB servers and overwrite your chart data (which was recorded by NT). The replay data is a different (!) pool which is not affected.

However, the chart will show data pulled from IB and not the data you recorded to the replay pool before reconnect.

To clarify: IB's historical data never can match your recorded replay data, since replay data is built of realtime data which on IB only is snapshot data (every 0.7 secs) and will not give you all ticks.

[MJT]

Sorry Dierk, please bear with me as I'm not getting this.

The Realtime data comes in from IB and is being recorded which will be called Replay data. These data sets are identical as long as I don't reconnect to IB and have my chart data overwritten with historical data pulled from their servers.

Assuming I have that correct, lets go back to the charts I posted.
The Realtime chart shows the last few bars of the prior day session, then shows the realtime data for the new day starting at 0931 bar. The screen shot was taken that morning with no reboot of computer since the start of the new session.

The Replay chart was produced the following day. On openning chart, historical data is loaded up to the end of the day prior to the replay file I am connected to (the same few bars can be seen at the left of the chart). Then replay data is fed to the chart. This replay data should be identical to that seen on the Realtime chart, as they were the same thing at time of recording.

I am failing to understand the issue of historical data from IB when I reboot computer at 0900, but do not chart any data on either chart until 0930.

I hope that makes sense to you.

[NinjaTrader_Dierk]

>> The Realtime data comes in from IB and is being recorded which will be called Replay data. These data sets are identical as long as I don't reconnect to IB and have my chart data overwritten with historical data pulled from their servers.

Correct.

Sorry, I don't know why 7th is different I only could guess: my guess would be that "somewhen" historical data was pulled from IB e.g by reopening a chart or reconnecting to IB on the 7th.

[dargente]

Dierk, can you provide update on your quote below. Is granularity same for NT7 at 1/4 second or is it now equal to live timestamps? :)

"replay timestamps do not match live timestamps down to the millisecond since the granularity on recording is 1/4 of a second"FYI I am still using legacy version and in the process of upgrading.

Additional details:
The strategy i am running is based on OnMarketData().
Market replay is giving me different results (better) than live simulated trading.
I have set Real-time data - record for market replay (from IB live feed).
Same exact results between v 6.5 and 7 in the market replay, so no problems there.

Shouldn't a strategy have the same results in market replay or a live simulation? Please correct me if i'm wrong, but they should have the same exact data-feed granularity, correct?

Also, can you let me know if this quote below is still true:
realtime data which on IB only is snapshot data (every 0.7 secs) and will not give you all ticksI highly doubt it because I see the ticks come in faster than .7 seconds - i don't believe it is snapshot anymore; just wanted confirmation since the quotes i am referencing are couple years old.

Thanks

[NinjaTrader_Austin]

dargente, I will have Dierk get back to you on Monday.

[NinjaTrader_Austin]

dargente, Dierk has informed me that nothing has changed in this area from NT6.5 to NT7.

[dargente]

Sure no problem, I'll wait for his response (or other NT) on Monday. I posed a few questions and still trying to find out why the market replay is not giving same results as live simulation. Some trades are exactly the same, others are at totally different times.

[NinjaTrader_Austin]

dargente, I'm not sure if you saw my most recent response (post #14), but Dierk has provided me with an answer to your question and will probably not respond himself.

The answer is: nothing has changed in this area from NT6.5 to NT7.

[dargente]

I would appreciate if someone else over there at Ninja Trader besides Austin would approach my questions.

My main concern is not differences between 6.5 and 7. I have just upgraded to NT7 and i see that I am getting the same results.

However, my inquiry is regarding the differences between market replay and live simulated results. What differences should I expect between the two if any? otherwise its something else that I need to look at which is affecting the performance. Market replay is giving me many of the same trades but some other trades as well, boosting my performance results considerably.

Also, the other questions were concerning the quote regarding IB realtime data being a .7 second snapshot and doesn't contain all trades/bid/ask which occured. Is this still the case?

[NinjaTrader_Josh]

dargente,

Market Replay vs. live simulated vs. live real are similar, but are all "different".

For instance, depending on how your indicator/strategies are programmed and what exactly is entailed in their logic you may run into limitations of L1 and L2 data in Market Replay which you would not have in live simulated. To read up on this please see the "Understanding how the Market Replay works" section here: http://www.ninjatrader.com/support/helpGuides/nt7/replay.htm In a nutshell basically the syncing of the events together may not necessarily be the exact sequence you receive if you were live simulated or live real. That could play a factor in getting things like limit fills, etc.

For the IB question I am unaware of anything changing on IB's end in this regard. It may be best to clarify with IB directly though to get firsthand information on what exact kind of data they are offering.

[dargente]

Josh thanks for the link, I will look into it. However, what I just realized after looking at my strategies today is that it appears Market Replay is not following my entry criteria. I have tested my strategy 3 different ways today as follows:

1. Live using an IB simulation account.
2. Live using the NT simulation account.
3. Market Replay.

Results:
Both simulation accounts gave me pretty much the same results. However, Market Replay is overlapping trades, even though I have setup the criteria in every circumstance as follows:

EntriesPerDirection = 1;
EntryHandling = EntryHandling.UniqueEntries;

Market Replay is allowing more than 1 entry per direction. Can you look into this or do you have any idea maybe something on my end that I need to reconfigure?

Thanks, and btw congratulations with the release of NT7, its really impressive. If i can get the same results live as i do replay, i will be very happy. :)

--fyi i am using OnMarketData(MarketDataEventArgs e) only, no level 2, if that makes any difference.

[NinjaTrader_Josh]

dargente,

First things first, to set expectations, Market Replay should not be considered as definitive results that you would receive in live, real trading. No matter how good simulations are, it's never exactly the same in real-life.

As for what your strategy is doing in Market Replay I suggest you slowly debug it and step through the code as it runs through the replay to see what exactly you are facing. You will need to use Print() throughout your code and trace your orders to fully understand what is happening.