Problems with Market Replay

[dcobbold123]

Hi,

I've recorded some replay data but I'm having some trouble playing it back. The problem is that the bid/ask values change correctly but the last price is stuck as some other value not even close to the real price.

e.g. I'm replaying today's WDC action and by 09:34:02 the price had moved down to 33.50/33.51 but the chart is stuck at 33.65. The bid/ask on the T&S window is at the correct value but the tape is full of trades at 33.65.

After a while it will suddenly correct itself and the charts will jump to the correct value, but this can take a long time (in fact some charts never correct). This obviously makes it very difficult to paper trade.

Any suggestions?

Thanks,
David.

[NinjaTrader_Dierk]

The replay feature replays data recorded as submitted to NT by your provider. If replay data is off, then your provider did submit this "off" data before. Unfortunately there is nothing NT could do.

[dcobbold123]

damn IB feed. i guess it's time to get a real one.

[dcobbold123]

Hi,

I'm still getting this problem and I've done a little experiment today, and I'm now not sure if the problem is with IB.

Basically, I set up ninja trader to connect to Trader Workstation and show about 15 high to mid volume 1 minute charts. I also opened the same 1 minute charts in TWS and left them both running today.

I came back to them after work to compare the difference, and the problem doesn't exist in TWS which makes me think that IB's data is fine. The problem seems to be either with NT or with the API that IB provide.

I've uploaded some examples to help illustrate my problem. It's making replay mode completely unusuable because orders are filled at the wrong times and at the completely wrong price.

NinjaTrader chart:

http://img84.imageshack.us/my.php?image=ntxn1.jpg

TWS chart:

http://img84.imageshack.us/my.php?image=twsax4.jpg

Let me know if you have any more suggestions.

Thanks.

[NinjaTrader_Dierk]

Not sure what I'm looking at here, but if the NT chart is generated of replay data (=live data recorded from IB before), then you are probably comparing apples and oranges: IB live data is snapshot data (=ticks are filtered). There is no guarantee that their historical 1 minute really is built from the filtered, snapshot data which NT gets and records for replay.

[dcobbold123]

the two charts are the same stock over the same period. The chart isn't using replay data, it's the chart built from watching a live feed over the day... Although the same chart would be produced if I replayed it.

I understand IB data is filtered, but I dont see how this could explain the difference between TWS and NT charts? If IB is building the charts over the day with no problems, and NT is using the same data, how come there are big gaps only in the NT charts?

The areas highlighted are times when the last price didn't change in NT, even though the Bid/Ask follow the price properly. I've highlighted both charts so you can see that TWS displays the data properly.

[NinjaTrader_Dierk]

>> it's the chart built from watching a live feed over the day
correct and that's why they are different. NT chart = realtime data to build a chart, TWS chart = TWS own historical data.

Both data sets are different and would result in different charts.

[dcobbold123]

But the TWS chart was left open during the day along with the NT chart, so just like the NT chart it was built using realtime data not historical data.

[NinjaTrader_Dierk]

Quick test: Close NT chart and open it again while connected to TWS. Does it look the same like TWS? If so, then both charts now (but not before) are built from IB's historical data.

[dcobbold123]

If I reopen NT and view the chart again it is backfilled properly and displays correctly.

But that's the problem, I need the realtime data displayed by NT to be the same as the realtime data in TWS, so that I can rely on the recordings being accurate. There's no problems with historical data.

[NinjaTrader_Dierk]

Unfortunately there is nothing which could be done since IB's live feed does not match IB's historical data.

[dcobbold123]

ok, I thought that may be the case. IB have recommended that I make sure that only java 1.5.0_15 is installed so i'll try that today. I'll also try connecting quotetracker to TWS as well so that I can make sure that it has the same problems.

[dcobbold123]

>> Unfortunately there is nothing which could be done since IB's live feed does not match IB's historical data.

I've carried out the same test using TWS, NT, and QuoteTracker displaying the same realtime charts at the same time. QuoteTracker and NT are both connecting to TWS for the data.

What I'm seeing is that NinjaTrader is displaying certain charts with gaps missing (as shown in the example images i posted earlier), but QuoteTracker is displaying the same charts correctly with no gaps and no problems.

Since they're all using realtime data to build the charts, I dont think it's a reasonable conclusion to say that the data TWS is sending out is incorrect, since QT has no problems with it.

Is there any chance this can be looked at in more depth?

I prefer NinjaTrader because it has the best looking charts out there, and theoretically the Replay function is extremely useful, but at the moment it's pretty much unusable, and I think the signs are saying that the problem lies with NT.

I've also tried ib's suggestion of using Java 1.5.0_15 but still no improvement.

[NinjaTrader_Dierk]

Unfortunately we're hitting the wall here. We implemented the IB data feed interface using all info available (official and inofficial), thus we would not know what causes these issues.

[dcobbold123]

ok, i guess i'll start looking for an alternative to NT then.

I've got to say, i'm pretty amazed at NT's blasé attitude towards bugs... from my experience it seems that the first approach is always to try to dismiss any problems.
I work for a leading provider of trading software, and if I dismissed blatant bugs in the same way as NT I'd be sacked. Sure, you've created an interface going on all available knowledge... but it has problems with it. The proper approach would be to look into it and communicate with IB, rather than just say 'well we tried our best, we can't do any more'.

Anyway, over and out. Despite all it's recent improvements NT still has big problems with... I look forward to a version 7.