Daily Data False when Merge Policy set to MergeBackAdjusted

[Harry]

This occurs with NT 7.0.0.6.

Setting merge policy to "MergeBackAdjusted" will merge intraday charts correctly - as far as I saw. However, the daily data is not merged correctly, but for ES 03-10 the merged contract shows data that is about 17 points lower for today. So it is completely off-track.

To reproduce this, you only need to load daily data of ES 03-10 and ES 12-09 and set merge policy to "MergeBackAdjusted", when not connected.

[NinjaTrader_JoshP]

Harry,

Please check your offset settings in the Instrument Manager. Then check daily charts of ES 12-09 and ES 03-10 directly. The offset used on intrabar charts not necessarily will match your daily chart data.

[Harry]

Do not understand

Hi Josh,

thanks for answering. I am afraid you did not understand my question.

If I set merge policy to "MergeBackAdjusted" this will both change intraday and daily data of ES 03-10. Intraday data is correctly merged whereas daily data of ES 03-10 is completely false afterwards.

The offset I use under instrument settings is the standard offset that is downloaded from your servers, for ES 03-10 it is -5.25 and I did not change it. Anyhow if I change it, it will be automatically overwirtten by your servers, even if I do not want it (another problem).

[NinjaTrader_JoshP]

That is what I am saying. There is a single offset value that matches your intrabar data. This same offset value does not necessarily match your daily data and when used on your daily data it will "look" wrong because simply, the offset value doesn't match.

You can have the price end at 1100 on intrabar and price end at 1101 on daily. This means the offset required to match intrabars is different than daily bars. You will need to check your daily bars data to see the proper offset amount needed for daily bar offsetting.

[Harry]

How can I set a different offset for daily data than for intraday data? Also the offset is different for every day?

Again to be clear: The merging does affect the daily data of the contract, so I am not talking about a matching problem, but a completely false merging procedure. This has nothing to do with matching, I am 18 points off, the offset is 5.25 points.

18 points of ES means that the daily data is at 1100, whereas intraday data is at 1118, that is a difference of 1.6 %!

[NinjaTrader_JoshP]

It is precisely the offset that is in question. The merging is very simple. Take the expiring contract's last price and the new contract's first price and calculate the difference. That difference is the offset. Take all prior expiring contracts information and bump it either up or down based on that offset.

Now go to your intrabar and daily data. They can have completely different information for these critical values. Ending price + starting price could require a completely different offset for daily than what was required for intrabar. This is why I am suggesting you check your data at these critical points. There is no way to set different offsets. You have one offset you can set. It is simply a data fidelity issue. You can use the Historical Data Manager and take a look to put an end to all speculation.

[Harry]

This is a bug!

Josh, we are not getting any further this way.

Please listen, I am trying to explain a bug. Have a look at the charts please!

(1) I have the same data for NT 6.5. and NT 7. For NT 6.5. the merging procedure works without problem.

(2) For NT 7 I get completely false daily data. Usually the offset should only be applied to the backadjusted contract (ES 12-09). The problem is that the daily data after the rollover date is completely false as well. Tell me: How can the offset affect the data after the rollover date?

I am quite sure that this is a bug, because the software leaves me no option to display data correctly!

[NinjaTrader_JoshP]

See attachments. Suggest you try reinstalling your NinjaTrader 100% cleanly and delete all your old data.

[Harry]

thanks for showing the charts, looks correct. Mine are false.

So if there is a difference, it is probably data or timezone related.

I just started checking the UNMERGED daily charts ES, and this came to my attention: While NT 6.5. displays daily data of Interactive Brokers correctly, NT 7 displays them with a date one day ahead (x-axis date, but not cursor date). The difference can be seen in the charts below.

After the merging procedure by NT 7 the bars (same shape) are displayed with the correct date but false values (see the 2 charts of my first post). It seems that NT 7 has a problem with the handling of dates and therefore creates a merged contract with false data.

This can be either related to the timestamps of Interactive Brokers and/or to the timezone of my PC, which is CET (Central European Time). The problem does not occur with NT 6.5., so I would say it is a NT 7 bug....

[Harry]

Checked this for other instruments now. A whole bunch of daily charts has wrong dates, including all CME and NYMEX futures, so it might be a compatibility issue between NT7 and IB.

[NinjaTrader_Dierk]

Harry, we already discussed several times that on your setup (timezone related?) IB would report incorrect dates to daily data on some instruments. I even sent issue reports to IB to no avail. As long as this is not addressed by IB, NT would be left with any potential confusion and malfunction in regards to that data.

Thus, I again suggest you considered an alternate data provider as it makes no sense to raise the same issues again and again (even from a different perspective).

Thanks for your understanding.

Merry Christmas.

[Harry]

Hi Dierk,

thank you for your answer.

This is a new problem with NT7. IB daily data for all US futures cannot be displayed. All intraday indicators relying on daily data will not work as well.

This bug is not identical to the known NT 6.5. bug (for TF) you are referring to!

I would appreciate, if NT 7 would read IB data just as NT 6.5. does!

Have a great Christmas!

Harry

[NinjaTrader_Dierk]

Let's eliminate variables. Here is what I just tried:
- deleted complete context of folder <my documents>/NinjaTrader7/db (you may want to create a copy before)
- my PC is Berlin timezone
- merge option set to MergeBackAdjusted
- connect IB
- no Default chart template
- load a 1-Day chart for ES 03-10 just 5 days back, rest default
- I see 4 bars on the chart, dates 21th through 24th
- Tools->Historical data manager->Edit: same dates are there

Do you see something different?

[Harry]

Hi Dierk,

thank you for answering (even late on Christmas day), just came back from my last shopping tour......

I did what you suggested: Deleted the db and started with a new db. I got 4 bars, albeit 22nd through 25 th.

Actually you will only see the false dates on the x-axis, when you expand the chart. The crosshair-cursor shows the correct date: cmp. screenshot.

Again, have a great Christmas, I guess this issue can wait a few days, as I am happily trading with NT 6.5.!

[NinjaTrader_Dierk]

Thanks for your feedback. Pls see attach what I see.

Ok, let's stick to that exact simple as possible scenario to understand why you see something different.