Connectivity problems

[adamus]

I am unable to connect efficiently through IB TWS to the IB historical data server farm.

For a long time I was told that this was either my version of Java, or my version of IB TWS, or just down to my broadband and the internet.

Now however there is a fortuitous alignment of recommended versions for Java, IB TWS and NinjaTrader and it is the ideal opportunity to report here that NinjaTrader just isn't getting through to download data.

I'm using Java 1.6.0_21 on Windows 7 64bit. I've got IB TWS 906.8, and NT7 beta 20.

IB TWS has no problem downloading data itself. I can open a chart and the past data will download within 5 seconds.

However in NT, it won't download and it waits for minutes and then gives an error typically as follows:

Error on loading chart data for '$EURJPY Default': IB for historical data: HMDS data farm connection is broken:ushmds2a HDMSDataFarmConnectionBroken

When I go back to TWS and try to reload the TWS chart, I'll see the red disconnected alert but then it will immediately reconnect.

If I try anything in NT, it will disconnected after a long pause.

I'm running charts, market analyzer with indicators, and I have strategies enabled. They all want historical data before they will start. It's been such a long drawn out problem that I just subscribed to IQ-Feed for historical data instead, where it works perfectly.

[NinjaTrader_Jason]

Hello adamus,

As informed at the forum thread below, the error message comes from IB and indicates the connection did not work at that point in time.

http://www.ninjatrader.com/support/forum/showthread.php?t=29462&highlight=HMDS+data+farm

Please check with IB why you experience this error message to troubleshoot your issue.

[adamus]

Jason,

guess what I was doing in between May and now?

[NinjaTrader_Jason]

Ok, so what clarification did IB provide regarding the error message?

[adamus]

Ok, so what clarification did IB provide regarding the error message?

Here's what IB had to say on the issue:

Initial Description: HMDS data farm connection continually breaking
Response from rayros at 02-Sep-2010

Hello There,

Base on the log file, it appear that Ninja Trader was submitting invalid historical data request periodically. It is feasible that connection between TWS and NT immediately due to countless invalid request from Ninja Trader.

For further investigation, I strongly recommend that you connect TWS and Ninja Trader, however do not request any information in NT yet. Increasingly, request one contract at a time until receiving the disconnection message again. It is feasible that the core issue is only related to one of the existing contracts.

If you have more questions, please send an e-mail to [email protected].

Regards,
Raymund
IB API Support

So basically what he is saying is that I have to sit there and caress the hapless computer until it can't take it anymore, and then he might deign to look into it further. Is there anything more constructive and less time consuming I can do from a NT perspective?

[NinjaTrader_Jason]

Unfortunately there is nothing I can think of to troubleshoot the issue. I suggest to proceed as IB Support has informed you to further investigate the error.

[adamus]

That's tough because this problem has a very good work-around which involves subscribing to IQ-Feed for $80 a month - but for that, the historical data stretches back 5 times further and the download speed is 25 times greater.

The other problem with IB + TWS that I discovered is that while IB don't provide pure tick data as history, they do provide 1-second OHCL bars. The download manager doesn't want to know about 1 second bars though.

[NinjaTrader_Jason]

Yes, unfortunately historical tick data is not supported with IB. Please see the link below under 'Understanding the data provided by your connectivity provider'.

http://www.ninjatrader.com/support/helpGuides/nt7/historical_data.htm

[adamus]

Unfortunately there is nothing I can think of to troubleshoot the issue. I suggest to proceed as IB Support has informed you to further investigate the error.

IB are not being very helpful. I got more advice out of them:

I strongly recommend that you also consult with Ninja Trader vendor. It appears that their system was requesting historical data incorrectly. Please consult with examine the invalid request periodically.

Regards,
Raymund
IB API Support

.

[adamus]

And then this:

When you open the related log file, look for this information:

"Invalid time period, using default"

This means that the time or duration or bar size combinations are incorrect.

Regards,
Raymund
IB API Supprot

I checked in the log files for IB.

"Invalid time period, using default"

This appears in the log files for the last week over 2000 times.

I think the guy might have a point.

[NinjaTrader_JoshP]

adamus,

We would need to know exactly what was on that chart last week. What interval, time frame, and dates were used would be important.

Please try reproducing that message on your end and when you get it please note down what chart settings you were using to make the request and then we can take a look from there.

[adamus]

Josh,

more connectivity issues with IB TWS API and again IB Support are putting the ball in your court:

Initial Description: HMDS data farm connection continually breaking
Response from rayros at 28-Sep-2010

Hello Adam,

In this particular case, base on the message on the log file, your API program must validate the time, duration, and barsize combinations before requesting historical data.

The core issue is within your API program, not the TWS. I strongly recommend that you consult with the third party vendor to resolve the issue.

Regards,
Raymund
IB API Support

I can see this error "Invalid time period" appearing repeatedly in the IB TWS log. I will attach it.

I do two things which cause it: loading chart data with 5 min bars for any of the forex pairs in my default list (e.g. EURUSD, GBPUSD, USDJPY), and enabling a strategy running on any of those pairs. The chart had chart trader loaded. Any other settings you want to know?

[adamus]

Here's the IB log file.

And here's the link to the NT7 logs and traces: TracesAndLogs.zip

[NinjaTrader_Jason]

Hello adamus,

Could you please tell me what session template the chart has selected. How much days of historical data is loaded in the chart as well?

Did you have any indicators applied in the chart. If so, please tell us which ones and what settings they used. Perhaps it is easier to send a screenshots of the Indicator menu, Data Series menu and Chart Properties menu.

Am I correct the two things combined cause the issue or do each thing separately cause the issue?

[adamus]

It's the default forex session template and it's trying to load 3 days of data.

I had no indicators.

It is not a combination of the two things, it is just virtually impossible to access historical data from IB in any way. It is easier after 23:00 GMT+1. I don't know why.