No data from IB hangs strategy analyzer

[Pete S]

Seeing something for the first time this morning. I am running optimizer against an instrument list. When it gets to ASML, it hangs on 9/29. I have to abort, it will go no further. I see the following message in the NT log (see attachment):

Is it possible to just ignore this so the run can continue?

[NinjaTrader_Dierk]

We'll look into.

However, you still would have a problem due to IB throttling the data requests (which could not be overcome by NT).

[NinjaTrader_Dierk]

I just checked the code, it's like I suspected: You hit the wall on some IB internal throttling (log in green) and any further request for historical data was rejected by the log messages in yellow.

Just just need to wait I guess 5 minutes until the IB server allows you to process more data requests.

[Pete S]

Hi Dierk, thanks for your response. First of all: this is not a critical issue for me, for the time being I am just not using that symbol & that is fine, so I am not asking to have any time spent working on this.

I will say, however, that I am (unfortunately) very familiar with the IB pacing error and this is a different case. In this particular case, the request was accepted by IB, but IB itself responded there was no data for that day. Normally during a backtest run the error will be ignored and the run will continue.

[NinjaTrader_Dierk]

Thanks for clarification. However, what's the trouble you experienced?

[Pete S]

The backtest run hung at that point. I had to abort out of it. Like I said, normally it will just go ahead and finish up, and if there is data for the following symbols, they will finish fine.

[NinjaTrader_Dierk]

To double check: you ran a backtest for a symbol where there's no data on the IB servers, no?

- what symbol was that?
- what interval (1 minute ? else?) did you backtest?

[Pete S]

ASML, would have been either 15 minute or 60 minute.

[NinjaTrader_Dierk]

Sorry, there is data for ASML on IB's server (I just tested).

[NinjaTrader_Dierk]

Please make sure you have set up ASML with Default *and* Nasdaq exchange (see instrument manager).

[Pete S]

To reproduce, reload historical data with these exact settings (see attachment), happens even with Nasdaq selected.

[NinjaTrader_Dierk]

This will not work. You would need to add the stock in the instrument manager as pointed out below (this is an IB limitation).

[Pete S]

Sorry, I should have mentioned I had already done that, even prior to your instructions.

[NinjaTrader_Ryan]

Hi Pete:

Thank you for posting.

I will work with Dierk and we will try to re-produce and report back to you.

Enjoy your day.

[Pete S]

I just want to reiterate this is not a big deal for me; please prioritize accordingly.

[NinjaTrader_Ryan]

Hi Pete:

I was able to investigate a little further into your inquiry this afternoon.

Please check for a response from Dierk or myself tomorrow.

Have a great day and happy trading. :)

[NinjaTrader_Ryan]

Hi Pete:

As per Dierk's initial response, there is a limitation within IB that you cannot request more than 60 days worth of historical data. When I requested 90 days of historical data for the ASML, I received the same error and my chart would only go back as far as 10/01/07.

Moving forward, we would be concerned if the Progress window of the chart or back test does not disappear, but remains on screen when submitting this request. In addition, we would be concerned if the historical data requests no longer work within IB after waiting 5 minutes to request additional historical data.

Thanks again for your post and enjoy your day.

[luitom]

I also have a problem with IB. I just subscribed to real time trading with Ninja (after testing with the simulator for some time) and I see two problems:

1) the positions window does not update as fast as TWS
2) some symbols do not have data in Ninja even though they have data in TWS. For example the symbol SA

I just want to point out that I followed the procedure for the IB connection as from your site and I added the symbols to the Instrument manager

Hope you can help. Thanks

[luitom]

I also do no see any data for QM 12-07

[luitom]

Another serious issue: look at the daily bars of MRVL in the last few days. It is completely wrong (especially on 7/11/07)

[NinjaTrader_Ray]

1) There is a known issue with IB TWS position reporting. This is an IB issue, not an NT issue.

2) Please send note in to support at ninjatrader dot com requesting help on the QM, someone will follow up

3) In general, we display the data we receive from IB. If an IB daily chart (in TWS) looks different than NT, let us know

[luitom]

Regarding MRVL I do not know if it is a NT issue or a TWS issue but I can guarantee that the daily chart is wrong compared to my Metastock or Yahoo chart.

Could it be a problem coming from the fact that I was using Opetick when I was only in simulator with NT? Is there a way to cancel all the historical data from NT and start from scratch?

[NinjaTrader_Ray]

You can overwrite the data via Tools > Historical Data > Reload

[luitom]

As it has been said in this thread IB historical data is supposed to a max of 60 days. For some reason I cannot get more thatn one week. Can somebody please tell me how to solve this?

[NinjaTrader_Ray]

60 days is likely for daily data, is this what you are having a problem with?

Minute data is likely only about 1 week.

[luitom]

My problem is that I get daily data only for one week and minute data only for one day. Something must be wrong on my end if this does not normally happen.

I have already tried to use the "repair DB" function and also tried to delete the DB file altogether to start from scratch. I still have the same problem

[NinjaTrader_Dierk]

please make sure you are connected to an IB live account (demo and paper trade will not work for backfill)

[luitom]

yes I am connected to a live IB account and I just subscribed to a live trading NT account. on the daily chart I cannot get data beyond 10/3/07 and on the intraday only the data of today

[NinjaTrader_Dierk]

If you don't get any more data even on Tools->Historical data->Reload then there is no more data available on IB's end in which case I suggest contacting their support.

[luitom]

The problem is that it is something that has to do with NT. Using Quotetracker, for example, I can create a chart of APPL using IB backfill all the way to June.

With NT, even using the reaload historical data feature, I can go back to 10/3/07. There has to be a problem with NT.

What can I do?

[NinjaTrader_Dierk]

Which NT version is this? (Help->About)

[NinjaTrader_Dierk]

Also: which TWS version are you on?

[NinjaTrader_Dierk]

I just tested 365 days of daily data and 30 days of 1 minute data on a fresh standard DB using 6.0.1000.6 and TWS 877 -> works as expected and I get data for the full timeframe.

[luitom]

The versions are:

Ninja 6.0.1000.6 (live)

and TWS 878.5

[NinjaTrader_Dierk]

I suggest retying with TWS 877 (which is the one we recommend: http://www.ninjatrader.com/webnew/support_interactivebrokers.htm) and a fresh DB.

[luitom]

Tried exactly as you suggested. Downloaded and installed TWS 877 from your link. Deleted the DB files from the Documents folder and from the Pogram files\Ninjatrader 6, created a new connection to IB

Same problem: daily back to 10/3/07, intraday only today

I really do not know what I can do

[Pete S]

luitom: I had plenty of problems with IB's data feed and ended up switching (I am on a trial with IQFeed right now and will likely keep it).

But I never had the problem you are describing. Are you getting any messages about historical data pacing being exceeded? Can you load the exact chart in TWS and get the data you expect? What exchanges have you set the symbol up with in Instrument Manager?

I know that NT can load the data, given the volume restrictions IB places on it; I suspect it has to do with your instrument set up? [Note: I am just a user and have nothing to do with NT.]

[luitom]

I am not sure how much is cost of iqfeed, but the problem is that I have been a client of IB for a while and I like them as brokers.

To answer your question I do not receive any error message: I simply get in the chart (whatever instrument I request) only the one month of daily data and only today's intraday data.

I am thinking of trying to unistall and then reinstall Ninja.

Dierk, could you please give me some final advice? If I want to reinstall NT, how can I save the script, the custom templates and strategies and the isntruements that I added?

[luitom]

Yes I can the data in the TWS data (for example AAPL) for more than one day. The instrument setup with NT is the default setup

[NinjaTrader_Dierk]

>> Dierk, could you please give me some final advice?
Please see post #35.

[luitom]

I already followed your advice and I have version 877 but I still have the same problem. I also tried with a new DB

However, if I wanted to try to reinstall completely NT, how could I go about saving my scripts, instruments and templates?

[NinjaTrader_Dierk]

Just uninstall via control panel and reinstall from fresh download. Your data will be maintained.

[luitom]

I tried this also. But, unfortunately, I still have the same problems. It looks as if there is no simple solution. I really do not know what to do.

[NinjaTrader_Dierk]

Sorry, I'm out of ideas. It all works as expected here.