Interminable Reload Historical Data

[KBJ]

I tried reloading historical data with 6.5 and found that NinjaTrader just hung, saying "This request can not be aborted."

It's been sitting there for upwards of 1/2 hour, and it hasn't completed. Is there a way to debug this?

[zoltran]

I have this too from time to time

Thought it was just me.

I am using IQFeed. But also get it on opentick

[KBJ]

Well, as you can see from the screen clip, this is ZenFire, so the problem seems to occur with multiple datafeeds.

I've been letting the program just sit there in hopes that it would come out of it, but it seems to be completely hung.

Ray, Dierk, anyone... while this is still running and before I use the Windows Task Manager to kill the process, is there a way to get you more info on this so it can be debugged?

Also, if I kill this off with the Windows Task Manager, will I have to do any cleanup to correct the database, or will NinjaTrader do that automatically when restarted?

[NinjaTrader_Ray]

Go ahead and kill it, I can reproduce here.

[KBJ]

Will do. Thanks for for the info.

[NinjaTrader_Dierk]

Guys,

Please do some math to understanding the issue you're facing: 150,000 ticks per day on the ES * days to reload * ~25 bytes per tick (NT internal mem usage).

Here are the potentials sources of trouble:
- data download just takes it's time at the provider's server (please try downloading just one day of ticks from DTN/IQFeed and you'll see, no issue on Zen though)
- NT client consumes a lot of memory so locally buffer the data before saving it (expected). This could force your machine to "swap" memory to disk

Sorry, these are the limitations. We'll see what we can do there. Up to then I suggest working around as per info above.

[zoltran]

Understand all that Dierk

But NT really has some issues here.... more than other platforms I would say.

For what it's worth .. I find NT far more reliable when I do NOT connect to my historical data feed.
(I use IB for R/T, IQFeed for historical)

At least this way it doesn't lock up or pop up the dreaded 'loading data' box.

[KBJ]

When I had this problem, I had loaded a couple of periods of data for other markets before it hung up on this one with no problems.

Could it be that .Net isn't freeing up the memory for re-use when NinjaTrader is done reloading the historical data?

[NinjaTrader_Dierk]

Originally Posted by zoltran

Understand all that Dierk

But NT really has some issues here.... more than other platforms I would say.

For what it's worth .. I find NT far more reliable when I do NOT connect to my historical data feed.
(I use IB for R/T, IQFeed for historical)

At least this way it doesn't lock up or pop up the dreaded 'loading data' box.

Where would you expect that NT does not (!) reload data from the provider? Thanks

[zoltran]

No , of course not

Just that the current method has problems.

Once that loading data bx gets displayed, you can get into trouble

[NinjaTrader_Dierk]

Next NT6.5 will hold enhanced traces for your scenario. Next time you got stuck on the "Loading data..." box (after next update), please mail me traces and logs and timestamp on when this happened to "dierk AT ninjatrader DOT com". I would want to understand if it's the provider that did not answer the request or some NT internal issue. Thanks