Data constantly reloading in Strategy Optimizer

[wayneFH]

I don't know if you consider this a 6.5 beta issue or a feature request.

But I observed that the optimize function of Strategy Analyzer, it reloads the data for every single iteration of the testing.

I'm using tick data for 2 weeks which I can see takes about 100 Meg due to the increase in the NinjaTrader process memory footprint when I first load the data.

When optimize runs, the process cycles through increasing it's memory footprint by 100 Meg and then dropping by 100 Meg and growing by 100 Meg and dropping on and on.

That makes back testing tick data ridiculously slow.

Can you please simply take the data loading and unloading out of that loop?

There's no need for it. The data never changes when testing, it's only the strategy parameters than need to iterated and re-run.

Please fix this. I consider it to be a bug and hope you do also.

Sincerely,
Wayne

[NinjaTrader_Dierk]

NT tries to reload data from the connected provider as the backtested data set is not completely available in the DB. Pls disconnect to avoid this.

[wayneFH]

I am disconnected when this happens. But I have "store real time data" turned on. So it apparently reloads from the database for every iteration of the test.

Even if it was connected, you should improve the smarts to only load from the data vendor on the first time. After that, just iterate using the same data.

Sure you can see that it's silly and a waste of resources to attempt to load the data from vendor or DB on every iteration of an optimization, right?

ESPECIALLY since the data vendor might, possibly, give different data each time it could skew or confuse the optimization results.

Please improve this functionality in the next release.

Still, I consider this a bug. Optimization should never reload data from anywhere--DB, Vendor or otherwise--for every iteration of the optimization.

Sincerely,
Wayne

[NinjaTrader_Dierk]

>> So it apparently reloads from the database for every iteration of the test.
a) How to you figure? What exactly do you see?
b) whats is you exact scenario to reproduce?