Optimizer on [Dual|Multi]-Core CPUs

[tomcat]

Hello !
When I run the optimizer, it uses about 50% of the CPU power of the dual-core cpu, either one core at 100% and the other idle or both at about 50%.

-> How about running the Optimizer in a multithreaded way so that each CPU/core can handle the calculation for a subset of the dataseries and speed up the optimizing by the number of CPU-cores available?

-> this should apply to single-instrument optimization as well as multi-instrument optimizer runs.

Many Thanks
Tomcat.

[NinjaTrader_Ray]

Thanks for your suggestion.

[pcole]

Is there any news about using all CPU cores by optimizer (and walk forward also)?
I have a Intel Quad Core CPU and only one core is used during optimize phase...

Thanks

[NinjaTrader_Dierk]

Sorry, nothing has changed there yet.

[whitegun]

Will multicore optimization finaly be availeble in NT7??

Or NT didn't plan to implement this at all??

Will the memory problems relatet to optimizasion finaly be resolft in NT7 (this seems to be a forever bug!)?? Or NT also didn't plans to correct this bug??

[NinjaTrader_Dierk]

We will make announcements as appropriate.

[whitegun]

When??

You told in an other post NT7 beta will be available and of this year, so when we can expect to get some information about what will be implemented and what not??

[NinjaTrader_Dierk]

Likely towards the end of this year.

[whitegun]

Acording to an other post of yours (if I am not wrong) NT7 beta will aper end of the year! So you plan to not disclose anything until the beta?? Even will not answer concret questions (to specific points)??
Let us wait and wait for later found out that what is maybe indispensable for us was not implementet and after loosing half year waiting however we will need to adopt to an other platform to fullfil the task that NT can not do!!!!!!?????

[NinjaTrader_Dierk]

We will make NT7 commitments as we announce featured towards the end of this year, which is after we have been able to successfully implement all NT7 features. We have multi-core support on our NT development list though.

[whitegun]

Thats, what I wantet to know! So multicore will be suported.

Will the memory problem in optimizasion (bug, NT crushing) be resolfed? This is not a feature but a bug (and a serios one wich already lasts forever) so I supose you can give me feedback on it.

I supose you know what I am speaking about?? Or should I explain??

[NinjaTrader_Dierk]

>> Or should I explain??
Yes please. At best based on a the NT sample strategy SampleMaCrossOver.

[RL8093]

We have multi-core support on our NT development list though. Not sure how to vote for prioritization of new features - but multi-core functionality should be high on the list. Seems that only one competitor has this support - if others like eSig finally add it to their program - it could hurt NT in the mkt. With the proliferation of multi-core CPUs - it would seem to be a miss to not have this functionality already ....

R

[NinjaTrader_Ray]

Will be part of NT7.

[auspiv]

in defense of the ninjatrader dev team, writing programs to use multiple cores is not an easy task, especially if the task isn't parallel, which im guessing NT is not.

[cassb]

in defense of the ninjatrader dev team, writing programs to use multiple cores is not an easy task, especially if the task isn't parallel, which im guessing NT is not.

Yeah, geez guys, give 'em some slack! :p

[webart]

I know you get a million requests for new features ... here is another one !

It would make Nina trader a leader in optimization if you not only had Multi Core, but if Ninja could work with slave machines to optimize. So you could run the problem through a farm of computers across a network.

I think Ninja just uses bruteforce for optimization ? It works, and covers all possibilities, but its slow. If you could split the tast up between serveral machines, things that just can't be done now, could become possible. It is especially slow using a few months of tick data, with 3 levels of factors to optimize.

Having the ability to split a tast between a farm of machines would really make Ninja Trader very powerful.

[NinjaTrader_Dierk]

Thanks for your suggestion. We'll add it to the list of future considerations.

[Elliott Wave]

For an example of incredible/seamless/easy use of slave machines for creating pseudo-server-farms, check out REAPER (http://www.reaper.fm/) (created by Justin Frankel of Winamp/GNUtella fame). (having a release schedule averaging 1-2 new public betas a week is also quite amazing)

Rea-mote:
http://www.cockos.com/wiki/index.php/ReaMote

Advantages
Easy to set up: simply install REAPER and whatever plug-ins you need on the slave machines. Run the ReaMote slave on the slave machines, then enable ReaMote in the REAPER preferences on the master, and you're ready to go!
Easy to use: Each FX chain has the option of running remotely. When a chain is running remotely, you can still adjust its configuration using the same plug-in UIs you are used to.
Network efficiency: since you can run entire chains remotely, you avoid having to send data back and forth excessively, reducing network use. ReaMote also has silence detection, mono detection, and optional 32-bit mode to further reduce local network utilization.
Reliability: if a ReaMote slave needs to go away, FX assigned to it will automatically run locally (and the user will be notified)
It just works: REALLY. It works amazingly well on 100/1000mbps wired networks. You can even run softsynths remotely with very minimal latency for live monitoring!I'm beginning to think that regardless of whatever performance enhancements come with NT7 its still going to be much more resource hungry than 'the competition'. This is a trade-off for the other advantages provided by the 'openness' of the platform.

Having the ability to take advantage of networked slave PCs, would go a very long way to neutralizing any potential performance issues. If it can be be done brilliantly and eloquently with sample-accurate precision (up to 192,000 samples per second, likely above the ticks per second level of all financial markets combined), I'm sure the same potential exist for financial data.

I know you get a million requests for new features ... here is another one !

It would make Nina trader a leader in optimization if you not only had Multi Core, but if Ninja could work with slave machines to optimize. So you could run the problem through a farm of computers across a network.

I think Ninja just uses bruteforce for optimization ? It works, and covers all possibilities, but its slow. If you could split the tast up between serveral machines, things that just can't be done now, could become possible. It is especially slow using a few months of tick data, with 3 levels of factors to optimize.

Having the ability to split a tast between a farm of machines would really make Ninja Trader very powerful.

[webart]

I know that software Elliot, personally I use Mackie Tracktion.

Another package I use that operates on slaves is,

http://www.ultrafractal.com/