Complete Freeze of NinjaTrader

[Harry]

This is by far the biggest problem with NinjaTrader. I am an experienced long term user of the product, and still I cannot tame it. Today it crashed and is completely dysfunctional.

This is the problem:

With any workspace and any chart (old workspace, new workspace, default chart, complex chart), if I right click on a chart and select "Indicator", it is the end. Same if I select Ctrl-I, so it is not mouse related. See screenshot below.

NinjaTrader simply stops working and is frozen. No CPU load, moderate RAM usage (160 k). Completely dead.

After restart of NT I made a number of tests:

-> checked assemblies: no assembly installed
-> edited an indicator and compiled: no problems, anyhow same indicators as yesterday
-> checked trace file -> no messages
-> checked log file -> no messages
-> checked other options from drop down menu -> no problems

These are the really frightening NT experiences. It simply stops working for no reason. Only way out is to kill the NinjaTrader.exe process.

[Harry]

Exploring the Problem

I am fortunate that this happened on a week-end, so I will now try to find the error. First thing I did, was a complete reinstall of NinjaTrader 7.1000.3, but the bug persists.

NinjaTrader will still freeze, when I try to open the indicator dialogue box. So I will now explore the files in the directory My Documents -> NinjaTrader 7

[cjsiding]

Me 2 absolute total freeze up. Where to start to look? Or should I just delete 7.3beta and reinstall earlier version? this is ridiculous

[Harry]

I am further exploring.

I have now replaced the files under NinjaTrader 7 -> bin -> Custom one by one, and I have found the culprit. The culprit is the file NinjaTrader.Custom.dll. If I replace the file of my NinjaTrader installation with the original file, then the problem will not occur. The original file which is installed when downloading NT 7.0.1000.3 has a size of 344 KB, my file had the size of 4.492 KB, as I had added some indicators.

So I replaced my NinjaTrader.Custom.dll with the original file. Then I added all my indicators and the problem occured again. So we have the paradox situation that

-> compiling all my indicators works out correctly (no error message)
-> after compiling the indicator dialogue box cannot be opened anymore

Interesting enough the same set of indicators did not produce any problem yesterday.

I will now add the indicators one by one to check, whether the problem can be tracked down to a specific indicator, or whether NinjaTrader cannot compile more than 250 indicators at once. I am now at 250 indicators, and what I notice is that NinjaTrader hangs a short moment, when I open the indicator dialogue box.

[Harry]

After further testing I could track the problem down to the jtEcoNews indicator. I have installed this indicator since weeks, but today when I use it with NT 7.0.1000.3 it produces the problem. There were no problems however during the last week with NT 7.0.0.1000.3 , and I am 100% sure that this indicator was part of my NinjaTrader install.

So this is the intermediate result:

NinjaTrader 7.0.1000.3 + jtEcoNews + WeekEnd -> NinjaTrader freezes.

The strange thing is that this indicator compiles without problems, but once it is installed it prevents the indicator dialogue box from popping up.

This is reproducible and was also tested by adding this indicator to the default system indicators without interference of other custom indicators.

[NinjaTrader_RyanO]

Hi Harry,

Thank you for your post and for the clarification of the issue, and so far I am unable to duplicate the issue. The JtEconNews indicator available from the forum that I imported did not produce the issue either, and to further test I am curious if you could provide the file you identified to see if I can duplicate what you have found.

[Harry]

I used a non-assembled version

Hi Ryan,

thank you for answering. I did not download the indicator from the NinjaTrader forum as per link below

http://www.ninjatrader.com/support/forum/local_links.php?action=jump&catid=4&id=437

because the NT forum version is a compiled assembly and not a .cs file. I am always careful not to use assemblies, as I am not aware of any side effects..

The author also provides an indicator, which can be downloaded here:

Blog: http://johnthom.blogspot.com/2010/12...njatrader.html
Indicator: http://johnthomcom.web702.discountas...EconNews2a.zip

I will send you a copy of the current file jtEconNews2.cs via e-mail as well, which is the version that can be downloaded as per now.

I have tested this indicator on both my machines (XP 32-bit and Vista 32-bit) with NT 7.0.1000.3 and both machines freeze after compilation of this indicator, when I try to access the indicator dialogue box. I had no problems with NT 7.0.1000.2, and none with NT 7.0.1000.3 until yesterday.

The indicator calls "http://cloud.forexfactory.com/ffcal_week_this.xml". This address is non-existant.

[Harry]

Mystery

The main problem is that NinjaTrader always compiles all indicators, even if they are not used, so gradually the NinjaTrader program library (NinjaTrader.Custom.dll) grows to a significant size.

What is most alarming. The EconNews indicator compiled fine. It worked for months without problems. I forgot it, as I do not use it.

All of a sudden NinjaTrader stops working correctly due to this indicator that I never use.

The fact that NinjaTrader compiles all indicators means that any bugs contained with those files can crash NinjaTrader, even if you never use them.

It would be an improvement if NinjaTrader allowed you to mark files as active (compiled) and inactive (non-compiled). Of course I can achieve this by creating a new directory to the folder NinjaTrader 7 and moving the non-used indicators to that directory. Feasible, but not very user-friendly.

[NinjaTrader_RyanO]

Hi Harry,

I have tested this issue, and was able to duplicate it using that particular file, and so based on that I have forwarded this issue to development so that we can continue working to identify the precise cause.

[NinjaTrader_Dierk]

Harry,

To clarify:
- NT does not control the code provided by 3rd party vendors or customers
- using a CS file vs an assembly does not protect you from the experiences with incorrect 3rd party code which you just had
- compilable code is no guarantee that this code is bug free and would not cause any harm to the "hosting environment" (here: NT). This is nature of programming
- coding bugs in 3rd party code could lock up NS easily. The approach we have taken to manage the NinjaTrader.Custom.dll is targeting optimal performance. Any other approach (multiple assemblies, attempts to shield NT from faulty 3rd part code etc.) would decrease the performance of NT significantly and would not be in the best interest of the vast majority of our users.
- experiencing that 3rd part component works no problem for months/years is no guarantee if would be issue free forever
- NT does not fix issues related to 3rd party or customer NS components. Please contact they author for resolution

Thanks

[Harry]

Thanks for your answer, Dierk.

I agree with everything you stated below, and I have already contacted the author. I was just astonished that an indicator, even if not used, would lock up NinjaTrader. Without the indicator NT 7.0.1000.3 runs as expected.

I just do not use free assemblies because I am generally afraid of malicious code, which can be hidden within an assembly. With an open code .cs file it is easier to check for such code. For that reason I had not downloaded the assembly from the download section of this forum, but used the open code indicator file that I had found on the website of the author.

Harry,

To clarify:
- NT does not control the code provided by 3rd party vendors or customers
- using a CS file vs an assembly does not protect you from the experiences with incorrect 3rd party code which you just had
- compilable code is no guarantee that this code is bug free and would not cause any harm to the "hosting environment" (here: NT). This is nature of programming
- coding bugs in 3rd party code could lock up NS easily. The approach we have taken to manage the NinjaTrader.Custom.dll is targeting optimal performance. Any other approach (multiple assemblies, attempts to shield NT from faulty 3rd part code etc.) would decrease the performance of NT significantly and would not be in the best interest of the vast majority of our users.
- experiencing that 3rd part component works no problem for months/years is no guarantee if would be issue free forever
- NT does not fix issues related to 3rd party or customer NS components. Please contact they author for resolution

Thanks

[NinjaTrader_Dierk]

>> I was just astonished that an indicator, even if not used, would lock up NinjaTrader
Yes, that easily could happen as you had bugs in your code in Initialize(). I have not analyzed the indicator in question in depth but I see this method being called: LoadNews ... which leads me to speculate this might be culprit.

That's one of the reason why we recommend the usage of OnStartup and OnTermination over Initialize()/Dispose() (introduced with NT7)

BTW: we appreciate that you did help us isolate bugs/issues related to merging, data series management etc. That definitely helped us going through the "rough waters" on that subject and even gave us some ideas on what to improve for future development of NT.

[Harry]

Hi Dierk,

thanks for your answer. As you know, I am basically a happy NT user, but also trying to find bugs and improve the product in a way a key user can do.

The problem of the indicator was that it had set a string variable to an inexistent value

private const string ffNewsUrl = @"http://cloud.forexfactory.com/ffcal_week_this.xml";

The domain simply produces a DNS error. This was enough to freeze NinjaTrader, even when the indicator was not used.

I agree that OnStartUp() is a huge improvement, in particular for allotting values to variables.

>> I was just astonished that an indicator, even if not used, would lock up NinjaTrader
Yes, that easily could happen as you had bugs in your code in Initialize(). I have not analyzed the indicator in question in depth but I see this method being called: LoadNews ... which leads me to speculate this might be culprit.

That's one of the reason why we recommend the usage of OnStartup and OnTermination over Initialize()/Dispose() (introduced with NT7)

BTW: we appreciate that you did help us isolate bugs/issues related to merging, data series management etc. That definitely helped us going through the "rough waters" on that subject and even gave us some ideas on what to improve for future development of NT.

[Harry]

In fact things are quite simple.

When you open the indicator dialogue box, NinjaTrader executes the part of the indicator code which contains variables and properties, as both are required by the indicator dialogue box to set the values.

The indicator that caused the problem had tried to set that inexistant string

@"http://cloud.forexfactory.com/ffcal_week_this.xml"
in the variables region.

Correct procedure: Just declare variables in the variables section and do dirty stuff like this in OnBarUpdate() with a try{} catch{} protection. In that case NinjaTrader will not freeze, but the indicator will not work and throw a proper exception.

The question here is, whether it is possible to let NT check for coherence of indicator code, before executing it, just to avoid the freeze.

[Harry]

The link to the indicator is in post #7. I had also sent it to Ryan_Olson via email.