Index out of range errors

[raintree]

I keep getting this error while trying to write a strategy that adds a Daily data series to the 5M chart on which the strategy is running.

In this strategy I want to make certain calculations at a certain time of day (2:00 AM EDT) and these calcs call data from both time frames. I'm using detailed prints to the Output window to print the calculations and debug the strategy.

I understand that I must check that have enough bars on the chart for both data series and I've tried various ways of using BarsInProgress and BarsArray to explicitly denote which data series I'm trying to call from which time frame. I basically need to make sure the strategy doesn't start running on the 5M time frame until I have enough data on the Daily time frame to calculate the following variable:

double dailyATRpips = ATR(BarsArray[1], 15)[1] * 10000

I've tried two ways of making the sufficient bars check. The first one doesn't work, the second does.

1) if(BarsInProgress == 1 && CurrentBar <40) return; // called from Daily TF
2) if (BarsInProgress == 0 && CurrentBar < 8650) return; // called from 5M TF

Q1: It seems that I cannot call the bars check from BarsInProgress == 1, but must instead call it from the primary data series. It this correct? Must the bars check always be called from the primary series?

I'm posting the code that works best so far below. This code contains no trade logic yet because I'm trying first to get the calculations right before proceeding further.

Running this strategy from the 5M chart seems to print all the calculations correctly at the correct time, but it only prints for a small fraction of the backtested days, i.e. 2/14- 2/28/2011, when the strategy is running 1/1-7/10/2011. And it still prints the “out of range” error to the log file.

Q2: Why is this happening and how can I correct it?

[NinjaTrader_Bertrand]

raintree, there's a CurrentBar check for multi series scripts called CurrentBars - http://www.ninjatrader.com/support/h...urrentbars.htm

It would be important to incorporate this into your script for both series.

[raintree]

Thanks, Bertrand, it's good to know about that.

I've incorportated that into my script but, unfortuntely, it still returns the same backtest behavior: It still gives an "out of range" error and prints only a segment of the backtest period.

Would you have time to run the revised script and see what else might be causing this?

Edit: Sorry, the revised code I tested had an error (using && instead of ||). When I fixed that it still gives the out of range error and now it doesn't print anything!

[NinjaTrader_Bertrand]

Correct, it would need an || so either series check can lead to the needed return out of the OBU.

First thing you would need to debug is your beginBarsAgo value - it would need pretty high CurrentBars check to work as you would use it then as well in your Draw() method calls referencing this amount of bars back.

[raintree]

Hmmm, I don't see how my beginBarsAgo value could be causing it. That is only meant to reference back 8 hours the 18:00 EST on the evening before the European Open at 2:00 EST. That's WAY less than the 8650 bars ago I'm already using for the 5M and the 40 bars I'm using for the Daily TF in the CurrentBars check.

Any other suggestions?

[koganam]

Hmmm, I don't see how my beginBarsAgo value could be causing it. That is only meant to reference back 8 hours the 18:00 EST on the evening before the European Open at 2:00 EST. That's WAY less than the 8650 bars ago I'm already using for the 5M and the 40 bars I'm using for the Daily TF in the CurrentBars check.

Any other suggestions?

Would you care to post the error message verbatim?

[raintree]

Hi koganam. Thanks for responding. Here's the error message:

**NT** Error on calling 'OnBarUpdate' method for strategy 'AsiaSetupEuroTrade/f2ede4ff1d3948aeba99658ef306e59b': You are accessing an index
with a value that is invalid since its out of range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.

[koganam]

Hi koganam. Thanks for responding. Here's the error message:

**NT** Error on calling 'OnBarUpdate' method for strategy 'AsiaSetupEuroTrade/f2ede4ff1d3948aeba99658ef306e59b': You are accessing an index
with a value that is invalid since its out of range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.

That is definitely a bars escape issue. You will need to put in some print statements around any statements that access a dataSeries by index. That should quickly tell you which statement is the issue.

[raintree]

That is definitely a bars escape issue. You will need to put in some print statements around any statements that access a dataSeries by index. That should quickly tell you which statement is the issue.

Thanks for the suggestion, konganam. I guess I wasn't doing that systematically enough around enough statements. I'll try what you suggested. Thanks for jumping in here.

[koganam]

Thanks for the suggestion, konganam. I guess I wasn't doing that systematically enough around enough statements. I'll try what you suggested. Thanks for jumping in here.

Ultimately, I have found that when using the Print() statement for debugging, the shotgun approach seems to be the most efficient.

I just do this around every index access statement, even those that look obviously correct.

Print("B4 cp1");
// statement that accesses an index
Print("After cp 1");

// other statements

Print("B4 cp2");
// statement that accesses an index
Print("After cp 2");

//etc

The first "After cp ..." that does not print, and I have the initial culprit.

You will be surprised how many times, it was one of the "obviously correct" statements that turned out to be the culprit.

[raintree]

The mystery deepens

Thanks for detailing your "shotgun" debugging method, koganam. It makes a lot of sense and seems like it would be foolproof. But I certainly didn't expect the behavior it produced!



I applied your method literally, wrapping "before cpN" and "after cpN" print statements (where N = 1 to 8) around each of the 8 index access statements. See full code with all print statements attached. Here's what happened in the Output window:

1. It only prints the first print check ("cp1" before and after) -- NONE of the others. I expected the shotgun method to print all of the print checks. What am I missing here?
2. I also did not expect that the "culprit" would be the sufficient bars check itself! i.e. if(CurrentBars[0] < 8650 || CurrentBars[1] < 40) return; Is there something wrong with this statement (apart from the obvious redudancy of repeating approximately the same lookback for the 5M and Daily timeseries)? Could there be other culprits that we are not detecting?
3. Looking at the cp1 print record, it clearly shows "before" and "after" prints up to a certain point. At that point it prints the same print check twice (this never happened before) and fails to show "after". At this point I changed the Print statement for the cp1 check to PrintWithTimeStamp, which showed that the culprit index reference stopped working correctly at exactly 2/28/2011 5:00:00 PM. This is consistent with what I reported in #1 and I'm not surprised that it stopped at exactly 5:00 PM because this strategy is trading EURUSD and that is the close of the daily forex session. I've always suspected that the issue with this strategy is the statement on line 73, which is a call from the 5M BarsInProgress to the daily BarsArray[1]: dailyATRpips = ATR(BarsArray[1], 14)[1] * 10000; The strategy runs with no errors without this statement but I can't really remove it because it's essential to the strategy.
4. Why it should perform correctly up uptil 2/28/2011 and only then start misbehaving is still a mystery to me! The only thing I can think of is that there's some issue with the Strategy Analyzer itself.

Well, that's everything I have to report right now. The devil must be in these details somewhere!

Btw, I love a good puzzle -- but am not always so lucky to find a mystery that keeps deepening like this one! LOL. I am also lucky to have access to a forum in which, sooner or later, someone will pop up and point out an error that will may seem obvious in retrospect (. . . or acknowledge a bug in the platform, Btw, I;m using build 4.). Until then I am still very grateful for additional help!

[koganam]

That sounds like you may be generating a processor exception. Look through your trace and log files to see if you are generating one.

Also look for an invalid object reference statement.

We will crack this one yet. LOL

[raintree]

That sounds like you may be generating a processor exception. Look through your trace and log files to see if you are generating one.

Also look for an invalid object reference statement.

We will crack this one yet. LOL

Hi koganam. Thanks for your persistence in this matter!

There are no log or trace messages anywhere near the time frame of any of my backtests, including the one I just now performed. In fact, I'm surprised to notice now that the "out of range" error is also NOT printing from the latest posted version of the strategy. However, it still fails to print the "after" string at 2/28/2011 5:00:00 PM before cp1, and just keeps printing only "before cp1" on every 5M bar until 7/8/2011. And, of course, it still fails to print the calculation outputs that the code is intended to produce. The calcated outputs print if I comment out line 73, but of course there are INTOLERABLE math anomalies associated with dividing by zero! LOL.

Oh, now I notice, another piece of the puzzle: The exact same behavior is produced even with line 73 commented out. So apparently, it has nothing to do with the call to Daily bar data from the 5M BarsInProgress. Crikey!

This is kinda where I would expect some Ninja like Bertrand or Josh to come back in and say this kinda thing is "known behavior" of NT7 build 4.

I've confired that the respective Daily and 5M charts have more than the needed amount of data. Hmmm, maybe I should post the backtest settings I'm using (see attached). Since I don't really understand everything about those settings, who knows, that might be where the devil is hiding!

If that's not it, did you try to run the last cs file to see if this same behavior this is reproduced on your machine? If so, what build of NT 7 are your running?

And may I ask you whether it is expected that the scattershot print debugging method would ever print only the first cp sandwich test and not the other 7? That's still a missing piece of the puzzle for me.

[koganam]

And may I ask you whether it is expected that the scattershot print debugging method would ever print only the first cp sandwich test and not the other 7? That's still a missing piece of the puzzle for me.

Yes, the method is designed to show where there was no return from the processor to the strategy/indicator. It can happen on any of the index accesses. In this case, as you see, it seems to have been trapped on the first "obviously correct" (yeah, right) statement.

Normally, wherever we cannot get to the "after" statement tells us where the error is, as essentially that is saying that the indicator/strategy never returned from the call to the processor. That is why we would expect to see some indication of the error. That there is no indication of the error in the normal NT places, seems to imply that the strategy spun so out of control that it could not even return information from the processor to NT.

As I seem to never be able to step through an NT strategy bar-by-bar, in such intransigent cases, I have ended up recoding the strategy as an indicator, then stepping through it bar-by-bar until I trigger the error.

[koganam]

I just looked into the .cs file, and found this snippet in OnBarUpdate()

			PrintWithTimeStamp("before cp1");
			if(CurrentBars[0] < 8650 || CurrentBars[1] < 40)
			PrintWithTimeStamp("after cp1");
				return;

Given that sequence, I would expect you to never process anything as the return; statement will always be executed.

The PrintWithTimeStamp("after cp1"); statement will be executed throughout your initial bars escape period, then should not print.

Methinks that the strategy is doing what you asked it to do., not what you intend it to do? You probably want to switch the 2 statements.