Update problem with refrerenced indicators on FirstTickOfBar

[poseidon_sthlm]

Hello!

I have some indicators with CPU intense calculations (array sort etc) that only need to be recalculated on bar close, so I do this FirstTickOfBar using "if (FirstTickOfBar)". But I have now encountered an unexpected situation when referencing an output series of another indicator FirstTickOfBar in real time. According to my print statments it seems as the calling indicator in real time (COBC = false) is updated before its referenced indicators have called the OnBarUpdate() method on the first tick of a bar. The situation only occurs if you call the refernced indicator series one or more bars ago ( Indicator().outputSeries[1] ) but not if you call the indicator on the current bar ( Indicator().outputSeries[0] ). Calling Update() in the getter doesn't solve this problem. How do I make the referenced indicator update on FirstTickofBar before it is called by the caller?

I have attched a simple as possible example with a caller and a callee for you to reproduce. These indicators generated the following print statments in Market reaplay with COBC = false.

CALLER:
************************************************************************
Bar Close Time: 2011-10-03 08:10:00
CurrentBar: 25
[COLOR=Red]TickCount: 1[/COLOR]
CalculateOnBarClose: False
************************************************************************

[COLOR=Red]TestCalleeFTOB().BarNumber[x]: 0
squareOfBarNumber[x]: 0[/COLOR]

CALLEE: (The referenced indicator)
************************************************************************
Bar Close Time: 2011-10-03 08:10:00
CurrentBar: 25
TickCount: 1
CalculateOnBarClose: False
************************************************************************

barNumber[x]: 24

CALLER:
************************************************************************
Bar Close Time: 2011-10-03 08:10:00
CurrentBar: 25
TickCount: 2
CalculateOnBarClose: False
************************************************************************


CALLEE: (The referenced indicator)
************************************************************************
Bar Close Time: 2011-10-03 08:10:00
CurrentBar: 25
TickCount: 2
CalculateOnBarClose: False
************************************************************************

barNumber[x]: 24
TestCalleeFTOB().BarNumber[x]: 24
squareOfBarNumber[x]: 576

Regards,
poseidon_sthlm

[poseidon_sthlm]

Hello again!

I just want to make sure that you haven't failed to notice my original post, since I know that NT's customer support is very quick. Can you please confirm that you have been able to reproduce the situation I describe in my original post?

Regards,
poseidon_sthlm

[RJay]

Hi Posidon,

I also found this shortly after NT7 was released and reported it.

You are in luck. I did find something that works even better than FirstTickOfBar because it works in OnMarketData as well as OnBarUpdate.

Try this
--------------------------------------------------------------
Variables Area

private int activeBar = -1;

---------------------------------------------------------------
OnBarUpdate or OnMarketData Area


if(Bars.Count != activeBar)

{
"Do Something"

activeBar = Bars.Count;
}

----------------------------------------------------------------

I use this in all my retail products.


RJay

http://www.innovative-trading-solutions-online.com/

[NinjaTrader_Brett]

Hello,

Thanks for the re-ping as your note appears did get buried.

What RJay recommends should do the trick. This is required since NinjaTrader methods run asynchronously and NinjaTrader cannot guarantee a certain order of methods firing.

-Brett

[poseidon_sthlm]

Hello!

I have now tested the work around that RJay recommend, but the described issue still remains. The print statments show that the calling indicator in real time (COBC = false) is updated before its referenced indicator has called the OnBarUpdate() method on the first tick of a bar.

Can you please take a look into this? (If my presumption is correct, I suppose that this means that there are a lot of NT live strategies that produce incorrect results daily, since one of the most common appoches to enter a trade is to fire a stop limit order 1 tick above/belowe the current bar at the first tick of a new bar.)

This is required since NinjaTrader methods run asynchronously and NinjaTrader cannot guarantee a certain order of methods firing.

What does this mean? Is it not reliable to reference an indicator from another indicator in NT?

Regards,
poseidon_sthlm

[NinjaTrader_Brett]

Hello,

This is in reference to NinjaTrader methods.

For example, you can't expect OnMarketData() to always fire before OnBarUpdate() does.


However after re reading your original question this appears not to be related you are working only in the indicators.


I ran your code and looked though it and spend some time trying to figure out exactly what you are showing me however I still dont have an exact picture of what your expected vs what your getting. With these more complex issues we needed to simplify it even further.




For example I'm unsure of the relation this has to the issue you are reporting:

        // Shift between RT and COBC
            x = (CalculateOnBarClose || Historical) ? 0 : 1;
            if ( CurrentBar < x) return;

Also I'm unsure what the is in relation too:

squareOfBarNumber.Set(x, TestCalleeFTOB().BarNumber[x] * TestCalleeFTOB().BarNumber[x]);
            
            Print("squareOfBarNumber[x]: " + squareOfBarNumber[x]);

So that I'm on the same page, can you please outline the sequence of events and values you expect in this format.


For example:


->Tick received for new bar
->OnBarUpdate Called for CALLER
->CALLER references value from the CALLEE Where FTOB = true
->CALLEE value is returning 0, I expect X


Furthermore for me to test this is this only in historical or live? What type of chart you running on?


Thanks with this extra information this should get us on the right page and then we can go from there.

-Brett



-Brett

[poseidon_sthlm]

Hi!

The issue I want to show is a situation where the Caller is updated before the Callee on the first tick of a new bar in real time (COBC = false). (This should by definition be an unexpected situation since the concept of referencing indicators requires the referenced indicator, the callee, to run the OnBarUpdateMethod() when called by the caller.)

This issue only occurs:
in real time and on the first tick of a bar AND
if you call the referenced indicator series one or more bars ago ( Indicator().outputSeries[1] ),
but not if you call the indicator on the current bar ( Indicator().outputSeries[0] ).

I use the variable x in all my indicators and strategies to make sure that the output on bar close will be the same whether it runs on realtime data or historical data. For real time data x= 1 and for historical data x = 0 . if( FirstTickOfBar){Close[x]} returns the same value on historical data and on live data. This concept is epecially useful for troubleshooting real time indicators since you kan compare the results for the indicator on live data and historical data.

    
 // Shift between RT and COBC
 x = (CalculateOnBarClose || Historical) ? 0 : 1;     
if ( CurrentBar < x) return;

For this particular issue the variable x doesn't matter. You can replace x with 1 and run the code on live data in Marker Replay. The outcome will be the same. Therefore I have replaced x with 1 in the code below.

CALLER:

        protected override void OnBarUpdate()
        {                            
            squareOfBarNumber.Set(1, TestCalleeFTOB().BarNumber[1] * TestCalleeFTOB().BarNumber[1]);
            
           [COLOR=Gray] Print("squareOfBarNumber[1]: " + squareOfBarNumber[1]);[/COLOR]
        }

CALLEE:

        protected override void OnBarUpdate()
        {
            if(FirstTickOfBar)    // Only FTOB calc in real time
            {
                barNumber.Set(1, CurrentBar - 1);
            }

          [COLOR=Gray]  Print("barNumber[1]: " + barNumber[1]);[/COLOR]
        }

The sequence of events and values that I expect for the first tick of a new bar on real time data:
->First tick received for new bar
->OnBarUpdate called for CALLER
->CALLER references value from the CALLEE, where FTOB = true, 1 bar ago ( TestCalleeFTOB().BarNumber[1] )
->CALLEE value 1 bar ago (BarNumber[1]) is returning 0, I expect: CurrentBar - 1
-> CALLER value 1 bar ago ( squareOfBarNumber[1] ) is returning 0, I expect: (CurrentBar-1) * (CurrentBar-1)

The print statments in post #1 show that the OnBarUpdate() method for the Callee isn't called on FirstTickOfBar, but it is called on every subsequent tick. Futher more this issue only appears when referencing the calle output series 1 bar ago ( TestCalleeFTOB().BarNumber[1] ) . If I call the Callee FirstTickOfBar and reference the calle output series on the current bar ( TestCalleeFTOB().BarNumber[0] ) the Calle is called as expected.

I appreciate your time and your intrest in this matter.

Best Regards,
poseidon_sthlm

[NinjaTrader_Brett]

Thanks for detailing that, I red through and think we are on the same page.

I believe what you are running into is described below.

http://www.ninjatrader.com/support/helpGuides/nt7/multi_time_frame__instruments.htm

In the section:

How Bar Data is Referenced

Please carefully read and understand the considerations in how MTF work in live mode vs historical and let me know if you have any questions.

If this is not what you are running into let me know and I will run your sample and do a quick debug to see exactly what is occurring.

-Brett

[poseidon_sthlm]

Hi again,

I have read the section on referencing MTF bars objects again, but I cannot see that this is related to the situation described below. This is not an MTF indicator. ( I'm sorry if I missled you by writing "Indicator().outputSeries[1]" below. With OutputSeries I here mean a normal DataSeries variable.) The print statements in post #1 is the result of running the caller on a simple 5 minute bars series in Market Replay.

For the situation in question (in detail described in post #7) I persist to claim that the Caller is updated before the Callee on the first tick of a new bar in real time. Futhermore the output from NT is inconsistent between referencing the output series one or more bars ago Indicator().output[1], compared to referencing the output series on the current bar Indicator().output[0]. IMHO this futher supports that we may have some unexpected behaviour here.

Since the provided indicators, Caller and Callee, are a very simplified example, it is easy to verify what the expected result should be in the oulined situation by coding the two indicators as one indicator. I'v done that and this also supports that the situation described below isn't expected.

         protected override void OnBarUpdate()
        {
            if(FirstTickOfBar)    // Only FTOB calc in real time
            {
                barNumber.Set(1, CurrentBar - 1);
            }
            
           [COLOR=Gray] Print("BarNumber[1]: " + BarNumber[1]);[/COLOR]
                        
            squareOfBarNumber.Set(1, BarNumber[1] * BarNumber[1]);
            
        [COLOR=Gray]    Print("squareOfBarNumber[1]: " + squareOfBarNumber[1]);[/COLOR]

        }

The code above generates the expected output in real time (COBC = false) on the first tick of a new bar, opposed to the example in post #1.

CALLEE In CALLER:
************************************************************************
Bar Close Time: 2011-10-03 08:10:00
CurrentBar: 25
[B]TickCount: 1[/B]
CalculateOnBarClose: False
************************************************************************

[B]BarNumber[x]: 24
squareOfBarNumber[x]: 576[/B]

CALLEE In CALLER:
************************************************************************
Bar Close Time: 2011-10-03 08:10:00
CurrentBar: 25
TickCount: 2
CalculateOnBarClose: False
************************************************************************

BarNumber[x]: 24
squareOfBarNumber[x]: 576

Best Regards,
poseidon_sthlm

[NinjaTrader_Brett]

Hello,

Yes you are correct, Thanks was trying to get this answered via a brief overview of your code however it appears I will need to run this and analyze it.

I will run this today and get back to you on this FirstTickOfBar item. I will let you know if I have any follow up questions.


-Brett

[NinjaTrader_Brett]

Hello,

Thanks for your patience.

I simplified your script to the basics and attached it here.

I ran this on a 10 second chart and got the expected output. Please let me know if you have any questions or you do not see the expected output with my changes.

My Output I wrote CurrentBar should be BarNumber instead as I switched this at the end basically to start with the working CurrentBar Scneario and then swtich to the custom data series.

Let me know if any questions.

3/16/2012 8:41:20 AM 28782 : Main Indicator FTOB TRUE
3/16/2012 8:41:20 AM 28782 : Hosted Indicator FTOB TRUE
Hosted Indicator CurrentBar is 28781
3/16/2012 8:41:30 AM 28783 : Main Indicator FTOB TRUE
3/16/2012 8:41:30 AM 28783 : Hosted Indicator FTOB TRUE
Hosted Indicator CurrentBar is 28782

[NinjaTrader_Brett]

Specifically with my code:

->First tick received for new bar
->OnBarUpdate called for CALLER
->CALLER references value from the CALLEE, where FTOB = true, 1 bar ago ( TestCalleeFTOB().BarNumber[1] )
->CALLEE value 1 bar ago (BarNumber[1]) is returning 0, I expect: CurrentBar - 1
-> CALLER value 1 bar ago ( squareOfBarNumber[1] ) is returning 0, I expect: (CurrentBar-1) * (CurrentBar-1)

[Brett] ->CALLEE value 1 bar ago BarNumber[1] is returned CurrentBar - 1.

-Brett

[poseidon_sthlm]

Thanks!

I see now that the reason for the issue outlined below was that I didn't have this line in my Caller. When I add this I get the expected output.

            //This initializes the hosted indicator for that it is cached
            double x = TestCalleeFTOB()[0];

Should a referenced indicator always be initialized this way?

Best Regards,
poseidon_sthlm

[NinjaTrader_Brett]

Hello,

Yes is always a good idea to Initalize it this way it is in sync and its constructor is called. Then you can reference the members/properties of the synced hosted indicator correctly.

You dont have to do this when you are just getting the standard Value output from the indicator as these calls are combined into one with NinjaScript.

-Brett

[poseidon_sthlm]

Hello!

I learned from the prior post how to initialize a referenced indicator in OnBarUpdate() of the calling indicator like this:

     //This initializes the hosted indicator for that it is cached
     double x = ReferencedIndicator()[0];

Does this initialization of the called indicator replace the need for adding Update() in the property getter (of the referenced indicator) when exposing an output variable that is not a DataSeries? This situation is described in the user guide here:

http://www.ninjatrader.com/support/helpGuides/nt7/index.html

Best Regards
poseidon_sthlm