NT checking my stop order price against the wrong timeframe Bar

[adamus]

From what I can tell, it appears that NinjaTrader is checking the stop order prices against the Close of the higher timeframe bar when I am submitting the orders on the lower timeframe.

From TraceOrders, it seems that it is also logging the time of the correct timeframe bar in the log statements (00:11:00), but the error log line shows the time of the incorrect bar (00:00:00):

01/02/2000 00:11:00 Entered internal PlaceOrder() method at 01/02/2000 00:11:00: Action=BuyToCover OrderType=Limit Quantity=0.1M LimitPrice=0.9700'5 StopPrice=0 SignalName='X S @LMT' FromEntrySignal='Enter S'
01/02/2000 00:11:00 Entered internal PlaceOrder() method at 01/02/2000 00:11:00: Action=BuyToCover OrderType=Stop Quantity=0.1M LimitPrice=0 StopPrice=0.9711'5 SignalName='X S @STP' FromEntrySignal='Enter S'
**NT** A BuyToCover stop order placed at '01/02/2000 00:00:00' has been ignored since the stop price is less than or equal to the close price of the current bar. This is an invalid order and subsequent orders may also be ignored. Please fix your strategy.
01/02/2000 00:11:00 Ignored PlaceOrder() method at 01/02/2000 00:11:00: Action=BuyToCover OrderType=Stop Quantity=0.1M LimitPrice=0 StopPrice=0.9711'5 SignalName='X S @STP' FromEntrySignal='Enter S' Reason='Invalid order price, please see log tab'

I have a high timeframe as the primary timeframe, and I add the One Minute timeframe in the initialization() method. Perhaps it is meant to be the lowest timeframe is first?

Or is this a known bug?

[adamus]

I have successfully worked around this issue by programming my strategy to check its stop prices against the primary bar instead of the 1min bar.

I think this shows that there must be something amiss in the NT order validation.

[NinjaTrader_Austin]

adamus, could you please post your strategy so we can check to see if this is actually a bug?

[adamus]

Austin,

I have just sent the strategy and associated files to you in a zip attachment to an email.

You will have to revert the work-around to demonstrate the bug, and you should test it on $EURUSD from 2000 to present day with a 30min timeframe.

The work-around is in 4 places: file OrderStruct, lines 200, 219, 250 and 265.

Change the code to check the 1min timeframe bar instead of the default BarsInProgress=0 bar.


//work-around:
if (this.price <= this.parent.Closes[0][0])

//desired correct code:
if (this.price <= this.parent.Closes[this.timeFrame][0])


This is the code for buying - swap for the < for a > for selling on the other lines.

There is another issue that I have, discussed in a different thread. The guys there also wanted the code so I'll direct them to you.

[NinjaTrader_Austin]

Hi there, I've received the files and will be taking a look on Monday. Thank you.

[NinjaTrader_Austin]

adamus, your code compiled and ran fine, unfortunately I'm not sure exactly what to look for. Would it be possible for you to create a very stripped down strategy that just demonstrates the stop being checked from the wrong data series? If not, could you please let me know of a specific date/bar that shows this? A ten year backtest generates a whole bunch of output, and it is hard to sift through it all...

[adamus]

Austin,

see the date on the logs in the first message. It's $EURUSD. Run it for year 2000 on 30 mins bars. You should see the **NT Error** like the copy I pasted in above.

[NinjaTrader_Austin]

adamus, unfortunately we don't have access to historical forex data that far back. The first time I ran it I specified the whole 10 year backtest, but only 5 years of data was downloaded, which I didn't notice until just now. Can you either zip up the appropriate data and upload it to something like sendspace.com or find a different data (in the last 5 years) where this occurs?

[adamus]

Hi Austin, after wasting a fair amount of time trying to upload a 300mb file to sendspace.com, I re-ran the strategy and the issue occurs in every market in every year I ran it on.

Just run it from 2009 to present on $EURUSD and watch out for that **NT Error** statement in the output window or the control center log.

[adamus]

**NT** A Sell stop order placed at '19/06/2009 10:00:00' has been ignored since the stop price is greater than or equal to close price of the current bar. This is an invalid order and subsequent orders may also be ignored. Please fix your strategy.

[NinjaTrader_Austin]

adamus, are you submitting your stop orders to the correct BarsInProgress index? How are you setting your stop loss? Are you checking if the price is valid before sending the order?

[adamus]

Of course. That's why I think there's a bug. It's all obvious in the code. Well perhaps not so obvious but it's all in there.

[NinjaTrader_Austin]

I am sorry, but it's not very obvious to me due to the use of many helper classes and methods. I do see how the stops are submitted,

this.order = this.parent.EnterLongStop(this.timeFrame, true, this.quantity, this.price, this.orderLabel);
but you will have to completely break this strategy down to the bare basics (as in only one strategy script and no variables in the stop submission) so we can understand exactly what could be going wrong.

[adamus]

Maybe I was being a bit too self-deprecatory with my comments about my code and put you off.

I know it's not the greatest job to trace through someone else's code, but I'm not in a position to recode the whole thing at this point, so if you won't or can't trace through it, then that's were this thread stops.

If I do get a spare couple of hours I'll recode it for you, but it won't be anytime soon, and I will be very surprised if I find out that it's my fault and not NT7.

You might have seen all the logging I do. I can see 95% of what is going on in my code as it runs, and plus with the output from TraceOrders, I can see that TraceOrders picks up the right timeframe, but when the OrderUpdate and Execution events are fired, the BarsInProgress is showing the other timeframe.

[NinjaTrader_Austin]

Thank you for your time on this issue, adamus. Your assistance is greatly appreciated. If you do come up with a smaller strategy to demonstrate this, we'd love to see it.

[adamus]

Austin, did you actually see the errors?

[NinjaTrader_Austin]

adamus, yes, I did see the errors. I will be out of the office until next Monday though, but I will have a coworker try to help you out.

[adamus]

OK, how about looking at this from another perspective and perhaps you can tell me whether i should be seeing this or not.

Set up any old quick strategy and add the 1min timeframe of the same instrument in your initialization.

Set TraceOrders to true.

Set a print statement in the OnOrderUpdate and OnExecution to show what BarsInProgress is.

Place an order from the onBarUpdate using the method to stipulate that it is on the 1 min time frame.

Check the Output window.

The TraceOrders says it is handling the order on the 1 min BarInProgress, but the OnOrderUpdate and OnExecution say that it is the slower time frame BarInProgress.

Let me know what you think.
[

[adamus]

When you do the Print() statements, include the time in the print-out as well and you'll see time of the wrong bars is output by print statements in the OrderUpdate and Execution events.

[NinjaTrader_Bertrand]

adamus, I'll setup your test scenario provided here in a strategy and check, however please keep in mind the trades would be executed on the next bar in backtesting, as this is the first possible trade location after your entry was triggered.

[adamus]

Go for it. You'll see what I mean. I can't imagine how this would be an artifact of the trade placement as you say.

I see the bars tick by in logging on the minute scale,

05:01
05:02
05:03
05:04
05:05
05:06

then the order I placed with the 1min timeframe BarsInProgress index gets executed on the correct bar on the correct timeframe but it's logged with the BarsInProgress index from the primary timeframe like this:

05:01
05:02
05:03
05:04
05:05
05:06
05:00 OnExecution ....
05:07
05:08
05:09

Of course my issue is not the logging output. My issue is that the order price is checked against the wrong timeframe too and so it causes an **NT** error which is seriously hampering some of my strategies, even with the work-around.

Notice that the time of the execution bar is previous to the 1 min bars, not post-.

I haven't tested this in real time trading yet since I'm only running strategies with the work-around, but from the logging I suspect the issue is the same.

[NinjaTrader_Bertrand]

adamus, I checked into here - it would return a BIP, but it could be either one as it's irrelevant in those methods, as they do not run off a bars object as the OnBarUpdate().

[adamus]

And the time? Why is the time wrong? Maybe you can get your simulation to show the time as well and you'll see that's wrong too.

[NinjaTrader_Josh]

Printing Time[0] is also irrelevant from OnExecution() method as it is linked to bars. The only relevant timestamp would be the execution's timestamp accessible through execution.Time.

[adamus]

Basically what you have then is something providing the wrong time. Should I have to come to your support forum to find this out? I don't think so. If somebody wants robust, reliable software that he/she can stake their money on, they won't want to see that kind of stuff. It raises questions about what is going on on the other side of that event code.

Anyway, I'll change the printed time to show the time stamp on the execution and I'll get back to you.

[NinjaTrader_Josh]

It is not the "wrong" time. It is just the time of whatever the pointer was pointing to at the moment in time the OnExecution() event triggered. Whether that is BIP0 or BIP1 depends on whichever bar update happened last. Since the OnExecution() event is not a bars tied event and as such going Time[] is just not useful in this context. Time[] is a bars tied concept and using it in a non-bars tied context would simply yield the results you saw.

[adamus]

I don't suppose there's much point discussing what's right or wrong here since I have zero influence over it. However in my logging it is a secondary concern to log the correct time especially during real time trading. I can live with out-of-sync time stamps in the backtesting logs but it would be frustrating to have to deal with them live.

The primary concern though is still that I can't test against the close of the actual 1 min timeframe bar that I am using to submit the orders.

[NinjaTrader_Josh]

You cannot backtest trade on the close of a signal bar because the bar has already closed. There is no more tradeable locations on that bar. A closed bar is terminal and is impossible to trade on. Next tradeable location is the open of the next bar and that is where you will see the trade show up. Unfortunately allowing the trade to go on a historical point in time in a backtest is "cheating".

[adamus]

When I submit a stop order I have to check that I am not submitting it on the wrong side of the market so I check the stop price against the close of the bar [0].

I don't' know how you would cheat in the way you describe. I submit orders relating to the bar at [0] which has just completed. I am used to the paradigm but I don't know how to cheat. Perhaps you should explain it further and give a line of code to show how it's done, and if I'm doing the same, then I'll realise.

Thanks

[NinjaTrader_Bertrand]

adamus, you could check on FirstTickOfBar (the 1min) before submitting while running the strategy with CalculateOnBarClose = false.

[adamus]

That's the way to check if my stop isn't the wrong side of the close? Or is that the cheat?

[NinjaTrader_Bertrand]

Sorry I don't follow you - the first tick of the new 1 min bar is your first location to submit the stop to, so if you check against this bar or the current bid / ask series you should be good.

[adamus]

You cannot backtest trade on the close of a signal bar because the bar has already closed. There is no more tradeable locations on that bar. A closed bar is terminal and is impossible to trade on. Next tradeable location is the open of the next bar and that is where you will see the trade show up. Unfortunately allowing the trade to go on a historical point in time in a backtest is "cheating".

All I know is that I sent over my strategy to you guys and one of you says my strategy is too complelx, one of you says I'm "cheating" and now you say I need to submit against the first tick of the new bar and basically none of this advice makes much sense to me.

As the title of the thread states, I think NT has got a problem. I thought there was more and I told you what I thought the symptoms were (with the BIP and the timestamp in the OrderUpdate & Execution events), so unless you say something to the contrary now, I'm going to stick with my strategy with the work-around, because it works in backtesting and it works in simulated trading, and maybe I'll have time to boil down my ninjascript into something short that shows the issue.

Thanks

[NinjaTrader_Josh]

Checking prices is not a "cheat". "Cheating" would be placing orders at a moment in time which has already passed. Meaning you cannot see data from the future and then after seeing such data decide that you wanted to back pedal and place a trade 1 second ago. This is already prohibited and not possible to be done in NT.

[gregschr]

After reading this thread, I believe I am having the same problem Adamus is having.

I am calculating indicator values on a 5 min bar (BIP=0) and entering trades on a 1 min bar(BIP=1). I submit a stop order in OnExecution after an entry is filled. I'm using ExitShortStop to submit the stop order.

Here is logging data from the Output window showing a Short entry at 1.2685 on the 6:33 bar. The stop order submitted at 1.2694 was rejected even though it is above the high of the 6:33 bar. I believe it was rejected because it is below the high of the 5 min bar at 6:35.

8/20/2010 06:33:00 EURUSD 5 Qty=0 Short Entry Conditions Met
8/20/2010 06:33:00 EURUSD 5 Qty=0 Short Entry
8/20/2010 06:33:00 EURUSD 5 Qty=0 High[0]=1.2692 < dSRValue-Stop0Ticks*TickSize=1.2694
8/20/2010 06:30:00 EURUSD 5 Qty=60000 In OnExecution
8/20/2010 06:30:00 EURUSD 5 Qty=60000 Order ID => NT-00021, Current Order => Order='NT-00021/Sim101' Name='Short' State=Filled Instrument='$EURUSD' Action=SellShort Limit price=0 Stop price=0 Quantity=0.06M Strategy='SineWaveBreakout_Dev' Type=Market Tif=Gtc Oco='' Filled=60000 Fill price=1.2685 Token='39bee9699d5a4cb7af4fe7d27a035be6' Gtd='12/1/2099 00:00:00'
8/20/2010 06:30:00 EURUSD 5 Qty=60000 Short Entry Fill at 1.2685
**NT** A BuyToCover stop order placed at '8/20/2010 06:30:00' has been ignored since the stop price is less than or equal to the close price of the current bar. This is an invalid order and subsequent orders may also be ignored. Please fix your strategy.
8/20/2010 06:30:00 EURUSD 5 Qty=60000 Submitted Stop S0, 60000 at 1.2694
8/20/2010 06:30:00 EURUSD 5 Qty=60000 Submitted Target T1, 20000 at 1.268


I've also attached a screen shot. The top frame is 5 minute bars, the bottom frame is 1 minute bars.

[adamus]

Is nobody interested in this issue anymore? Surely if it's not just me seeing the problem, then it's probably not a coding error on my part.

[adamus]

You cannot backtest trade on the close of a signal bar because the bar has already closed. There is no more tradeable locations on that bar. A closed bar is terminal and is impossible to trade on. Next tradeable location is the open of the next bar and that is where you will see the trade show up. Unfortunately allowing the trade to go on a historical point in time in a backtest is "cheating".

I'm writing a new strategy and I'm getting the same error message as I showed before when trying to submit stop orders in the OnExecution() event.


09/09/2010 14:28:44 Strategy A Buy order placed at '22/09/2009 02:00:00' has been ignored since the stop price is less than or equal to the close price of the current bar. This is an invalid order and subsequent orders may also be ignored. Please fix your strategy.
I work out where I want to place my stop based on the entry. How do I test that I have a valid price for my stop when the OnExecution method isn't linked to any bars and the close price of the current bar is inaccessible?

[gregschr]

Adamus,

I have not tried it yet, but I wonder if using ExitShortStop or ExitLongStop with the following signature will solve the problem.

ExitShortStop(int barsInProgressIndex, bool liveUntilCancelled, int quantity,double stopPrice,string signalName,string fromEntrySignal)

If I understand this correctly, barsInProgressIndex should link the order to the correct BIP.

[adamus]

Hi Greg, I already do that. It's not the BIP alone that's the problem, it's a combination of the logic for submitting orders (particularly in backtesting) and the hidden price check logic on stop orders that NT does.

There must be a logical method but I'm currently in the dark about it.

Just realised that it's also happening to Stop orders submitted during OnBarUpdate().

First things first, during OnBarUpdate(), I have access to the Close of the bar. So I want to use that to submit a stop order, but I'm generating a batch of NT errors saying that the Stop is the wrong side of the market. Not all the time, just about 25% of the time.

[adamus]

Would you give me your attention for a minute on this one? I'm beginning to suspect you've flagged this thread with "ignore".

The strategy I am writing is a multi-instrument strategy trading $EURUSD and $EURGBP.

I just looked through the TraceOrders output and realised that it is only throwing this error on long orders on the second instrument.

NT is consistently refusing to enter buy stop orders for $EURGBP stating that they are all invalid due to being lower than the market. This is even in places when the market never even reached that price!

So I figure what is happening is that NT is checking the stops against the primary bars. The price of EURUSD (e.g. 1.3456) is always going to be way above any EURGBP price (e.g 0.8989), so my Buy Stop will always look bad.

This explains why the Short stops are always fine.

I can send you my strategy if you give me a bit of time to pull in all the code snippets from the helper files.

[NinjaTrader_Bertrand]

Hi adamus, submitting to the correct BIP is needed as Greg pointed out - for sure we would like to take a look at the case if you have a reproducible scenario with a simplified version of your strategy - thanks for offering. You can direct it to support at ninjatrader dot com to my Attn.

[NinjaTrader_Bertrand]

Guys, please recheck your scenarios on our Beta 22 as we fixed a related issue. Thanks all for reporting in and participating.