Order cancelled when using liveUntilCancelled

[ntVic]

I'm seeing 1/2 of a reversing order I'm submitting with EnterShortLimit() get
canceled immediately after it's submitted, even though I'm passing true for
'liveUntilCancelled'.

The 'Close' part of the order remained in the market and ultimately got filled,
but my position went to flat, rather than short in this case, since the second
order to sell was cancelled right after being generated.

Here's the code fragment that's executing in OnExecution() after my strategy
enters a long position --

protected override void OnExecution(IExecution execution)
{
/// ... details removed for clarity
if ( Position.MarketPosition == MarketPosition.Long ) {
EnterShortLimit(0, true, Qty, reverselimitPrice, "****Target");
}
}

Here's the related part of the trace log -- for some reason a cancel gets
generated on the reversing part of the order right after it's generated.

Can anyone explain this ???


--- The fill, position is long 1, which executes
EnterShortLimit(0, true, Qty, reverselimitPrice, "****Target");
in OnExecution() --

2008-12-09 09:33:02:828 (InteractiveBrokers)
Cbi.ExecutionEventArgs.Process:
Execution='0d63a406a75c412cad974cc557bec314' Instrument='YM 12-08'
Account='Sim101' Exchange=Default Price=8795 Quantity=1 Market
position=Long Operation=Insert Order='722f2187182448aca3396b6'
Time='12/9/2008 9:33:02 AM'


--- Close position order is generated ---
2008-12-09 09:33:02:937 (InteractiveBrokers)
Cbi.Connection.ProcessEventArgs.OrderStatusEventAr gs:
Order='bb5710da54ca4c1582c2d159d1f29786/Sim101'
Name='Close position' New State=Initialized Instrument='YM 12-08'
Action=Sell Limit price=8893 Stop price=0 Quantity=1 Type=Limit
Filled=0 Fill price=0 Error=NoError Native error=''

--- Short order is generated ---
2008-12-09 09:33:02:937 (InteractiveBrokers)
Cbi.Connection.ProcessEventArgs.OrderStatusEventAr gs:
Order='137a3263a7ab47e2b9ef614ca1f368c6/Sim101'
Name='****Target' New State=Initialized Instrument='YM 12-08'
Action=SellShort Limit price=8893 Stop price=0 Quantity=1
Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''

--- What's causing this cancel to get generated ???? ---
2008-12-09 09:33:02:937 Strategy.StrategyBase.Process.ExecutionUpdateEvent Args:
cancel order: Order='3cda6f89a91b4e3e9947d1a259bf1757/Sim101'
Name='****Target' State=PendingSubmit Instrument='YM 12-08'
Action=SellShort Limit price=8893 Stop price=0 Quantity=1
Strategy='AutoTest' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0
Token='**7' Gtd='12/1/2099 12:00:00 AM'

2008-12-09 09:33:02:937 (InteractiveBrokers)
Cbi.Order.QueueCancel: order queued, since it's in state 'PendingSubmit':
Order='3cda6f89a91b4e3e9947d1a259bf1757/Sim101'
Name='****Target' State=PendingSubmit Instrument='YM 12-08'
Action=SellShort Limit price=8893 Stop price=0 Quantity=1
Strategy='AutoTest' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0
Token='***' Gtd='12/1/2099 12:00:00 AM'


--- Close position order working ---
2008-12-09 09:33:03:296 (InteractiveBrokers)
Cbi.OrderStatusEventArgs.Process: Order='ac08/Sim101' Name='Close position'
New State=Working Instrument='YM 12-08' Action=Sell Limit price=8893
Stop price=0 Quantity=1 Type=Limit Filled=0 Fill price=0 Error=NoError
Native error=''

--- Short order accepted ---
2008-12-09 09:33:03:296 (InteractiveBrokers)
Cbi.OrderStatusEventArgs.Process: Order='***/Sim101' Name='****Target'
New State=Accepted Instrument='YM 12-08' Action=SellShort
Limit price=8893 Stop price=0 Quantity=1 Type=Limit Filled=0
Fill price=0 Error=NoError Native error=''

--- Cancel on Short order sent ---
2008-12-09 09:33:03:296 (InteractiveBrokers)
Cbi.Globals.ProcessEventArgsInThreadContext.RetryC ancel:
Order='3cda6f89a91b4e3e9947d1a259bf1757/Sim101'
Name='****Target' State=Working Instrument='YM 12-08'
Action=SellShort Limit price=8893 Stop price=0 Quantity=1
Strategy='AutoTest' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0
Token='3cda6f89a91b4e3e9947d1a259bf1757' Gtd='12/1/2099 12:00:00 AM'

2008-12-09 09:33:03:296 (InteractiveBrokers)
Cbi.Order.CancelNow: Order='3cda6f89a91b4e3e9947d1a259bf1757/Sim101'
Name='****Target' State=Working Instrument='YM 12-08'
Action=SellShort Limit price=8893 Stop price=0 Quantity=1
Strategy='AutoTest' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0
Token='3cda6f89a91b4e3e9947d1a259bf1757' Gtd='12/1/2099 12:00:00 AM'


--- Short order canceled (position went to flat later instead of short)
when the Close order was filled ---

2008-12-09 09:33:03:609 (InteractiveBrokers)
Cbi.OrderStatusEventArgs.Process: Order='***/Sim101' Name='****Target'
New State=Cancelled Instrument='YM 12-08' Action=SellShort
Limit price=8893 Stop price=0 Quantity=1 Type=Limit Filled=0
Fill price=0 Error=NoError Native error=''

[NinjaTrader_Bertrand]

Hi ntVic,

Welcome to the NinjaTrader forum.

Please add TraceOrders = True to your Initialize() to debug where the cancellation comes from.

Here is a link to our helpguide for TraceOrders - http://www.ninjatrader-support.com/H...aceOrders.html

[ntVic]

Hi,

So I ran the strategy on the Market Replay connection.
Here's the relevant output with TraceOrders = true;


12/9/2008 9:33:02 AM Entered internal PlaceOrder() method at 12/9/2008 9:33:02 AM: Action=Buy OrderType=Limit Quantity=1 LimitPrice=8819 StopPrice=0 SignalName='****Buy' FromEntrySignal=''

12/9/2008 9:33:02 AM Entered internal PlaceOrder() method at 12/9/2008 9:33:02 AM: Action=SellShort OrderType=Limit Quantity=1 LimitPrice=8893 StopPrice=0 SignalName='****Target' FromEntrySignal=''

12/9/2008 9:33:02 AM Cancelled pending entry order on opposite side of the market: BarsInProgress=0: Order='3b43d5cddcb14e139b43faa7c8defb5f/Replay101' Name='****Target' State=Working Instrument='YM 12-08' Action=SellShort Limit price=8893 Stop price=0 Quantity=1 Strategy='Auto****' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='3b43d5cddcb14e139b43faa7c8defb5f' Gtd='12/1/2099 12:00:00 AM'


So that 'Cancelled pending entry order on opposite side of the market'
line seems odd -- what does 'opposite side of the market' mean ?
Is it 'opposite side of current position' ?

[NinjaTrader_Josh]

Please post the complete logs as outputted by TraceOrders.

What kind of orders were concurrently in play when you submitted the EnterShortLimit()?

[ntVic]

Hi, those are the only entries up to the cancel for that day.
As far as I know there were no other orders currently in play.

On the prior day the strategy had a stop order resting in the market,
and so there are pages of lines like the first two below saying the order
is already in the market, up to the point where that stop order was
cancelled after the close yesterday, as shown by the line at
12/8/2008 4:06:00 below.

...
12/8/2008 4:04:00 PM Entered internal PlaceOrder() method at 12/8/2008 4:04:00 PM: Action=SellShort OrderType=Stop Quantity=1 LimitPrice=0 StopPrice=8791 SignalName='***Sell' FromEntrySignal=''
12/8/2008 4:04:00 PM Ignore order amendment: Action=SellShort OrderType=Stop Quantity=1 LimitPrice=0 StopPrice=8791 SignalName=RunGapUp' FromEntrySignal='' Reason='Order already has this stop price/limit price/quantity'

Cancelled custom managed order at 12/8/2008 4:06:00 PM: Order='NT-00016/Replay101' Name='***Sell' State=Working Instrument='YM 12-08' Action=SellShort Limit price=0 Stop price=8791 Quantity=1 Strategy='DowGapOpenShortAuto' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='08ce795c26f74172a3042dfe93ccbcba' Gtd='12/1/2099 12:00:00 AM'


12/9/2008 9:33:02 AM Entered internal PlaceOrder() method at 12/9/2008 9:33:02 AM: Action=Buy OrderType=Limit Quantity=1 LimitPrice=8819 StopPrice=0 SignalName='***Buy' FromEntrySignal=''
12/9/2008 9:33:02 AM Entered internal PlaceOrder() method at 12/9/2008 9:33:02 AM: Action=SellShort OrderType=Limit Quantity=1 LimitPrice=8893 StopPrice=0 SignalName='***Target' FromEntrySignal=''
12/9/2008 9:33:02 AM Cancelled pending entry order on opposite side of the market: BarsInProgress=0: Order='3b43d5cddcb14e139b43faa7c8defb5f/Replay101' Name='***Target' State=Working Instrument='YM 12-08' Action=SellShort Limit price=8893 Stop price=0 Quantity=1 Strategy='Auto****' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='3b43d5cddcb14e139b43faa7c8defb5f' Gtd='12/1/2099 12:00:00 AM'

[NinjaTrader_Josh]

Stop order in which direction? This order was not active when you tried to submit your new EnterShortLimit() correct? What I am interested in are the active orders and positions as you sent EnterShortLimit(). Thank you.

[ntVic]

Hi, sorry for the confusion -- notice the first trace log entry in my previous post shows a sell stop -- Action=SellShort OrderType=Stop

And you're correct -- that sell stop order was NOT active when EnterShortLimit() was called the next day -- that stop order was active
on the previous day, and cancelled prior to the previous day's close.

EnterShortLimit() was called in OnExecution() in response to a fill on a BUY order.

There were no open positions and no other orders when that order was filled, so the position went from flat to long 1 contract at the time, and
the 'Control Center' didn't show any open orders -- the buy order went to a 'Filled' state, EnterShortLimit() was then executed, which generated two
orders to sell -- the 'Close' order and the 'Target' order -- and then the target order was immediately cancelled, which generated the 'Cancelled
pending entry order on opposite side of the market' message in the trace log --

12/9/2008 9:33:02 AM Cancelled pending entry order on opposite side of the market: BarsInProgress=0: Order='3b43d5cddcb14e139b43faa7c8defb5f/Replay101' Name='****Target' State=Working Instrument='YM 12-08' Action=SellShort Limit price=8893 Stop price=0 Quantity=1 Strategy='Auto****' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='3b43d5cddcb14e139b43faa7c8defb5f' Gtd='12/1/2099 12:00:00 AM'

[NinjaTrader_Josh]

Are you sure ntVic? Your logs say this:

12/9/2008 9:33:02 AM Entered internal PlaceOrder() method at 12/9/2008 9:33:02 AM: Action=Buy OrderType=Limit Quantity=1 LimitPrice=8819 StopPrice=0 SignalName='***Buy' FromEntrySignal=''
12/9/2008 9:33:02 AM Entered internal PlaceOrder() method at 12/9/2008 9:33:02 AM: Action=SellShort OrderType=Limit Quantity=1 LimitPrice=8893 StopPrice=0 SignalName='***Target' FromEntrySignal=''

It seems to me that you were trying to enter long and enter short at the same time. This is not possible. You cannot submit orders in opposite directions simultaneously.

[ntVic]

Yes that's true, but notice from the trace log output in my original post that the fill on the first PlaceOrder() call in the trace order output fragment you highlight happened in the same second that the order was placed -- in this case my buy limit price of 8819 was above the current market price of about 8795, so the order was filled immediately. The resolution on the timestamps in the trace order output are to the second so that fact is concealed in that output, whereas the timestamps show more precision in the trace log written in My Documents\NinjaTrader 6.5\trace --

--- Fill from buy limit order ---
2008-12-09 09:33:02:828 (InteractiveBrokers)
Cbi.ExecutionEventArgs.Process:
Execution='0d63a406a75c412cad974cc557bec314' Instrument='YM 12-08'
Account='Sim101' Exchange=Default Price=8795 Quantity=1 Market
position=Long Operation=Insert Order='722f2187182448aca3396b6'
Time='12/9/2008 9:33:02 AM'


--- Close position order is generated ---
2008-12-09 09:33:02:937 (InteractiveBrokers)
Cbi.Connection.ProcessEventArgs.OrderStatusEventAr gs:
Order='bb5710da54ca4c1582c2d159d1f29786/Sim101'
Name='Close position' New State=Initialized Instrument='YM 12-08'
Action=Sell Limit price=8893 Stop price=0 Quantity=1 Type=Limit
Filled=0 Fill price=0 Error=NoError Native error=''

--- Short order is generated ---
2008-12-09 09:33:02:937 (InteractiveBrokers)
Cbi.Connection.ProcessEventArgs.OrderStatusEventAr gs:
Order='137a3263a7ab47e2b9ef614ca1f368c6/Sim101'
Name='****Target' New State=Initialized Instrument='YM 12-08'
Action=SellShort Limit price=8893 Stop price=0 Quantity=1
Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''

--- Cancel generated ---
2008-12-09 09:33:02:937 Strategy.StrategyBase.Process.ExecutionUpdateEvent Args:
cancel order: Order='3cda6f89a91b4e3e9947d1a259bf1757/Sim101'
Name='****Target' State=PendingSubmit Instrument='YM 12-08'
Action=SellShort Limit price=8893 Stop price=0 Quantity=1
Strategy='AutoTest' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0
Token='**7' Gtd='12/1/2099 12:00:00 AM'


Is a race condition possible here ? It seems like even though the execution event is shown to occur a fraction of a second before EnterShortLimit() is invoked,
the buy order is still somehow being treated as active.

Those two PlaceOrder() entries at 12/9/2008 9:33:02 AM in the trace order output you highlight must correspond to the EnterLongLimit() call that produced the long position, and the EnterShortLimit() call that got triggered in OnExecution() from the buy fill, since those are the only two calls shown in the trace order output up to that time, and the trace log only shows the fill and the short order at that time.

[NinjaTrader_Josh]

You are definitely hitting a race condition. Depending on how you are coding this. What are you checking? The IOrder for a fill or the actual execution event for a fill? You cannot check the IOrder for a state of filled because the IOrder can be filled while you have not received all the execution events for that fill yet. You need to run your own check for fill manually via tracking the quantity of the execution events received. execution.Quantity and compare it to your order quantity.

[ntVic]

So I wasn't using the IExecution value being passed to OnExecution().
I'm just checking Position.MarketPosition (so I'll fix that), but in this case the order was for a single contract, so a partial fill wasn't possible.

So given that, in this case shouldn't the check 'Position.MarketPosition == MarketPosition.Long' be sufficient to safely call EnterShortLimit() ?


I was also wondering if OnBarUpdate() might be executed at the same time as OnExecution() as a result of multi-threading, but note that I have
CalculateOnBarClose = true;
being set in Initialize(), so I was thinking OnBarUpdate() wouldn't be called again until the next bar started forming (this was a 1 minute chart).

Here's the code fragment that's being executed in this case --

protected overridevoid OnBarUpdate()
{
 /// ... details removed for clarity
  if ( 93300 <= ToTime(Time[0]) && ToTime(Time[0]) < 160500 )
  {
    // ... compute BuySignal, long entered here
    if ( (Position.MarketPosition != MarketPosition.Long) && BuySignal == true ) {
       entryOrder = EnterLongLimit(0, true, Qty, limitPrice, "***Buy");
    }
  }
}
 
protected override void OnExecution(IExecution execution)
{
    if ( Position.MarketPosition == MarketPosition.Long ) {
           entryOrder = EnterShortLimit(0, true, Qty, reverselimitPrice, "****Target");
    }
}

[NinjaTrader_Josh]

I recommend you not use MarketPosition from OnExecution. If you are using OnExecution I recommend you access the IExecution object directly. You may even want to go even further down and just use OnPositionUpdate.

[ntVic]

Ok. The documentation recommends using OnExecution() to drive strategy logic based on fills, and states that On Execution() is always called after OnOrderUpdate() -- but when is OnPositionUpdate() called ??

I didn't get a clear idea of the difference between OnExecution() and OnPositionUpdate() from the documentation.

Here's the section of the guide from OnOrderUpdate() I mentioned --

http://www.ninjatrader-support.com/H...?OnOrderUpdate

• CRITICAL: If you want to drive your strategy logic based on order fills you must use OnExecution() instead of OnOrderUpdate(). OnExecution() is always triggered after OnOrderUpdate(). There is internal strategy logic that is triggered after OnOrderUpdate() is called but before OnExecution() that can adversely affect your strategy if you are relying on tracking fills within OnOrderUpdate().

Thanks for all the help, and the quick replies -- I wish the support people at TradeStation would respond like this.

[NinjaTrader_Josh]

OnPositionUpdate comes after OnExecution. It is easiest to program a closed loop from OnPositionUpdate. You need to be careful though, you are not guaranteed to receive an OnPositionUpdate() event for every execution event.

[ntVic]

I'd like to understand the difference between OnExecution() and OnPositionUpdate() better.

Regarding OnPositionUpdate(), the documentation says
The OnPositionUpdate() method is called everytime a position managed by a strategy changes state.

So when would an execution for a strategy NOT generate an OnPositionUpdate() event ? My read of the documentation makes it seem like that shouldn't be possible.