Phantom trade

[Valyo]

Are you using Low[] within the same condition as a call to SetStopLoss(), by chance? I've just discovered an issue with that setup which can cause Low[] to return unexpected values.

In terms of debugging, the first thing I would do is to print the values of anything you are using to set your order prices immediately before placing orders. Something like this:

if(condition)
{
Print(Time[0] + "Low[0] + TickSize = " + Low[0] + TickSize);
EnterLongLimit(Low[0] + TickSize);
}

Obviously adjusting as needed to print out the variables and properties that are relevant to where/when/how you are placing orders. This way, you can see the data under the hood that the strategy is working with, which may not necessarilly be what you expect, based on what you are seeing on the chart. The point is to get the strategy to tell you as much as possible about the values that it is accessing and using to place and modify orders, including timestamps to show you the exact sequence of when things are being done.

You might consider checking TraceOrders, as well, for even greater detail on how, when, and where orders are being submitted.

Dave, there is no StopLoss. I am using Input[] to evaluate boolean values (conditions for entry orders). If you think that using Input[] can cause unexpected prices, I can replace it with Close[].

The entry orders are not Limit or Stop orders. They are just EnterLong/EnterShort:

this.myEntryOrder = EnterLong(this.orderQuantityScaled, "Enter Long Signal");
this.myEntryOrder = EnterShort(this.orderQuantityScaled, "Enter Short Signal");

So I don't set order prices.

OK, I will debug the strategy on Brian's machine, with output of Close[] prices immediately before placing orders. And I will enable TraceOrders.

[Valyo]

Dave, I tested the strategy on Brian's laptop with TraceOrders enabled and following outputs:

• in OnBarUpdate(), right before EnterLong()/EnterShort() and ExitLong()/ExitShort()
• in OnExecution(), when execution has corresponding order

I attach a short version of output file with the last orders and executions, the long version is 3.9 MB. If you need details for previous orders, I will upload them.

Here are some details:

There are executions with price 91.655:

6/23/2015 7:00:00 PM Entered internal PlaceOrder() method at 6/23/2015 7:00:00 PM: BarsInProgress=0 Action=SellShort OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Enter Short Signal' FromEntrySignal=''
Method: OnExecution
execution.Time: 6/23/2015 7:00:00 PM
execution.Price: 91.655
execution.order.Time: 6/23/2015 7:00:00 PM
execution.order.OrderAction: Sell
execution.order.AvgFillPrice: 91.655
--------------------------------------------------------
Method: OnExecution
execution.Time: 6/23/2015 7:00:00 PM
execution.Price: 91.655
execution.order.Time: 6/23/2015 7:00:00 PM
execution.order.OrderAction: SellShort
execution.order.AvgFillPrice: 91.655
--------------------------------------------------------

The last EnterShort() order before this execution is at 6/23/2015 3:50:00 PM.
The price at this time is Input[0]: 95.605, Close[0]: 95.605.

There is an output from OnExecution() with the same price 95.605. This output is after the the output with EnterShort() at 6/23/2015 3:50:00 PM, but the timestamp is 6/23/2015 3:46:28 PM - before 6/23/2015 3:50:00 PM.

After that, there are several outputs with one and the same EnterLong() order (before the output with execution at 91.655):

Method: OnBarUpdate, right before EnterLong()
Time[0]: 6/23/2015 4:00:00 PM
Input[0]: 95.635, Close[0]: 95.635

And there is another output with EnterShort():

Method: OnBarUpdate, right before EnterShort()
Time[0]: 6/23/2015 7:05:00 PM
Input[0]: 95.58, Close[0]: 95.58

It is before the output with execution at 91.655, but timestamp is 6/23/2015 7:05:00 PM and it is after the timestamp of the execution 6/23/2015 7:00:00 PM.

The next output is when Session Loss Limit is reached:

Method: OnBarUpdate, right before ExitLong()/ExitShort()
Time[0]: 6/23/2015 7:05:00 PM
Input[0]: 95.58, Close[0]: 95.58
Session Loss Limit reached, wait for next session

But the order is executed at price 102:

6/23/2015 7:00:00 PM Entered internal PlaceOrder() method at 6/23/2015 7:00:00 PM: BarsInProgress=0 Action=Sell OrderType=Market Quantity=0 LimitPrice=0 StopPrice=0 SignalName='Exit Long Signal' FromEntrySignal='Enter Long Signal'
6/23/2015 7:00:00 PM Ignored PlaceOrder() method: Action=Sell OrderType=Market Quantity=0 LimitPrice=0 StopPrice=0 SignalName='Exit Long Signal' FromEntrySignal='Enter Long Signal' Reason='This was an exit order but no position exists to exit'
6/23/2015 7:00:00 PM Entered internal PlaceOrder() method at 6/23/2015 7:00:00 PM: BarsInProgress=0 Action=BuyToCover OrderType=Market Quantity=0 LimitPrice=0 StopPrice=0 SignalName='Exit Short Signal' FromEntrySignal='Enter Short Signal'
Method: OnExecution
execution.Time: 6/23/2015 7:00:00 PM
execution.Price: 102
execution.order.Time: 6/23/2015 7:00:00 PM
execution.order.OrderAction: BuyToCover
execution.order.AvgFillPrice: 102

It seems that something happens with prices between OnBarUpdate() and OnExecution(). And the timestamps of the output are not in the correct order.

[NinjaTrader_DaveI]

Thank you for taking the time to put that together. I will review the scenario and get back to you as soon as I can with some insight.

[NinjaTrader_PatrickH]

Hello Valyo & Briansstocks,

I don't think this is the strategy. I think it is the replay data.

Can you run the SampleMaCrossover over the same test and advise if it produces an incorrect trade as an invalid level?

[Valyo]

Hello Valyo & Briansstocks,

I don't think this is the strategy. I think it is the replay data.

Can you run the SampleMaCrossover over the same test and advise if it produces an incorrect trade as an invalid level?

Hello Patrick,

I also think it is replay data. Deleting corrupted data and downloading replay data again sometimes helps and sometimes doesn't help. If it is replay data, it is corrupted by the download process (?).

I can test SampleMaCrossover with the same replay data and with the same output. But the strange prices are on particular bars and even on particular ticks (the strategy works on each incoming tick). There is no guarantee that SampleMaCrossover will generate orders with the same timestamps.

Is there any way to look at the replay data directly? Could it be converted to CSV or XML file for example?

[NinjaTrader_PatrickH]

Hello Valyo,

Thank you for your response.

That is true for the SampleMaCrossover. I will create my own test to run over the data on my end to isolate what appears to be corrupted data.

Please advise what time zone the two PCs are in for your tests.

[Valyo]

I am at EEST – Eastern European Summer Time. Brian is 8 hours behind me, so he is UTC - 5:00.

The problem is with the beginning of the session on 6/23/2015.

This is from my latest output:

6/23/2015 3:45:55 PM Entered internal PlaceOrder() method at 6/23/2015 3:45:55 PM: BarsInProgress=0 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Enter Long Signal' FromEntrySignal=''
6/23/2015 3:45:55 PM Ignored PlaceOrder() method at 6/23/2015 3:45:55 PM: Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Enter Long Signal' FromEntrySignal='' Reason='Exceeded entry signals limit based on EntryHandling and EntriesPerDirection properties'
Method: OnBarUpdate, right before EnterShort()
Time[0]: 6/23/2015 3:50:00 PM
Input[0]: 95.605, Close[0]: 95.605
--------------------------------------------------------
...
...
...
6/23/2015 3:59:50 PM Entered internal PlaceOrder() method at 6/23/2015 3:59:50 PM: BarsInProgress=0 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Enter Long Signal' FromEntrySignal=''
6/23/2015 3:59:50 PM Ignored PlaceOrder() method at 6/23/2015 3:59:50 PM: Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Enter Long Signal' FromEntrySignal='' Reason='Exceeded entry signals limit based on EntryHandling and EntriesPerDirection properties'
Time: 6/23/2015 7:05:00 PM, Historical: False, FirstBarOfSession: True, FirstTickOfBar: True
--------------------------------------------------------
Method: OnBarUpdate, right before EnterShort()
Time[0]: 6/23/2015 7:05:00 PM
Input[0]: 95.58, Close[0]: 95.58
--------------------------------------------------------
6/23/2015 7:00:00 PM Entered internal PlaceOrder() method at 6/23/2015 7:00:00 PM: BarsInProgress=0 Action=SellShort OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Enter Short Signal' FromEntrySignal=''
Method: OnExecution
execution.Time: 6/23/2015 7:00:00 PM
execution.Price: 91.655
execution.order.Time: 6/23/2015 7:00:00 PM
execution.order.OrderAction: Sell
execution.order.AvgFillPrice: 91.655
--------------------------------------------------------
Method: OnExecution
execution.Time: 6/23/2015 7:00:00 PM
execution.Price: 91.655
execution.order.Time: 6/23/2015 7:00:00 PM
execution.order.OrderAction: SellShort
execution.order.AvgFillPrice: 91.655
--------------------------------------------------------
Method: OnBarUpdate, right before ExitLong()/ExitShort()
Time[0]: 6/23/2015 7:05:00 PM
Input[0]: 95.58, Close[0]: 95.58
Session Loss Limit reached, wait for next session
--------------------------------------------------------
6/23/2015 7:00:00 PM Entered internal PlaceOrder() method at 6/23/2015 7:00:00 PM: BarsInProgress=0 Action=Sell OrderType=Market Quantity=0 LimitPrice=0 StopPrice=0 SignalName='Exit Long Signal' FromEntrySignal='Enter Long Signal'
6/23/2015 7:00:00 PM Ignored PlaceOrder() method: Action=Sell OrderType=Market Quantity=0 LimitPrice=0 StopPrice=0 SignalName='Exit Long Signal' FromEntrySignal='Enter Long Signal' Reason='This was an exit order but no position exists to exit'
6/23/2015 7:00:00 PM Entered internal PlaceOrder() method at 6/23/2015 7:00:00 PM: BarsInProgress=0 Action=BuyToCover OrderType=Market Quantity=0 LimitPrice=0 StopPrice=0 SignalName='Exit Short Signal' FromEntrySignal='Enter Short Signal'
Method: OnExecution
execution.Time: 6/23/2015 7:00:00 PM
execution.Price: 102
execution.order.Time: 6/23/2015 7:00:00 PM
execution.order.OrderAction: BuyToCover
execution.order.AvgFillPrice: 102
--------------------------------------------------------

[NinjaTrader_PatrickH]

Hello Valyo and Brian,

Would I be able to test the script on my end?

If so, can you send it to platformsupport[at]ninjatrader[dot]com with this thread in the subject line?

[NinjaTrader_PatrickH]

Hello Valyo,

Thank you for your patience.

I am able to reproduce the incorrect execution on the 3rd using the 06-15, but not on the 8th or 23rd with the 09-15. By chance do you see incorrect execution when running a backtest in the Strategy Analyzer?

[Valyo]

Hello Patrick,

I have not tested the strategy with Strategy Analyzer, because it is supposed to process each incoming tick. Brian said that this problem with the prices is inconsistent. Maybe you will see it sometimes only. I hope that you could find the reason for the incorrect execution on the 3rd using the 06-15 and it will be the same for all cases.

[NinjaTrader_PatrickH]

Hello Valyo,

Thank you for your patience.

I am unable to reproduce with other strategies. Can you test the strategy on Sim101 and real-time data? Do you see the same issue at any points?

[Valyo]

Hello Valyo,

Thank you for your patience.

I am unable to reproduce with other strategies. Can you test the strategy on Sim101 and real-time data? Do you see the same issue at any points?

Hello Patrick,

I can ask Brian to run the strategy for several hours on Sim101 and real-time data. It could be helpful to test the strategy on session end and the first bar of next session.

But this test would be helpful only if we get the same strange prices. If everything is OK, the test doesn't prove anything.

I think that you could figure out what happens if you could track the wrong prices backward. I mean that the prices we see on the UI are provided by some method or are saved somewhere. So if you could track these wrong prices backward step-by-step, finally you could find the root of the problem.

Thanks,

Valentin

[Briansstocks]

Patrick,

The phantom trade happens at 1700 every time. Since all 3 of us have seen the phantom trade I think we can say it is not on our end. You reproduced the phantom trade 1 time on your end before I sent you the script.

The issue isn't going to go away by running the same test from multiple computers, We have all seen it, I have run the tests on 4 different computers myself not counting you or Valentin.

Running the script in Sim101 with real time data is not very realistic. I just ran the same test on DX 9-15 from 6-21 to 7-10 the phantom trade didn't show up till the 7th or 8th. How long am I supposed to run a test on live data?

I get different results regularly. I changed the trade triggers and re ran the same test with the same computer and data and got a different phantom trade. I changed the trade triggers back to where I started and then the Phantom trade happened at the same time as test #2 but not the same as test #1 when I originally ran the test.

The script was completed well within the agreed time but for a month we have been stuck in the same place and I cannot complete testing the script. Something is not working, Programmer has not been paid yet, I have a expensive piece of software that don't do me any good and money tied up in a brokerage account I don't even want.

You can't seriously say that you would personally go live with an untested strategy or one that works in Sim101 with real time data but not in Market replay. Who knows what would happen if I tried to turn it on live.

I paid for a working program, I don't have one. You confirmed that this issue happened to you without the script. When can we expect a resolution from NT so we can finish testing the script and see if there is any problems in the script or if the only problem we have is being caused by NT?

Regards
Brian

[NinjaTrader_PatrickH]

Hello Brian,

Thank you for your response.

We would not recommend going live with a system before you are comfortable. I do see the same behavior with the strategy in Market Replay, but I cannot reproduce with any other strategy.

I am asking if you can test on real-time data but use the simulation account to see if the strategy produces this behavior. This is only a test to verify if the strategy performs the same on real-time data as in Market Replay.

The data does not produce the offset ticks to produce the fills in Market Replay, so it is possible it has to do with the replay fill engine - the same one used for the simulation engine for the Sim101 account. So testing on real-time with the Sim101 would potentially produce the same case.

I will continue to test the Market Replay with other strategies on my end.

[Briansstocks]

Patrick, here is the output file for the Sim101 test with live data, the phantom trade did not happen today.