Trying to eliminate false intrabar back tested winners

[shodson]

So I have a strategy that, when backtested, shows intrabar winners that, in a live market, are actually losers. When I enter I set stops and targets and I get false winners when my target is smaller than the bar's range. See the attached charts on a 5min time frame. All entries are limit orders on BarsIndex=0

I thought I could mitigate this problem by adding a lower time-frame data series like

Add(PeriodType.Tick, 1);

And thought that my stops and targets would be respected, tick-by-tick, but for some reason the stops and targets are resolved only on the 5min time frame. I then tried entering on BarsIndex=1 but when I back test that no orders are placed during the back test.

So, is there any way to back test my trades so they are more inline to how the market truly acts and acurately track how it would work in a live market when stops/targets are smaller than the size of a bar's range?

[Trader.Jon]

shodson,

I feel your pain.. this is a challenge. You can setup slippage but that IMHO isnt real solution.

ATM I am rewriting the strategy and building indicators that are MTF so I dont rely inside the strategy for indicators to be running on BIP=0 at all, only used for trade entry.

Jon

[NinjaTrader_Bertrand]

shodson, one way to backtest more accurately would be adding a bars since entry check to submit the stops and targets for the next bar only so they would not fill intrabar in your case.

[shodson]

@Trader.Jon - adding a second time frame won't help. I thought it would but it doesn't. As I mentioned in my post, I had a 2nd timeframe of 1 tick and I thought that Ninja would examine/respect stops and targets on the 1 tick time frame, but it doesn't. Then I tried placing my orders on the 1-tick time frame, but no trades ever fired in the backtest when I did that. I just think this is a major flaw with Ninjatrader that hasn't been addressed.

@Bertrand - Thanks for your suggestion. While this would help avoid false intrabar winners/losers, it will also eliminate intrabar winners/losers that you would get in a live market, or at least, the wins/losses would be different than in a backtest. Is there any plan to fix Ninjatrader to have stops and targets actually monitored and executed on lower timeframes during a backtest so we can get better backtesting results? This seems to be a major flaw toward any system analysis.

Also, if I set a target and stop for a trade, then the trade completes, how to I clear a stop loss or profit target? If I SetProfitTarget() and SetStopLoss() using ticks, every entry I make will have that same stop/loss parameters, unless I set it with different values. But how do I then later enter a trade that won't have a stop/loss order placed automatically after that, so I can do it on the next bar instead of at entry?

[Baruch]

adding a second time frame won't help. I thought it would but it doesn't.

Of course it works. I do it all the time.
Your problem is:
1. You don't have tick data for the tested time
2. You coded it wrong

Baruch
p.s.
Don't use SetStopLoss(). Use IOrder.

[NinjaTrader_Bertrand]

As Baruch pointed out with using the Exit() methods / IOrders you could more accurately code this out, as you could submit your stop / target order directly to a finer series.

With the IOrder methods you would also get more control on your orders, such as the ability to CancelOrder() them -

http://www.ninjatrader.com/support/helpGuides/nt7/iorder.htm

http://www.ninjatrader.com/support/helpGuides/nt7/cancelorder.htm

[shodson]

Hmm, use IOrder. So what method(s) do I call to "use IOrder" other than the usual EnterLong/Short? Are you saying instead of using SetStopLoss() and SetProfitTarget() I manually set up my OCO target/limit and stop loss orders? So if I was entering long, then I would set up two ExitLongLimit() orders at my price target and at my stop loss level like an OCO? Or are you saying if I had, say, a 1-tick lower time frame, after each tick I just check to see if it hit my target/loss level and then just ExitLong() or ExitLongLimit() once I hit one of them? Actually, I think I've done the latter on a different strategy, I guess I just forgot about that! I'll give that a try.

[NinjaTrader_Bertrand]

Yes, would need to use the Exit() methods which provide IOrder object return to be able to manage those as needed, as they would offer more advanced overloads offering to submit to a particular series. Your other approach should work as well, monitor a condition on the finer tick chart and ExitLong / ExitShort then.

[adamus]

shodson, can't you just set your faster time frame to be your primary timeframe, and if you need indicators on the original timeframe, you add that timeframe as the secondary timeframe and pass the secondary bars into the indicator when initialising it.

be aware when coding your own target and stops without SetStopLoss and SetProfitTarget. When I tried it, NT7 will not make them OCO with your broker, NT7 will simulate that, in other words you can't just leave the orders in the market and turn off NT7 - the 2 orders will not be OCO.

[Trader.Jon]

@Trader.Jon - adding a second time frame won't help. I thought it would but it doesn't. As I mentioned in my post, I had a 2nd timeframe of 1 tick and I thought that Ninja would examine/respect stops and targets on the 1 tick time frame, but it doesn't. Then I tried placing my orders on the 1-tick time frame, but no trades ever fired in the backtest when I did that. I just think this is a major flaw with Ninjatrader that hasn't been addressed.

shodson,

My experience is somewhat parallel. To explain more explicitly, I am trying to do a workaround with making the indicators multitimeframe, including the counting of bars in the trade etc, (only BIP=0 is in the strategy logic, counting bars since entry is done on an indicator counting bars on 5 range, entry conditions are done with various indicators built on 3, 4, 5 range and 30 sec time frames.

Jon

[bbucha_stk]

I have been challenged by results being much different on Back test vs Market replay.
Just to throw in a wish list it would be nice to be able to run optimization against Market Replay. Maybe get closer to settings that will experience on real data...

[luitom]

Problems with backtesting

I am trying to backtest a breakout strategy (which enters long or short on the high and low of a certain bar) with ninjatrader; I have minute and tick historical data. unfortunately, they both do not work. The minute historical executes trades on the following bar (not on the actual bar where the entry condition becomes true) therefore the entire strategy becomes useless. On the other hand tick historical does not seem to executes trades the right way and sometimes gaps appear which create unrealistic stop exits. Has anybody encountered my problems and found a way to solve them?

[billr]

*I decided to edit this post out. It read more negative than my intent. Thanks.*

[bbucha_stk]

Hello All-
I must also agree with you. I spent many many months doing baqcktest thinking I had a solution to find out when I started testing against Market Replay data I had vastly different results. So then I have been manually tweaking Market Replay...IS it going to prove vasltly different than Live????
Yes I have read their disclaimers on why there are difference, but a 50 to 70% difference betwen BAckTest and Replay....I think there are issues.
I hope they read and understand that when 10 people complain ther are 100's more back there somewhere!