Backtesting results v7 vs v6.5

[lookOutBelow]

I'm getting different results from backtesting the same strategies with the same settings on version 6.5 and 7.0. Is this normal?

I'd think that they would behave the same. These strategies that I've written aren't really that complicated. I wouldn't think it would cause problems.

Over the last week, the difference is between a 83% success rate and a 62%. Enough to cause concern.

Since I cannot open both v6.5 and v7 at the same time, it is hard to recognize what exactly is going on between the two. I can tell you that the strategy made 6 trades on the v6.5 one, and 8 on the v7. Only one trade was the same on both versions. The rest were completely different.

Any ideas on why this is happening? Is there a list somewhere warning us of what type of things would cause these problems?

[lookOutBelow]

Update:

You know when you import a Ninjascript and it asks you in not so many words-- "the following file already exists on your computer, do you want to replace?"

The first time, I said "no". And I got the different results posted above. I re-imported the strategies again. But this time I told it "yes", replace the files.

The results are different yet again. This time the strategy performed with a 100% success rate. While I like that number, it isn't what I'm looking for. I'm looking for consistency between versions.

Here are the .cs files that were included in my export of my strategy from v6.5

@DM.cs
@FisherTransform.cs
@MAX.cs
@MIN.cs
@RegressionChannel.cs

[NinjaTrader_RyanO]

Hi lookOutBelow,

Thank you for your post.

Can you please clarify the settings you are using for your back-tested strategy in both 6.5 and 7.

[lookOutBelow]

Hi lookOutBelow,

Thank you for your post.

Can you please clarify the settings you are using for your back-tested strategy in both 6.5 and 7.

What settings are you referring to? The strategy calls for input on a number of items in the parameter section. Identical parameters were used on both 7.0 and 6.5. Other than that, I'm using the dates 9/9/2010 to 9/13/2010 for the backtest, on a 1 minute chart.

Is that what you need to know?

Thanks

[NinjaTrader_RyanO]

Hi lookOutBelow,

Yes, I am looking for those parameters, but rather than letting me know they are the same can you please attach screenshots of each of the Strategy Analyzer parameters. I would like to know the specific parameters for Historical Fill Processing, Order Handling and Order Properties. I should only need a screenshot of version 7 since the properties are the same in 6.5.

To send a screenshot press Alt + PRINT SCREEN to take a screen shot of the selected window. Then go to Start--> Accessories--> Paint, and press CRTL + V to paste the image. Lastly, save as a jpeg file and send the file as an attachment.

[lookOutBelow]

here are the two screenshots

[NinjaTrader_Jason]

Hello lookOutBelow,

I noticed that NinjaTrader 6.5 uses session begin and end times of 12:00 AM and 12:00 AM. NinjaTrader 7 uses <Use instrument settings> as session template. Likely the session template used by the instrument in question does not use 12:00 AM and 12:00 AM as session begin and end times. Please select the 'Default 24/7' session template in NinjaTrader 7 and check if this will match the results.

You can check what session template currently is used at Tools-->Instrument Manager. Highlight the instrument in question and click Edit.

[lookOutBelow]

Thanks for the reply Jason.


Unfortunately, even after changing to Default 24/7, the results are off by quite a lot. Any other ideas?

[NinjaTrader_Jason]

What happens if you backtest the SampleMACrossOver strategy in both versions?

Do you make use of the same connectivity provider in both versions? If not, please do so and ensure both versions use the exact same historical data and check again.

[lookOutBelow]

What happens if you backtest the SampleMACrossOver strategy in both versions?

Do you make use of the same connectivity provider in both versions? If not, please do so and ensure both versions use the exact same historical data and check again.

I'm using Zenfire for everything. So yes, same provider. I'll give the Sample strat a try tonight after the market is closed.

Thanks.

[NinjaTrader_Jason]

Sounds good, please let me know the outcome.

[lookOutBelow]

I've finally gotten around to testing the sample MA crossover on both 6.5 and 7.0. The differences are huge. Here are the resulting screen shots.

I've tried changing the fill settings to both liberal and default. No matter the combination, they don't match up between versions. I've also tried changing the session times. The numbers did change, but they were still off by several thousand dollars and a ton of trades.

[NinjaTrader_Bertrand]

Please use the Default 24 / 7 session in NT7 to compare to your default NT 6.5 sessions, then the results should be much closer.

[lookOutBelow]

I tried that. Results are still off by literally hundreds of trades.

[lookOutBelow]

Anyone have any ideas? Putting in 24/7 doesn't do anything but make the results even further away. Here is a screen shot using default 24/7 template.

This is a pretty huge difference in outcome between 6.5 and 7.0