strategy performance data problems

[edstaffin]

Hi,

I'm finding myself extremely frustrated trying to pull any meaningful strategy performance results. Whether it be from a trade performance report, or even directly from the individual strategy performance report itself!

Over the past several months I've developed several strategies. I've spent days/weeks back testing, optimizing, and tweaking code to finally having these ready to test out in real time. For the past several weeks I have been running my strats on the sim account to attempt to prove out that my real time results and expectations compare to those that I have optimized and back tested too. In order to determine this, I need to be able to obtain some type of meaningful real time results and reporting. NT purports to provide this in several forms, but honestly this seems to be woefully (and frustratingly) lacking.

Issue 1:
When I'm running my strats, I would periodically wander over to the strategies tab in the control center to get a quick take on what's happening. The only problem is that the realized column ALWAYS shows 0. In fact, I have never seen anything other than 0 except for one occasion and I have been unable to reproduce that.

Issue 2:
So, I tried a different approach. I right click on a strat and choose strategy performance, I get only the last day's trading if I'm lucky. This is in spite of the fact that I have had nt running for a week and never shut down and had trades each day. I would have expected to get results for as far back as there is data for. Or, at least, for a specified time frame (the current date doesn't cut it).

Issue 3:
Next up, the New/Trade Performance form/report. It's a train wreck. The only way I can get data to show half the time is to choose "No template" along with my strat. And when I choose that and show trades, I get data for my strat, but I also get data that has nothing listed in the strategy column. This is strange as the ONLY trades taken were via a strat. I suspect these trades are from my strat, but can't confirm or deny this. This form, as it sits, is unreliable at best and just plain wrong at worst.

Somehow I need to test and prove out these strategies that I've spent months developing, but due to the issues discussed with reporting, I seem to be blocked at every turn by NT. How am I supposed to get meaningful data from NT? At this point I'd be better off going back to Excel spreadsheets to track my strat performance. At least then I could make sense of the data.

I could sure use some guidance.
Thanks ... Ed

[sledge]

Finally someone joins my world

[NinjaTrader_Matthew]

Hi Ed,

Sorry to hear you're having so much trouble. I'd be happy to offer some guidance here but as some of this behavior is unreproducible we'll need more information to look into.

Perhaps if you have time you can send an email to platformsupport[at]ninjatrader[dot]com with your phone number and a good time window you can be reached, and one of our support specialists can follow up with you.

Regarding item #1:

I've spent some time testing several different strategies on two different machines and the Realized PnL always shows up correctly for me.

There were some items in previous release related to that, so just confirm you are on B6, otherwise we'll need more details on how to reproduce (e.g., specific steps such as if you shut down and edit the strategy after its already running did it stop working, etc).

Item #2,

You said you have not shut down. Do you mean you have not shut down NinjaTrader, or have not shut down the strategy? And is that 'real-time' performance you're checking? Please note that if you disable the strategy and start it back up, your real-time results are reset, so that may account for why you only see the current day trades.

For Item #3, I am also unable to reproduce this behavior. If I select a template for a strategy, it shows trades only from that strategy. The only time the results are empty is if i unselect all templates. Are you sure you do not have other filters like "accounts" or "Instruments" unchecked which may be filtering out some results? Our support team can also help remotely to identify why that may not be working for you.

[edstaffin]

I sent an email w/my phone # to platform support. If they call, I will be able to demo the issues easily. Once I can demo the issues, it should be easy for you to see and repro.
Thanks ... Ed

[edstaffin]

I did as you suggested and sent an email to platform support asking them to call me. In response I got a request to submit sample scripts and detailed instructions ... blah, blah, blah. You guys asked me if I could demo this to you. I can. I'm available. I can show you the problem in 10 minutes or less. I'll wait to hear from you.
Thanks ... Ed

[NinjaTrader_Matthew]

I did as you suggested and sent an email to platform support asking them to call me. In response I got a request to submit sample scripts and detailed instructions ... blah, blah, blah. You guys asked me if I could demo this to you. I can. I'm available. I can show you the problem in 10 minutes or less. I'll wait to hear from you.
Thanks ... Ed

I trust you are experiencing issues, but unless we can reproduce, demoing the problem does not leave us with much to work with. If you have scripts and steps to reproduce, our support team can look on their end and if more is needed we can always log in to reduce any variables.

[edstaffin]

Hi,
Thanks for the quick response. I have some issues with what you are asking of me.
You said, and I quote:

Perhaps if you have time you can send an email to platformsupport[at]ninjatrader[dot]com with your phone number and a good time window you can be reached, and one of our support specialists can follow up with you.

I did and was happy to do so. Now you don't want to call. As I've previously stated, It'll only take 10 minutes or so.

1. I can't give you my sources as they are proprietary.
2. I don't see how my scripts would have any effect on what does or does not display in the realized column of the strategies tab. Presumably, that info is coming from accounts and/or trade executions.
3. What shows by right-clicking in the strategy performance window I would again presume is not dependent on my script (other than the fact that the script makes managed trades).
4. Same thing with the Trade Performance filters on the report. How would my code have an effect on your calcs?

I am happy to work with you as it will make the end product better for both of us.
Thanks ... Ed

[NinjaTrader_Matthew]

The tech you are working with is willing to call you, he was simply asking if you had any steps to reproduce as we work through various tasks throughout the day. Please continue to work with him over email to schedule a time that works best for both of you, as our NinjaTrader 7 queue takes priority which is why he was not able to call you right away. Realize it might take you a few minutes to setup but we have other responsibilities that take priority, especially on a Monday morning.

[edstaffin]

I got a call from Jesse and, as promised, was able to demo the issues to him. He asked that I run the sample strat for a couple of days to see if I get the same issues. I told him I would do that. However, if I don't get the same issues, I would like to get a hint from you guys as to what would cause NT to show 0 in the realized column in response to a strat script that I wrote. If you can give me a hint maybe I can expedite the process. I suspect, but do not know, that whatever it is that is causing a 0 realized pnl, is also causing the other problems previously mentioned in my other posts to this thread.
Thanks ... Ed

[sledge]

Hi,



Issue 3:
Next up, the New/Trade Performance form/report. It's a train wreck. The only way I can get data to show half the time is to choose "No template" along with my strat. And when I choose that and show trades, I get data for my strat, but I also get data that has nothing listed in the strategy column. This is strange as the ONLY trades taken were via a strat. I suspect these trades are from my strat, but can't confirm or deny this. This form, as it sits, is unreliable at best and just plain wrong at worst.



I could sure use some guidance.
Thanks ... Ed

Generate for no template, and look at the executions or trades report. See if you can get any idea about them, if the have a name or not.

[edstaffin]

More info

Hi,
I tried this last night and it should give you more to go on.
1. Add the sample MA Cross strat to the strat tab in the control center.
2. Set the data series to be a Range 5 and the instrument to be CL
3. Let it run overnight.
4. Presumably it took trades.
5. Realized column in strat tab of control center still shows 0.
6. Trade performance report filtered on the sim account, CL and the strat shows 0 trades, yet checking "no template" shows trades for the strat.
Let me know if you need anything more. This is pretty straightforward stuff.
Thanks ... Ed

[NinjaTrader_Matthew]

Hi ed,

Thanks for this info. I agree that it is straightforward, however there must be a variable missing because we're still unable to reproduce. I ran the same SampleMACrossover, but changed some settings to ensure that some trades were taken more rapidly rather than waiting for the Range 5 to build.

Here is a video of my results:

http://screencast.com/t/zoVyWhXhiI

As in the video, see if you can reproduce the Realized PnL not updating using the following settings;

• CL 01-16
• 5 second chart
• SampleMACrossover
• Fast = 2
• Slow = 7
• Calculate = OnEachTick

This should generate several trades in a short time. Please let me know if the realized PnL does not update.

Regrading the Trade Performance window, I am also see this matches and I can get it working with just a single "Template" for the SampleMACrossover strategy. I did point out in the video that one of the trades was exited manually, which is why it was not included in the performance report when just that strategy template is selected. Perhaps if you are mixing manual trades and automated trades, it is obscuring your results.

[edstaffin]

Using your settings I am getting a non-zero value!
Progress! Not sure where to go from here. Should you give me a call and we can fiddle with it until it breaks?
Thanks ... Ed

[NinjaTrader_Matthew]

Hi Ed,

I would not personally be available for live troubleshooting, but if you're having trouble isolating that, you can always setup a call with a technician.

I wanted to give you an update that we identified some issues with the the way strategies were being reported as a result of this thread. The Trade Performance window will be updated in the next beta release - NTEIGHT-9147

The ability to filter "templates" by NS strategies was reverted as there were some inconsistencies with the way that trade pairing logic works when you mix manual trades and NS strategy trades. Only ATM Strategies will be possible to filter in the next release.

This should help with issue you reported with some trades not showing up depending on your filter settings. You will need to use the "Strategies" tab performance report to filter executions based on only the strategy.

We've added part of that request to SFT-943 - Add Trade Performance filter to work with "NS Strategies" templates

We have not made any changes to the Realized PnL column of the strategies tab as we have not yet been able to identify any issues with that feature.

[sledge]

Hi Ed,

I would not personally be available for live troubleshooting, but if you're having trouble isolating that, you can always setup a call with a technician.

I wanted to give you an update that we identified some issues with the the way strategies were being reported as a result of this thread. The Trade Performance window will be updated in the next beta release - NTEIGHT-9147

The ability to filter "templates" by NS strategies was reverted as there were some inconsistencies with the way that trade pairing logic works when you mix manual trades and NS strategy trades. Only ATM Strategies will be possible to filter in the next release.

This should help with issue you reported with some trades not showing up depending on your filter settings. You will need to use the "Strategies" tab performance report to filter executions based on only the strategy.

We've added part of that request to SFT-943 - Add Trade Performance filter to work with "NS Strategies" templates

We have not made any changes to the Realized PnL column of the strategies tab as we have not yet been able to identify any issues with that feature.

Nt_Matthew:

Here's another way to get results into the No Template without using chart trader or dom, etc.

I ran a full replay, REMOVED all 32 strategies in Strategies tab(4 strategies on 8 ES contracts), then performed a disconnect from Playback-MR.

I then ran a Trade Performance report - and all those strategy results are now in the "No Template" with a strategy name.

This is on a machine with no chart trader, just complete strategy testing from Playback-MR.

These are definitely the results from my last run all totaled together.


p.s. they are still there are restarting NT8.