Backtest data is different from live data

[JamieHendrix]

I have four separate automated trading systems (based on similar algorithms). Yesterday I ran all four on the same account (two on a 500 tick chart and two on a 2500 tick chart). 1 of the systems acted properly but the others had fills that were not even close to being traded at according to my NinjaTrader charts. I have the NinjaTrader standard lease with Zen Fire as my data provider. My systems work well but need accurate charts (obviously). I had 4 point stops and they were hit almost immediately while there were not even close to any 4 point swings that morning. Where should I look to make sure this does not happen again. Is ZenFire unreliable? I don't mind taking a loss when I am wrong, but according to the chart, I should not have been even close to being stopped out.

Thank you, Jamie Hendrix

[NinjaTrader_Matthew]

Hello Jamie,

If these were live orders, NinjaTrader has no control over how your orders are order are filled. You will need to contact your broker for clarification on any fills you received.

[JamieHendrix]

Matthew. I understand this. I am just trying to figure out first if my charts were built inaccurately and if so then how this can happen. I am not trying to point fingers anywhere, just trying to identify what the problem could be and how to avoid it in the future. If my data is slow or inaccurate, I want to find out how to get the best data so my charts are built correctly in real time. My systems are only as good as the accuracy of the charts that are constructed in real time.

Thank you. Jamie Hendrix

[NinjaTrader_Matthew]

Hello,

I will need to compare the prices you were filled to the actual market data.

Please send me an email to support[at]ninjatrader[dot]com and include your log and trace files from the date this occurred, Attn: Matthew in the Subject line and a reference to this thread.

Please also include the time and price you were filled, and the order ID if that is available.

You will find the log file in the Documents > NinjaTrader 7 > Log folder.

• The log file will be named ‘log.20111130.txt’

You will find the trace file in the Documents > NinjaTrader 7 > Trace folder.

• The trace file will be named ‘trace.20111130.txt’

Please attach both of these files to your next response. If these files are over 4MB, please let me know and I can respond with alternative instructions.

[JamieHendrix]

The log file is 25 MB. I can not attach it to an email.

Thank you for your help Matthew. I want to be clear that I am not saying that NinjaTrader built the chart incorrectly, Maybe the data was incomplete that came in. I just want to make sure I have the best setup as far as data providers etc. if Zen Fire will not produce correct data for the NinjaTrader program to build the charts with, I will go with Kinetic or CQG or whoever will give the best results. I have traded full time for about 5 years now and have not had these problems in the past.... just trying to problem solve right now.


Thank you

[NinjaTrader_Matthew]

I completely understand - your diagnostic files should give us an idea of where your data was vs where the market should have been trading.

If it is 25 MB, please try going to Help--> Mail to support and checking 'log and trace files'.


Please also include 'Attn: Matthew' and a refernece to this thread, as well as the order details in question.

[JamieHendrix]

Matthew. I have tried sending you the trace files but NinjaTrader just keeps locking up when I try. Assuming that the prices were correct but my chart was not, What are some of the possible resons for this happening?

[NinjaTrader_Matthew]

The most likley reason would be a discrepancies between your local PC clock and the data feed clock.

Please compare your local PC clock time vs the Zen-Fire time stamps by going to File--> New--> Time and Sales.


From the Time and Sales window, please right click--> select "Instrument" and load any instrument.


You will now see the Time and Sales window populate with trades which are time stamped from the Zen-Fire servers.


You will want to open your PC clock and compare this time down to the second. Is your PC ahead, behind or equal to the Zen-Fire time stamps in the time and sales? Even a discrepancy of 5 seconds or less can contribute to issues, especially in high volume periods.


More information about using the Time and Sales window can be found below:
http://www.ninjatrader.com/support/helpGuides/nt7/index.html?time__ sales.htm


Please let me know if you have additional questions.

[JamieHendrix]

Would the clock / time issue affect tick charts? I understand possibly throwing off time based charts. I am more concerned that my data is not coming in fast enough or accurately.

[NinjaTrader_Matthew]

Jamie,

Actually tick charts are going to be more sensitive to this.

Let's say you have a tick (tick "A") with a time stamp of 10:31:00 AM which gets packaged into the 10:32:00 AM bar and happens to be the high of that bar. An hour later, you reload historical data from your historical data provider into NinjaTrader. This process will overwrite the existing data. The 10:32:00 AM bar now looks different since the high made by TICK "A" is now part of the prior bar, 10:31:00 AM. How is this possible?

•Your PC clock could have been off so the time stamp is delayed

•Your internet may have been lagging so the tick came in slightly delayed and therefore the time stamp is delayed

•Due to standard latency, even 50ms delay (which is normal) could be the difference between a 10:30:59 and 10:31:00 time stamp

•There is no way of knowing how the historical data provider packages their bars

[JamieHendrix]

Great Matthew thanks, that makes a lot of sense. So basically I should make sure my computer time is in sync and make sure my internet is as fast and reliable as possible ( would the actual data provider make a difference? (ie: Kinetick or CQG?) . I think I may have been running wireless that day (Although I never do that) haha I have traded out of Starbucks before ( but never automated)

Thanks buddy. You guys at NinjaTrader support are always very helpful.

[JamieHendrix]

I have all the bugs worked out and it is working very well. What is the best way to export these (4) systems to another computer?

[JamieHendrix]

Error Message.

While running my automated systems this morning, I received an error message saying that the strategy generated an "Overfill" and turned itself off (ceased to run). Any ideas what an "Overfill" is and how do I avoid this in the future?

Thank You, Jamie Hendrix

[NinjaTrader_Matthew]

This can occur when you have to orders to fill occurring at the same time. You will need to review your code to ensure there is not a set of statements in which both could be true at the same time.

Please review our Help Guide entry on this topic

Over-Fills
Over-Fills is a serious issue that can occur when using complex entry conditions that bracket the market in both directions end up with both entries being filled instead of one being cancelled. Over-Fills can also occur when you place a trade quickly hoping to close a position while a prior order to close the same position already had an in-flight execution. The exact scenarios in which an over-fill can occur is highly dependent on the specific strategy programming. By default, NinjaTrader will protect against over-fills even though you are using the Unmanaged approach by halting the strategy, but should you decide to custom program your own over-fill handling it is up to you to either prevent over-fills from being a possibility in your code or by introducing logic to address over-fills should one occur.

[JamieHendrix]

Order management

Thank you Matthew. I added a statement to my script to only allow a new position to be initiated when the current position is "flat" but I still got the overfill error.
if Position.MarketPosition == MarketPosition.flat then.... execute

I have upgraded my internet connection (just now) to a 50Mbs speed which is very very fast.... my computers are brand new and they are cabled to the modem (not wireless), Is there anything you can suggest for me to do to maximize the speed and correctness of my setup to run these scripts without these errors? they never came up before I started using these new computers and my scripts have not changed at all.

Thank You, Jamie Hendrix