Go Back   NinjaTrader Support Forum > NinjaScript Development Support > General Programming

General Programming General NinjaScript programming questions.

Reply
 
Thread Tools Display Modes
Old 05-16-2011, 01:40 PM   #1
8DTK8
Senior Member
 
Join Date: Jun 2008
Posts: 168
Thanks: 0
Thanked 0 times in 0 posts
Default Corrupt data files

I haven't done any backtesting since the upgrade to NT7.
After a few days of getting back into it for the first time in a while, I noticed that large chunks of my data are missing. I never checked to see if the database migrated properly when I went from 6.5 to 7.

The last backtests I ran were working great with both daily and minute data since 1995. Now There are random periods as long as a month missing and it will only go back to 2007.

Any ideas what happened or more imoportantly how I can fix this?

Alternatively, if someone can inform me as to where I can get day and minute data for (ES, NQ, GC, ZB, ZN, and most of the currency) futures, I would greatly appreciate it.

Thanks in advance.
8DTK8 is offline  
Reply With Quote
Old 05-16-2011, 02:07 PM   #2
NinjaTrader_RyanM
NinjaTrader Customer Service
 
NinjaTrader_RyanM's Avatar
 
Join Date: Sep 2009
Location: Denver, CO
Posts: 8,128
Thanks: 249
Thanked 427 times in 423 posts
Default

Hello 8DTK8,

Unfortunately there's no way to know what happened to your data. Historical data is provided by our supported brokers/data providers or can be imported if you already have txt files available You will get the most historical data with a data only provider like Kinetick, eSignal, or BarChart.

A complete list of our providers and the level of data supported for each is available here:
http://www.ninjatrader.com/support/h...rical_data.htm

If you have text files available, you can import if the're formatted correctly for NinjaTrader:
http://www.ninjatrader.com/support/h.../importing.htm

To prevent running into situations where you can potentially lose important data, we advise to run regular backups via our inbuild backup feature in NinjaTrader. You can include historical chart data in this backup. http://www.ninjatrader.com/support/h...up_archive.htm
NinjaTrader_RyanM is offline  
Reply With Quote
Old 05-16-2011, 05:20 PM   #3
8DTK8
Senior Member
 
Join Date: Jun 2008
Posts: 168
Thanks: 0
Thanked 0 times in 0 posts
Default

Thanks RyanM.

I just spoke with someone at eSig that told me that their daily data goes back 25 years, but their min. data only goes back to 2007.

Realizing tha my laziness will come shining though with my next request, does anyone know who has the most comprehensive (goes the furthest back) daily and minute data for futures out of the providers listed? http://www.ninjatrader.com/support/h...rical_data.htm

Thanks in advance.
8DTK8 is offline  
Reply With Quote
Old 05-19-2011, 02:41 PM   #4
NinjaTrader_Tim
NinjaTrader Customer Service
 
NinjaTrader_Tim's Avatar
 
Join Date: Nov 2009
Location: Denver, CO
Posts: 1,424
Thanks: 0
Thanked 3 times in 3 posts
Default

Hi 8DTK8,

Kinetick offers extensive historical data. Including at least 120 days of historical tick data, two years of minute data, and ten years of daily data. Much more for popular instruments, for example the ES minutes data goes back to 2005.
NinjaTrader_Tim is offline  
Reply With Quote
Old 05-19-2011, 04:22 PM   #5
Camdo
Senior Member
 
Join Date: Aug 2009
Location: Massachusetts, USA
Posts: 207
Thanks: 3
Thanked 1 time in 1 post
Default

My data file was corrupted in similiar manner. The cause was traced to reloading data from a chart with a range beyond what the data provider provides (ie a three year chart was reloaded with right click, but the data provided only supplies 6 months of minute data). The solution is to reduce the range of the chart, then reload, then expand the range back to what it was. My provider is MB Trading.

This is a bug in NT which should be addressed.
Last edited by Camdo; 05-19-2011 at 04:25 PM.
Camdo is offline  
Reply With Quote
Old 05-19-2011, 04:47 PM   #6
NinjaTrader_Josh
NinjaTrader Product Manager
 
NinjaTrader_Josh's Avatar
 
Join Date: May 2007
Location: Denver, CO
Posts: 17,460
Thanks: 1
Thanked 161 times in 82 posts
Default

Camdo,

This is not a bug, but unfortunately is indeed a limitation to the way NinjaTrader requests data. When you explicitly try to do a download of historical data based on a date range, NT achieves this in several steps.

1. Removes old data from the range you chose
2. Downloads what it can download from the provider
3. Repopulates the range with what it was able to download

If you request a range longer than what the data provider offers it would not repopulate those sections.

There is no logic in place to allow us to detect what the boundary is for what a certain data provider does or doesn't offer.
NinjaTrader_Josh is offline  
Reply With Quote
Old 05-19-2011, 05:24 PM   #7
Camdo
Senior Member
 
Join Date: Aug 2009
Location: Massachusetts, USA
Posts: 207
Thanks: 3
Thanked 1 time in 1 post
Default

AhHA !
In that case I would suggest a warning message pop up to remind the user of the range issue and lost data consequenses you mentioned above, and allow the user a chance to opt out before the reload is executed. It is too easy to forget chart range could corrupt a database.
Camdo is offline  
Reply With Quote
Old 05-20-2011, 12:06 PM   #8
NinjaTrader_Josh
NinjaTrader Product Manager
 
NinjaTrader_Josh's Avatar
 
Join Date: May 2007
Location: Denver, CO
Posts: 17,460
Thanks: 1
Thanked 161 times in 82 posts
Default

Camdo,

Thank you for the suggestion. This limitation is already on our list for things we would like to remove in a future version of NT.
NinjaTrader_Josh is offline  
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem with corrupt chart data? ohowie Version 7 Beta General Questions & Bug Reports 12 05-17-2010 08:36 AM
Repairing corrupt NT7 data? higler Version 7 Beta General Questions & Bug Reports 5 04-09-2010 11:08 AM
corrupt data deanldavis475 Miscellaneous Support 1 02-26-2010 05:55 AM
NT 7 corrupt MetaStock data kuwait Version 7 Beta General Questions & Bug Reports 0 12-31-2009 06:46 AM
Has anyone compiled Gain's weekly tick data files into larger files? bluelou Charting 0 06-18-2009 10:07 PM


All times are GMT -6. The time now is 06:21 PM.

FULL RISK DISCLOSURE: Futures trading contains substantial risk and is not for every investor. An investor could potentially lose all or more than the initial investment. Risk capital is money that can be lost without jeopardizing ones financial security or lifestyle. Only risk capital should be used for trading and only those with sufficient risk capital should consider trading. Past performance is not necessarily indicative of future results. View Full Risk Disclosure.