How do I import comma-delimited futures data (Pinnacle .csv) files?

[anachronist]

I've been a subscriber to Pinnacle Data Corporation's EOD data feed for futures. I use it with Tradestation.

Now that I'm trying to migrate things over to NinjaTrader, I can't figure out how to import my historical data! Pinnacle provides data in comma-delimited text files, formatted as:

YYYYMMDD,Open,High,Low,Close,Volume,OpenInterest

The file names are standard contract names; eg August 2008 Crude would be CL2008Q.csv.

It would be nice if NinjaTrader could "connect" to my data directory.

When I try to import a file, the log file says "Instrument is not supported by repository." And when I go to the Instrument Manager, it doesn't have "Future" available in the type drop-down! Just stock and index. What's going on?

I can't believe there isn't some way to import basic standard-format comma-delimited data. Am I missing something?

Is there a step-by-step tutorial somewhere that goes through importing futures contract data from a comma-delimited text file?

-Alex

[NinjaTrader_JoshP]

Please see this article: http://www.ninjatrader-support.com/H...V6/Import.html

[anachronist]

Please see this article: http://www.ninjatrader-support.com/H...V6/Import.html

Well, now.... Why isn't that in the PDF manual? I assumed it was the same as the online version, so I've been using the PDF version exclusively. I guess I'll have to bookmark the online version.

I was still having a problem with "Future" failing to appear as a choice in the Instrument Manager. That problem, however, cleared up when I disconnected from Yahoo.

I'm disturbed that the documentation insists the file name must follow certain conventions (do I have to rename hundreds of files?), and the data must be delimited by semicolons. Why not commas? That's been industry-standard for decades.

And what about reverse-adjusted contracts? That data doesn't have an expiration month, yet NinjaTrader's Instrument Manager wants to require it on the file name. The instrument name and the file name should just be the futures ticker, like CL for crude oil, GC for gold, etc.

I don't see how NinjaTrader can be compatible with Pinnacle data:

• Pinnacle uses a different file name convention. Instead of requiring "CL 09-08.txt" as the file name, why not allow something more in line with how contracts are really designated, such as "CL2008U.csv"?
• Pinnacle's data are comma-delimited, but as far as I can tell that doesn't matter; I suspect Ninja can already handle either way.
• NinjaTrader doesn't accept the correct fields! All end-of-day futures data I'm aware of comes with open interest! NinjaTrader wants just prices and volume, no open interest. It won't even ignore that "extra" field, but aborts with an error.

Some of these design decisions don't make sense to me. This the least flexible feature of the software I have encountered so far.

Thanks.
-Alex

[NinjaTrader_JoshP]

Alex,

That is the way it is in NT6.5. Improvements are enroute for NT7.

[anachronist]

That is the way it is in NT6.5. Improvements are enroute for NT7.

That's good to know. Should I post a suggestion for data import improvements on a feature request forum, or is it enough that you've already read this thread?

-Alex

[NinjaTrader_Dierk]

Thanks, we already have this on our list.

[anachronist]

Thanks, we already have this on our list.

I know this is an old thread, but I want to follow up.

I'm glad that improving the data file handling is already on your to-do list.

I want to ensure that your list includes the following necessary features:

• Capability to import comma-delimited data (like Excel .csv files), and optionally define the delimiter (comma, space, tab, whatever).
• Capability to import any stream of data that isn't a price bar. Examples include advancing issues, unchanged index volume, daily interest rates, closing spot market commodity prices, put/call ratios, option open interest, anything I want.
• Capability to define the format of the input file (e.g. TradeStation offers many choices, such as Date,Close or Date,Time,Close or Date,Open,High,Low,Close,Volume,OpenInt; and you can specify any date format such as YYYYMMDD or MM/DD/YYYY).
• No weird file naming conventions, and no required file name extension like .txt (we know it's a text file, it can have any extension like .csv, .prn, or none). Futures data should not be required to have a month and year in the file name, especially if the data is reverse-adjusted continuous contracts, or closing spot prices. Just requiring the commodity symbol to lead the file name should be sufficient. For single contract futures data, a numeric month in a file name is silly; for example, the exchange futures symbol for December 2008 Gold is GCZ2008 or GC2008Z or GC08Z, [i]not[i] GC-2008-12 or whatever.

Anyway, I hope those things are on your list. I note that open interest data is currently not supported in NinjaTrader -- but allowing arbitrary streams of input data would solve this problem as well as many others.

Thanks.
Alex

[NinjaTrader_Dierk]

Thanks for your suggestions. We'll add them to the list of future considerations.

[Farmer George]

I know this is an old thread, but I want to follow up.

I'm glad that improving the data file handling is already on your to-do list.

I want to ensure that your list includes the following necessary features:

• Capability to import comma-delimited data (like Excel .csv files), and optionally define the delimiter (comma, space, tab, whatever).
• Capability to import any stream of data that isn't a price bar. Examples include advancing issues, unchanged index volume, daily interest rates, closing spot market commodity prices, put/call ratios, option open interest, anything I want.
• Capability to define the format of the input file (e.g. TradeStation offers many choices, such as Date,Close or Date,Time,Close or Date,Open,High,Low,Close,Volume,OpenInt; and you can specify any date format such as YYYYMMDD or MM/DD/YYYY).
• No weird file naming conventions, and no required file name extension like .txt (we know it's a text file, it can have any extension like .csv, .prn, or none). Futures data should not be required to have a month and year in the file name, especially if the data is reverse-adjusted continuous contracts, or closing spot prices. Just requiring the commodity symbol to lead the file name should be sufficient. For single contract futures data, a numeric month in a file name is silly; for example, the exchange futures symbol for December 2008 Gold is GCZ2008 or GC2008Z or GC08Z, [i]not[i] GC-2008-12 or whatever.

Anyway, I hope those things are on your list. I note that open interest data is currently not supported in NinjaTrader -- but allowing arbitrary streams of input data would solve this problem as well as many others.

Thanks.
Alex

Regarding the above post, will this be possible in NT 8?
"Capability to import any stream of data that isn't a price bar. Examples include advancing issues, unchanged index volume, daily interest rates, closing spot market commodity prices, put/call ratios, option open interest, anything I want."?

Thanks

[NinjaTrader_DrewO]

Hello,

Thanks for your post.

I was unable to find this in the list of Major Enhancements of NinjaTrader 8, found publicly available at the following link:

http://www.ninjatrader.com/support/forum/showthread.php?t=67238

However, I am happy to add this as a feature request for consideration as a possible addition in future editions.