Historical Bid/Ask Data

[ofir_y]

Dear support team,

I've started to use indicators I purchased at Emini-Watch. One of this indicators is based on volume, and I've found that the displayed history of this indicator is absolutely wrong, although its real time display looks fine, plus other Emini-Watch indicators which are not volume based, work perfectly.

I've checked this out with the creator of the indicators, and it seems that the problem is that this volume based indicator requires bid/ask data, and unfortunately my data provider, DTN IQFEED does not provide historical bid/ask data. Indeed, according to the data providers table, http://www.ninjatrader.com/support/helpGuides/nt7/?historical_data.htm (under "Understanding the data provided by your connectivity provider", IQFEED does not support "Historical Bid/Ask Tick Data". However, although according to this table Kinetick do support "Historical Bid/Ask Tick Data", when I checked this out with Kinetick's customer support, they told me that with their data service I will only have real time bid/ask data for my volume based indicator, but I will not have historical bid/ask data.

So I'm a little bit confused now. How come Kinetick's data service does not include the historical bid/ask data I need although the table says it does?

Any ideas what can I do to solve this issue (quite critical for me)?
Or can you tell which other data service might solve this problem?

Thanks in advance,
Ofir

[NinjaTrader_Brett]

Hello,

Thanks for the forum post.

Kinetick does support historical tick bid/ask tick data. Therefor there may have been a miscommunication.in your note with Kinetick support as I just tested and I can get historical bid and ask tick data. I cannot get historical bid/ask minute or daily data however only tick.

-Brett

[ofir_y]

Dear Brett,

Since according to NinjaTrader data provider table, Kinetick does support historical bid/ask tick data, I thought there might be some kind of miscommunication in my note with Kinetick support, but they insisted that with their data service I will not have historical bid/ask data for my volume based indicator.

Here are the relevant quotes from their answers:
"With NinjaTrader you would be unable to use the historical bid/ask data for populating the indicator, but it is possible to generate charts based on the bid and ask chart when modifying the data series properties"
"With NinjaTrader it is not possible to use real-time bid/ask data to backfill an indicator, and with indicators that rely on bid/ask data they would only populate based on the real-time data."
"For your volume based indicators they will calculate in real-time only. For historical data your alternatives are to load bid and ask based charts, which would not populate volume based indicators with historical data."

I'll be happy to be proofed wrong, but seems they are quite sure about that.
Are you sure this historical bid/ask data will enable populating my volume based indicator? Is there some volume based indicator you check this out?

Thanks in advance,
Ofir

[NinjaTrader_Brett]

Hello,

Thanks for the clarification.

I will clarify everything a step further so we are all on the same page.

Kinetick does have historical tick data, that is not in question. However what Kinetick is referring to is that most likely the indicator is setup to run on realtime data only. Most volume based indicators that run off bid/ask are programmed to run live only.

You have to have specific historical processing coded into the indicator to allow for this. However what you will find is that most indicators are not coded for this as there is a limitation of NinjaTrader stored bid and ask data. We only have 1 second granularity on our data in our database. This 1 second granularity inst enough granularity to get 100% accurate results on bid/ask processing when it comes to if a trade occurred on a bid or an ask price.

In our next major version of NinjaTrader development is looking at if it is possible to remove this limitation from our platform. I do not have a release date at this time unfortunately.

Therefor the answer is two fold, yes Kinetick does support historical bid and ask tick data. However most likely your custom indicator is not coded to use it. You would need to check with the indicator developer to see if they have added the multi series coding needed to make this work.

-Brett

[ofir_y]

Dear Brett,
Thanks for the prompt, detailed and proffesional answer.

This support forum is awesome!!

Thanks,
Ofir

[alex.nt]

1) Could you please provide a sample for an indicator that uses historical Bid/Ask? Basically, what you saying is that each series of Bid/Ask/Last (previously added in Initialize) would generate its own OnBarUpdate event?

2) Assume I have 3 txt-files: MSFT.Ask.txt, MSFT.Bid.txt, and MSFT.Last.txt, all tick data. I have to import each of those files in order to run the aforementioned indicator, correct?

Thank you!

[NinjaTrader_Lance]

Hello,

In order to read the historical data values for bid/ask you would have to make a multi series script which added the correct marketDataType
See the 3rd overload here: http://www.ninjatrader.com/support/h....html?add3.htm

For more on multi series scripts: http://www.ninjatrader.com/support/h...nstruments.htm

2) Assume I have 3 txt-files: MSFT.Ask.txt, MSFT.Bid.txt, and MSFT.Last.txt, all tick data. I have to import each of those files in order to run the aforementioned indicator, correct?

Correct, you would need to have the historical data on your system to use it. Some of our data providers do provide historical bid/ask data: http://www.ninjatrader.com/support/h...rical_data.htm

Let me know if I can further assist.

[pbmi66]

Hi,

I am also looking into this topic - Historical Bid/Ask/Last data & their usage in custom indicators.

It seems to me that although it is possible to download historical bid/ask/last from a DTN / Kinetick data feed, and access the historical data in a multi-bar script, the limitations on the NinjaTrader side make it impossible to look at historical tick data in conjunction with historical bid/ask data.

- in particular, the Bid/Ask Data are stored as a separate dataseries for each, and are only time-stamped to a 1-second time interval

- as a result, there is no way to synchronise either bid or ask data to tick data

Therefore, the only usage I can see for this historical data is to populate an indicator that requires either Tick or Bid or Ask data - but not a combination thereof.

Is my understanding correct?

Regards
Peter

[NinjaTrader_Lance]

All the events and prices of the Bid/Ask will be valid but yes as you pointed out because of the 1 second granularity they may not be in the correct order.

Consider taking a look at a third party indicator such as for ways in which users have worked around this: https://www.bigmiketrading.com/wiki/trading-wiki/Gomi

Let me know if I can further assist.

[pbmi66]

It's not only that the order may be out (this is as far as I understand a problem for all data providers).

The real issue is that Ninja stores bid/ask/tick data with timestamps summarised to 1 second.

So for heavily traded markets/times of the day, where you have multiple bid/ask/trade events triggered within 1 second, it is not possible to match the data.

This is a problem when looking at backfilling charts & testing - especially for day traders who are reading the tape/order flow...

Is there a plan to fix this?

[NinjaTrader_Lance]

While I cannot confirm how or if we will be changing this I can tell you this is a popular feature request and I will add it to the list of feature requests for NinjaTrader 8. The more feature requests we get for a topic the more likely it is to make it into a future release.

For now I would suggest looking at the GOMI tools as they circumvent this by storing this data into custom files.

Let me know if I can further assist.

[broadblade]

I have a question related to this as well. Many other platforms do not have the function(s) that I am looking for specifically - #1 Look for stocks that have at least 5 - 150 trades so far and #2 Look for stocks where the trade increase is greater than 30% relative to the 20-day average and #3 Look for stocks with less than a 30,000,000 float. I am assuming this would require tick data, and at this time I am only aware of 1 company that has this functionally but I am not willing to pay the ultra-high price they are asking. Any help on this matter would be greatly appreciated.

Thank you
David Smith

[NinjaTrader_Lance]

#1: Possible by placing the VOL indicator (based on daily bars) onto the Market Analyzer and setting a filter condition to only show stocks with > 5

#2: Would be possible through a custom indicator and placing that indicator onto the Market Analyzer. Not sure if one has already been created but you might want to consider browsing our forum or https://www.bigmiketrading.com/
If you can't find one we have a list of developers here who could assist in creating this: http://www.ninjatrader.com/Ecosystem/NonBroker.php#81

#3: This is partially dependant on the data provider. I don't know of any way to get the float but I know you can get SharesOutstanding. For a list of all the fundamental data types you can get: http://www.ninjatrader.com/support/h...aeventargs.htm
Alternatively a developer could pull the float value from a website if this value was posted there

For instructions/videos on using the MA: http://www.ninjatrader.com/support/h...alyzer_win.htm

For a webinar recording: https://www.youtube.com/watch?v=RQSp9NODAEk

Let me know how I can further assist.

[pbmi66]

I'm not sure that these GOMI indicators are a real workaround.

It seems to me that GOMI justs records real-time data which can then be used at a later date.

I am trialling the Kinetick(IQFeed) for the reason that I would supposedly get historical tick data (as advertised) to backfill charts/indicators.

But it seems that this advertising is misleading, at best, due to the problems on the Ninjatrader side mentioned previously.

Or is there another solution not mentioned that allows NT users to actually use this historical tick data feed?

[NinjaTrader_Lance]

You will have access to historical tick data with Kinetick. Kinetick will provide every tick sent from the exchange. As you've noticed the historical bid/ask data can be out of order due to it only having 1 second granularity.

We plan to increase the granularity in the next major release of NinjaTrader.

Let me know if I can further assist.