BidAsk Historical...

[rt-trader]

I imagine there will be a considerable number of 6.5 users waiting for this Historical Bid/Ask capability to build historically what is now possible live via OnMarketData.

From these early comments it seems this may not be straightforward. Is it possible for some sample code on how this might be accomplished be posted?

Or better yet - an enhancement to (or derivative of) OnMarketData which will allow this to be more easily?

Thanks

PS I dont have NT7 yet so can not experiment myself.

[NinjaTrader_Ray]

I imagine there will be a considerable number of 6.5 users waiting for this Historical Bid/Ask capability to build historically what is now possible live via OnMarketData.

From these early comments it seems this may not be straightforward. Is it possible for some sample code on how this might be accomplished be posted?

Or better yet - an enhancement to (or derivative of) OnMarketData which will allow this to be more easily?

Thanks

PS I dont have NT7 yet so can not experiment myself.

Unfortunately there is no reference sample forthcoming anytime soon. This is strictly based on bandwidth and priority of other items however, this is one of those things on the list.

[rt-trader]

OK - thankyou for the reply Ray.

[funk101]

Ok, how can one accomplish "retrieving historical bid ask"? I asked before, is stream reading/writing from a serialized file the best way to achieve this? Is this a painfully slow approach? When a user refreshes the chart, the data get's loaded back in? What are the options for trying to implement this?

[darthtrader]

darthtrader,

It is possible to have many ticks with the same time stamp.

true, i soppose i was thinking about a thinner market earlier...
I was just kind of throwing that out there to hopefully get the ball rolling.
I can totally understand the reason for not messing with onmarketdata()...
breaking alot of other uses of it wouldn't make sense just to backfill this kind of stuff...

It just seems to me we do have the tools now to do this, just need to figure out how...I mean don't we basically have something like this if you were to visualize the code gomi posted as far as the arrays:
A - ask
B - bid
L - last

a a a aaa aa a a a aa a aaaa a
L L L L L L L L L L
bb b b bbb b bbbb b b bbbbb b

The problem is the index of the arrays will not line up but we do have the data we need to figure this out.
I could see how streaming from a file could work with onmarketdata but i would think we should be able to do this with just on bar update.

[NinjaTrader_Dierk]

Guys,

Let me outline the opportunities and limitations of the new historical bid/ask feature.
- you could have multiple bid/ask/last series on your indicator/strategy
- the historical bid/ask series would hold ALL bid/ask events sent out by the exchange. Thus, it would NOT be the bid/ask at the time a trade went off
- on the historical part they would be replayed by OnBarUpdate
- OnMarketData only would be triggered realtime
- on historical replay the series are synced by their timestamp
- timestamps in NT are down to 1 second granularity
- on realtime obviously the OnBarUpdate events on multiple series come in at the sequence as sent out by the exchange

So:
- the EXACT sequence of a mix of bid/ask/last events will NOT be maintained while processing the historical data from multiple series
- the EXACT sequence of the bid/ask/last events by themselves WILL be maintained while processing the historical data from multiple series

However, on events with DIFFERENT timestamps the sequence always IS maintained.

Hope this sheds some light.

[funk101]

Hmm, ok, but what about plotting based on price, vertically as it were? Same does not apply it would seem.

[NinjaTrader_Dierk]

Sorry I don't follow.

[dnoff]

Why is NT data only timestamped down to the second? Clearly in any type of normal instrument, the bid ask data synchronized to this level has zero chance of being useful if is not able to be sequenced? Is this a data provider limitation or a NT limitation?

If it is an NT limitation, this design decision does not make sense to as this implementation is not all going to help those who are looking to historically test or even reload indicators on charts that use onMarketData(). As a result I do not see the benefit of adding this to NT 7 in this manner???

(I may be misunderstanding your comments above Dierk, but I thought you mentioned that bid ask data on historical data is only synchronized by timestamp?)

[NinjaTrader_JoshP]

Not all of the data providers NT supports offers millisecond timestamping.

[dnoff]

Not all of the data providers NT supports offers millisecond timestamping.

Thanks for the reply and this makes sense. However I would have thought as a design solution, could you have taken the opposite approach here. For example, I would include a timestamp that is down to the millisecond, and when a data provider does not support millisecond timestamps, I would add ".000" to the timestamp when it is received.

This way you are enabling a solution for those those data providers and traders wanting this. Today it seems that instead it is degraded to the lowest common denominator, you can support those data feeds that do. Then as user of the platform I can chose to go to those data vendors that support if it fits my trading needs??

[NinjaTrader_Dierk]

Guys, here is what you needed to understand: no whatever fine granularity would ensure that bid/ask/last could be replayed in the right sequence. There always could be bid/ask/last events having the exact same timestamp (as sent by the exchange) and then your trading system relying on the correct sequence would just fail.

-> asking for finer granularity on the timestamp is the wrong approach. The only approach which really would resolve the issue would be sequence counter. We held off on implementing this with NT7, since the ressource investment would have delayed NT7 much further than the majority of our users would be willing to accept.

[trend]

Hi,
how far back the bid/ask data in the NT servers go?
Any guidance for the brokers on their bid/ask historical data
availability?

thanks,

[NinjaTrader_Dierk]

- On our data server I believe you should see a couple of months worth of data.
- please contact your broker to clarify how far their data would go.

[darthtrader]

-> asking for finer granularity on the timestamp is the wrong approach. The only approach which really would resolve the issue would be sequence counter. We held off on implementing this with NT7, since the ressource investment would have delayed NT7 much further than the majority of our users would be willing to accept.

Dierk, no offensive but I can't disagree more. What you posted makes sense from a software engineering standpoint of getting things EXACTLY right. From a trading standpoint though, the exact sequence would only matter to someone trying to develope a real high frequency strategy system in Ninja...
Anyone with any real knowledge of high frequency strategies is not going to bother with Ninja anyway.
With this design decision though you guys are throwing out the baby with the bathwater..
For alot of people, I imagine what seperates Ninja from other software is onmarketdata()...It strikes me as obvious that a less than optimal historic onmarketdata solution would have vastly higher utility than no solution at all..
A great example is Volume profile..while I simply love your software..its hard to "pimp" to my market profile type trading friends when you have to caveat how great volume profile is with that if anything happens with your computer, you lose all your information...or if you wake up at 11am to get ready for an afternoon session, you should have set your alarm earlier to start collecting data pre cash open...its absurd from a trading standpoint.