Maximum frequency of updates to OnMarketData/Depth

[taotree]

I am using the Zenfire data feed with NT 7. I have a Print call in OnMarketData (I tested it in OnMarketDepth with the same results) filtered for Bid/Ask or Position 0 updates.

I am seeing no more than about 4 updates to level 1 data on ES. I'm sure that's not correct. I'm sure it should be updating more than that at some times. Is there some type of throttle, filter, or aggregation somewhere going on so that we do not receive every update to the bid and ask (Level 1)?

The update of the output window is limited to a certain frequency--that's fine and I'm not talking about that. Each update to the output window is showing several lines. My concern is that it's not showing the number of lines I would expect to represent every update to the data.

[NinjaTrader_Bertrand]

taotree, there's no limiting imposed by NinjaTrader - you would access either the Level 1 / Level 2 stream directly with those methods - would you mind sharing the script with which you tested?

[taotree]

Put this in an indicator and watch the ouput. It's quite obvious there is a 250 ms (1/4 second) aggregation going on.

		long lastTime;
		
		protected override void OnMarketData(MarketDataEventArgs e) {
			long now = DateTime.Now.Ticks / 10; // now in micros

			if (e.MarketDataType == MarketDataType.Bid) {
				if (lastTime != 0) {
					long diff = now - lastTime;
					Print(diff);
				}
				lastTime = now;
			}
		}

[NinjaTrader_Bertrand]

That's what I see here on avg, too - could it be simply CME recent changes to tick aggregation to adapt to the risen volume?

[RJay]

Put this in an indicator and watch the ouput. It's quite obvious there is a 250 ms (1/4 second) aggregation going on.

        long lastTime;
        
        protected override void OnMarketData(MarketDataEventArgs e) {
            long now = DateTime.Now.Ticks / 10; // now in micros

            if (e.MarketDataType == MarketDataType.Bid) {
                if (lastTime != 0) {
                    long diff = now - lastTime;
                    Print(diff);
                }
                lastTime = now;
            }
        }

Hi taotree,

It looks like you are comparing the date time ticks from all of the updates being called to OnMarketData against just the Bid updates.

I don't think you are measuring delay. It looks like you are measuring time between ticks. The way the code is written, from one tick up to several ticks can slip by between readings.

Could make for erratic results.

Just my Opinion.

RJay

[taotree]

RJay,

Look at where the assignment to lastTime is. It's inside the if Bid block. So, I am only comparing the Bid times.

And as I mentioned, the output makes it very apparent there is a 250 ms granularity. It shows exactly 250000, 500000, 750000 often and the other values are very close to those (slight differences due to processing times, network hiccups, whatever).

[taotree]

That's what I see here on avg, too - could it be simply CME recent changes to tick aggregation to adapt to the risen volume?

No, that would be crazy. The CME couldn't get away with that.

[taotree]

Well, it appears to be a zenfire thing. I was able to do the same test direct to their API and it also had the same issue, so... it's not NinjaTrader. I'll continue this conversation direct with zenfire.

Thanks!

Update: I was incorrect. I was testing with the Java API (which isn't really supported) which apparently has the same issue as NinjaTrader. The C++ API has no 4/second limit. So... there is an issue with NinjaTrader.

[RJay]

http://forums.zen-fire.com/viewtopic.php?f=3&t=191

Timestamp from your CPU

string dateformat2="yyMMddHHmmssffffff";

DateTime MicroSeconds;

MicroSeconds = DateTime.Now;

string MicroTimeStamp2 = MicroSeconds.ToString(dateformat2);

Print ("MicroTimeStamp2 = " + MicroTimeStamp2);

[taotree]

Timestamp from your CPU
...

I think (DateTime.Now.Ticks / 10) (1 tick is 100 nanoseconds) is simpler and more efficient than the code you put there. Though I guess it doesn't do pretty output if that's what you want, but I generally don't care about that so much since I'm just looking for time differences.

[RJay]

I think (DateTime.Now.Ticks / 10) (1 tick is 100 nanoseconds) is simpler and more efficient than the code you put there. Though I guess it doesn't do pretty output if that's what you want, but I generally don't care about that so much since I'm just looking for time differences.

You saw the zenfire link?

http://forums.zen-fire.com/viewtopic.php?f=3&t=191

[taotree]

You saw the zenfire link?

http://forums.zen-fire.com/viewtopic.php?f=3&t=191

Yes, but I don't understand how it's related to the issue I'm bringing up.

[taotree]

taotree, there's no limiting imposed by NinjaTrader - you would access either the Level 1 / Level 2 stream directly with those methods - would you mind sharing the script with which you tested?

After a long email conversation with zen-fire support (I thank them for their patience!) I discovered that both NinjaTrader and the Java API are broken for this. I was told that the Java API was written "to an older connect point that is for users on a slow connection". Apparently NinjaTrader is either using a similar older connect point or is throttling it itself.

I tested the C++ example, and got best bids very quickly without any 4/second limit.

So... apparently there is a NinjaTrader issue with this after all.

[NinjaTrader_Ray]

After a long email conversation with zen-fire support (I thank them for their patience!) I discovered that both NinjaTrader and the Java API are broken for this. I was told that the Java API was written "to an older connect point that is for users on a slow connection". Apparently NinjaTrader is either using a similar older connect point or is throttling it itself.

I tested the C++ example, and got best bids very quickly without any 4/second limit.

So... apparently there is a NinjaTrader issue with this after all.

To clarify, there is no NinjaTrader issue.

- NinjaTrader is NOT throttling any data
- The connect points we are using are the ones that Zen-Fire asked us to use
- If these servers throttle data then of course, you will see throttled data
- If Zen-Fire wishes us to use different connect points then we will change them
- If you wish to connect to different servers, you may do so by editing your Zen-Fire connection, pressing the "Settings >> " button and manually entering the connection information. You would have to of course get the connection information from Zen-Fire.

[taotree]

To clarify, there is no NinjaTrader issue.
- The connect points we are using are the ones that Zen-Fire asked us to use

To clarify, there is an issue for NinjaTrader users who use Zen-Fire data feed. The default install and config does not provide all level 1 updates. If a user does not want to be limited to 4/second limit on level 1 updates, it sounds like they'll need to contact Zen-fire to find out how to do so.

In other words, there may not be any bug or limit in NinjaTrader itself, but especially considering the number of users that are using this combination, there is an "issue" in that most likely they are all unaware that they are not getting the data they expected. For many, that won't matter. But especially considering the discussions on this forum about bid/ask analysis, those people most certainly are going to want to know about this.

There has been discussion about data drops in zenfire before. But this isn't data drops, this is snapshotting.

I said there was an issue because I was speaking from a user-centric perspective. You stated accurately that there is no issue from a product-centric perspective. As a person representing your company, you might want to consider trying to speak from a user-centric position. That can be accomplished while still making clear what's going on.