Daily vs Minute bar mismatch?

[ErikHR]

I this simply an IQFeed data error or something else? I cleared historical data then check the daily chart and minute chart and numbers are different (see attachments below). I see this with other NASDAQ symbols from 1/1/2007 to present (at least 6-12 of the NASDAQ 100 list).

Also, I have a strategy that runs on the daily chart and adds Minute bars to execute the trades. I have looked at the documentation and think this is a valid Multi-Instrument strategy. Do you see anything glaringly wrong?
Thanks,
Erik

[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]protected[/COLOR][/COLOR][COLOR=#0000ff][COLOR=#0000ff]override[/COLOR][/COLOR][COLOR=#0000ff][COLOR=#0000ff]void[/COLOR][/COLOR] Initialize() {[/SIZE][/FONT]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]try[/COLOR][/COLOR] {[/SIZE][/FONT]
[COLOR=#008000][FONT=Courier New][SIZE=1][COLOR=#008000]//TraceOrders = true;[/COLOR][/SIZE][/FONT]
[/COLOR][FONT=Courier New][SIZE=1]Add([COLOR=#2b91af][COLOR=#2b91af]PeriodType[/COLOR][/COLOR].Minute, 5);[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]...[/SIZE][/FONT]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]if[/COLOR][/COLOR] (useDarkCloudCover == 1) Add(CandleStickPattern([COLOR=#2b91af][COLOR=#2b91af]ChartPattern[/COLOR][/COLOR].DarkCloudCover, swingStrength));[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]...[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]SetProfitTarget([COLOR=#a31515][COLOR=#a31515]""[/COLOR][/COLOR], [COLOR=#2b91af][COLOR=#2b91af]CalculationMode[/COLOR][/COLOR].Percent, profitTarget / 100);[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]SetStopLoss([COLOR=#a31515][COLOR=#a31515]""[/COLOR][/COLOR], [COLOR=#2b91af][COLOR=#2b91af]CalculationMode[/COLOR][/COLOR].Percent, stopLoss / 100, [COLOR=#0000ff][COLOR=#0000ff]true[/COLOR][/COLOR]);[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]IOrder1 = [COLOR=#0000ff][COLOR=#0000ff]null[/COLOR][/COLOR];[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]triggerFired = [COLOR=#0000ff][COLOR=#0000ff]false[/COLOR][/COLOR];[/SIZE][/FONT]
 
 
[FONT=Courier New][SIZE=1]EntriesPerDirection = 1;[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]EntryHandling = [COLOR=#2b91af][COLOR=#2b91af]EntryHandling[/COLOR][/COLOR].UniqueEntries;[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]ExitOnClose = [COLOR=#0000ff][COLOR=#0000ff]false[/COLOR][/COLOR];[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]TimeInForce = Cbi.[COLOR=#2b91af][COLOR=#2b91af]TimeInForce[/COLOR][/COLOR].Gtc;[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]CalculateOnBarClose = [COLOR=#0000ff][COLOR=#0000ff]true[/COLOR][/COLOR];[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]catch[/COLOR][/COLOR] ([COLOR=#2b91af][COLOR=#2b91af]Exception[/COLOR][/COLOR] ex) {[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]Print(ex.ToString());[/SIZE][/FONT]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]throw[/COLOR][/COLOR];[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[SIZE=1][FONT=Courier New][COLOR=#0000ff][COLOR=#0000ff]protected[/COLOR][/COLOR] [COLOR=#0000ff][COLOR=#0000ff]override[/COLOR][/COLOR] [COLOR=#0000ff][COLOR=#0000ff]void[/COLOR][/COLOR][/FONT][FONT=Courier New] OnBarUpdate() {[/FONT][/SIZE]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]try[/COLOR][/COLOR] {[/SIZE][/FONT]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]if[/COLOR][/COLOR] (BarsInProgress == 0) {[/SIZE][/FONT]
[COLOR=#0000ff][FONT=Courier New][SIZE=1][COLOR=#0000ff]...[/COLOR][/SIZE][/FONT][/COLOR]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]if[/COLOR][/COLOR] (useDarkCloudCover == 1 && CandleStickPattern([COLOR=#2b91af][COLOR=#2b91af]ChartPattern[/COLOR][/COLOR].DarkCloudCover, swingStrength).PatternFound[0] == 1) {[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]triggerFired = [COLOR=#0000ff][COLOR=#0000ff]true[/COLOR][/COLOR];[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]...[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]if[/COLOR][/COLOR] (BarsInProgress == 1) {[/SIZE][/FONT]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]if[/COLOR][/COLOR] (triggerFired) {[/SIZE][/FONT]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]if[/COLOR][/COLOR] (ToTime(Time[0]) >= 63000 && ToTime(Time[0]) <= 83000) {[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]IOrder1 = EnterShortLimit(BarsInProgress, [COLOR=#0000ff][COLOR=#0000ff]true[/COLOR][/COLOR], GetNumSharesToBuy(Close[0]), Close[0], [COLOR=#a31515][COLOR=#a31515]""[/COLOR][/COLOR]);[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]triggerFired = [COLOR=#0000ff][COLOR=#0000ff]false[/COLOR][/COLOR];[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[FONT=Courier New][SIZE=1][COLOR=#0000ff][COLOR=#0000ff]if[/COLOR][/COLOR] (ToTime(Time[0]) > 125000) {[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]CancelOrders();[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[SIZE=1][FONT=Courier New][COLOR=#0000ff][COLOR=#0000ff]catch[/COLOR][/COLOR] ([COLOR=#2b91af][COLOR=#2b91af]Exception[/COLOR][/COLOR][/FONT][FONT=Courier New] ex) {[/FONT][/SIZE]
[FONT=Courier New][SIZE=1]Print(ex.ToString());[/SIZE][/FONT]
[SIZE=1][COLOR=#0000ff][FONT=Courier New][COLOR=#0000ff]throw[/COLOR][/FONT][/COLOR][FONT=Courier New];[/FONT][/SIZE]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]
[FONT=Courier New][SIZE=1]}[/SIZE][/FONT]

[NinjaTrader_RyanM1]

Hello ErikHR,

Which exact figures are you comparing here?

There can be differences depending on a few different factors.

Is there an issue with the strategy you're using or just the data between the two intervals?

[NinjaTrader_JoshP]

ErikHR,

Using 30 minute bars may not necessarily be the best way to test this. A 30 min bar of timestamp 7:00 includes all data from 6:30 to 7:00. I am not sure what timezone you are in so would not know if that would include premarket data for you.

In general though, minute data and daily data are not the necessarily the same. Daily bars are not constructed from minute bars in most cases and what that means is if the data from the provider simply has different minute data versus daily data then you would see such discrepancies. Also, daily bars would sometimes be modified to reflect settlement prices and such. Different providers may use different session templates to build their daily bars too. So there are a lot of factors as to what you may/may not be seeing.

[ErikHR]

Basically if you look at attachment 2 the entire range of data for the day is between 43 and 45 and if you look at the bar for the daily data in attachment 1 the the low and the High are 29.38 and 30.17.

So my question is how do I do this? or will it not work since the data is different?

Erik,
I guess I still don't follow. When you are backtesting, the trades will have to go onto the NEXT bar. The only way you can get it to trade on the current bar is to add intrabar granularity.
__________________
Josh, NinjaTrader Customer Service

[NinjaTrader_Brett]

Erik,


Can you plaese further clarify what it is you are trying to do so that we may further assist.

I look forward to assisting you further.

[ErikHR]

I have a strategy: Daily Bars Calculate on close = true; When it gets a signal the minute bars place a limit order at the first bar of the regular session, and cancel the limit order if not filled by some designated time. Otherwise if a position is opened it uses a profit target, stop loss, and/or number of days to exit the trade.
This strategy back tests and optimizes fine for the majority of the NASDAQ 100 with the exception of some instruments like NVDA that have minute data and daily data that are totally different values.
If you look at attachment 1 below look at the left side of the image, those dotted green lines that fall from the top of the chart do so, because the minute bars range of values is off the chart. It is not like I caught a premarket entry or something if you use IQFeed to pull the minute chart (attachment 2 below) none of the values for that day are in the range of the daily values. minute bars are in the 40s and daily bars are in the 20s probably due to a split and/or bad data that is accounted for in one dataset, but not the other.

So the questions are:
1. Do I have NinjaTrader configured wrong?
2. Am I using NinjaTrader wrong?
3. Do I just need to not use these symbols for back testing and optimization until I can get IQFeed to fix the data?

Thanks,
Erik

[NinjaTrader_Brett]

Hello,

Thank you for your note.

This is not off by a little bit this is off for quite a bit. I have confirmed this on my side with the IQFeed and now understand the screenshots you provided were from 2007. THis may be part of the issue. Please contact IQ Feed and let them know what data you are getting from daily data and what data you are getting from Minute data and see if there is some issue with there data servers here or if this data matches what they have.

Let me know what you find out.

[AnotherTrader]

I believe the historical daily data provided by many data providers has been adjusted for historical splits and dividends, whereas the intraday data is "raw" data as provided at the time by the exchange.

Therefore, if you want the daily data (e.g. OHLC) that corresponds to the intraday data you are using, you may need to calculate it from the (unadjusted) intraday data itself, and not use the historical (adjusted) daily data.

Hope that helps.

[ErikHR]

SDS data from IQFeed seems to have this same issue I talked to IQFeed a while back and they said just what AnotherTrader said, their minute data is raw and their daily data is adjusted.

I have a strategy that I want to use daily bars for gathering indicator data (Vol, RSI, Trend, etc...) and daily bars to Buy the Close and Sell the Open.

Sample strategy:

[SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]protected[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]override[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]void[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] Initialize() {[/SIZE]
[SIZE=2]Add([/SIZE][SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]PeriodType[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2].Minute, 1);[/SIZE]
[SIZE=2]BarsRequired = 5;[/SIZE]
[SIZE=2]TimeInForce = Cbi.[/SIZE][SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]TimeInForce[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2].Day;[/SIZE]
[SIZE=2]EntryHandling = [/SIZE][SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]EntryHandling[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2].UniqueEntries;[/SIZE]
[SIZE=2]EntriesPerDirection = 1;[/SIZE]
[SIZE=2]ExitOnClose = [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]false[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2];[/SIZE]
[SIZE=2]CalculateOnBarClose = [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]true[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2];[/SIZE]
[SIZE=2]}[/SIZE]
[SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]protected[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]override[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]void[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] OnBarUpdate() {[/SIZE]
[SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]for[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] ([/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]int[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] i = 0; i < CurrentBars.Length; i++) [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]if[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] (CurrentBars[i] <= BarsRequired) [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]return[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2];[/SIZE]
[SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]if[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] (BarsInProgress == 0) {[/SIZE]
[SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]////Daily indicators go here.[/COLOR][/SIZE]
[/COLOR][/SIZE][SIZE=2]}[/SIZE]
[SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]else[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] {[/SIZE]
[SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//minute[/COLOR][/SIZE]
[/COLOR][/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]if[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] (Position.MarketPosition == [/SIZE][SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]MarketPosition[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2].Long && ToTime(GetEastStdTime(Time[0])) > 93000 && ToTime(GetEastStdTime(Time[0])) <= 103000) {[/SIZE]
[SIZE=2]ExitLong();[/SIZE]
[SIZE=2]}[/SIZE]
[SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]if[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] (Position.MarketPosition != [/SIZE][SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]MarketPosition[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2].Long && ToTime(GetEastStdTime(Time[0])) >= 155000) {[/SIZE]
[SIZE=2]EnterLongLimit(1000, Close[0] + 0.02);[/SIZE]
[SIZE=2]}[/SIZE]
[SIZE=2]}[/SIZE]
[SIZE=2]}[/SIZE]
[SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]// Need to dynamically adjust time since I am in Arizona.[/COLOR][/SIZE]
[/COLOR][/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]protected[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]DateTime[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] GetEastStdTime([/SIZE][SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]DateTime[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] time) {[/SIZE]
[SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]TimeZoneInfo[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] EastStdInfo;[/SIZE]
[SIZE=2]EastStdInfo = [/SIZE][SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]TimeZoneInfo[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2].FindSystemTimeZoneById([/SIZE][SIZE=2][COLOR=#a31515][SIZE=2][COLOR=#a31515]"Eastern Standard Time"[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]);[/SIZE]
[SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]return[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#2b91af][SIZE=2][COLOR=#2b91af]TimeZoneInfo[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2].ConvertTime(time, EastStdInfo);[/SIZE]
[SIZE=2]}[/SIZE]

What are the BKMs for getting this to work? Or am I SOL with this symbol?

Thanks,
Erik

Attached a backtesting image and one from yahoo.

[NinjaTrader_Dexter]

Hello ErikHR,

A note regarding adjusted daily data, you can overcome this by adding a series consisting of 1440 minutes. This way it will be constructed from the mentioned raw minute data.

Another thing to note regarding daily bars, daily bars do not observe data or date restrictions from session templates. They are timestamped from the provider so this can cause discrepancies as well.

I'm unsure what you mean by BKM, but the code you posted looks good.

[ErikHR]

BKM is just short for Best Know Method.

I will try the series consisting of 1440 minutes.

How should I have the attached picture configured?

Thanks,
Erik

Thanks,
Erik

[NinjaTrader_Brett]

Hello,

Sounds good. Also the way you have this setup is fine and how I would run it on my side. This makes sure that you always get historical data from your provider.

[ErikHR]

Since the minute data is raw, is there any good way to not show a big loss in my back testing and optimization every time a dividend is paid and the stock drops?
Thanks,
Erik

[NinjaTrader_Brett]

Hello,

You would need to adjust the data. You can try downloading the adjustment data from yahoo to adjust the data if this is on a stock.

Please see this information on how to do this:

http://www.ninjatrader.com/support/helpGuides/nt7/adding_splits_and_dividends.htm?zoom_highlightsub=splits

Let me know if I can be of further assistance.

[ErikHR]

If I use the 1440 minute bar instead of the daily bar do I have to use:
SessionData = "US Equities RTH";
or can I use:
SessionData = "US Equities ETH";
So I can see the premarket Volume etc...
Thanks,
Erik