Missing Yahoo Split Data

[wd2trade]

Hello,

I've been working with NinjaTrader for the past few months and it's worked well.

Today I tried to update the split data for NVDA, and nothing was downloaded. I know this is incorrect because this stock split 3:2 on 9/11/2007. I tried a different stock (SNHY) and it downloaded the correct splits.

I'm using the Yahoo feed for split and dividend data. Is there something I'm missing?

Thanks,
Bill

[NinjaTrader_Ray]

We will look into it and post back by tomorrow.

[NinjaTrader_Dierk]

Unfortunately YAHOO's data is inconsistent regrading NVDA: no splits nor dividends reported as request

http://finance.yahoo.com/q/hp?s=NVDA&a=00&b=1&c=1900&d=0&e=10&f=2008&g=v&y=0

... is issued. No problem on SNHY though

http://finance.yahoo.com/q/hp?s=SNHY&a=00&b=1&c=1900&d=0&e=10&f=2008&g=v&y=0

Sorry, there is nothing we can do about.

[wd2trade]

Unfortunately YAHOO's data is inconsistent regrading NVDA: no splits nor dividends reported as request


Thanks for the quick response on this, Dierk.

After seeing your query, I realized this is the same problem I ran into doing custom data grabs. I used the same query you are using, and I would not get any split data for stocks without dividends (i.e. I would get neither dividend nor split data, as is the case here).

I was able to fix it by using a "monthly price" instead of "dividend only" yahoo query:

http://finance.yahoo.com/q/hp?s=NVDA&a=00&b=1&c=1900&d=0&e=10&f=2008&g=m&y=0

This gets all dividends and splits, but also includes monthly price data which I ignore.

Is there any possibility of using this this query in NinjaTrader so I can download the data?

Thanks again,
Bill

[NinjaTrader_Dierk]

>> Is there any possibility of using this this query in NinjaTrader so I can download the data?

Unfortunately not. We'll look into to see what we could get done with one of the next beta/releases.

[NinjaTrader_Christian]

Hi wd2trade !

With the link you mention, i get only the splits, but no dividends for NVDA ?

Though I get the dividends for SNHY.

Kind regards,
Christian

[wd2trade]

With the link you mention, i get only the splits, but no dividends for NVDA ?

What you're getting is correct: NVDA has never paid a dividend.

I believe this is the core problem with the original query. If you select "Dividends only", and the stock pays no dividends, then it returns nothing (not even splits)

Thanks for looking at this,
Bill

[NinjaTrader_Christian]

Ok, I understand.

We will discuss, and see, what we can do here.

Thanks for bringing this up !

Kind regards,
Christian

[NinjaTrader_Dierk]

Thanks for your suggestion. We'll add it to the list of future considerations.

[rolfewiz]

Any word on this? This is a major flaw in the backtesting.

[NinjaTrader_Dierk]

Sorry, what flaw? Split data is available, just not for NVDA since there is a problem with YAHOO's data.

[wd2trade]

Sorry, what flaw? Split data is available, just not for NVDA since there is a problem with YAHOO's data.

Whatever the cause, the data is still screwed up for any stock which has never paid a dividend, and there is an alternative query which works correctly.

Thanks for your suggestion. We'll add it to the list of future considerations.Are you guys still considering changing the query to get the correct data, or is that a dead end?

Thanks,
Bill

[NinjaTrader_Dierk]

Unfortunately the alternative query does not work, since there have been other limitations which rendered it unusable (don't recall which ones though).

[rolfewiz]

Uh, the flaw that everyone on this thread is talking about. This is not a trivial problem. There are many equities with this problem (NIHD is another one). You can't run a backtest on a basket of stocks without weeding these out first else the missing split info will corrupt your results dramatically.

This is not a problem with Yahoo, it's a misassumption in the query Ninjatrader uses. Yahoo does provide the split data. The problem is that right now, the user (ie me) is completely helpless as far as I can see. There's no way I can remedy this problem myself. There's no workaround. Opentick provides it also. Why not allow split data from Opentick? I can't even add the splits manually.

You can get the splits from here:
http://finance.yahoo.com/q/bc?t=3m&l=on&z=m&q=l&p=&a=&c=&s=NVDA

And here is the code in C++ to parse the splits from the HTML. I did my best to preserve the code formatting:


struct Split
{
time_t time;
unsigned int from;
unsigned int to;
};

vector<Split> *DataManager_Yahoo::parseSplitDataFromHTML( string filepath )
{
static vector<Split> data;

data.clear();

FileBuf f;
if( !f.load(filepath) )
{
return NULL;
}

//looking for this:
//<center>Splits:none</center>
//or this:
//<center>Splits:<nobr>02-Sep-97 [3:2]</nobr>, <nobr>03-Aug-98 [2:1]</nobr>, <nobr>08-Feb-99 [2:1]</nobr>, <nobr>14-Feb-00 [2:1]</nobr>, <nobr>12-May-04 [2:1]</nobr></center>

string s(f.buf, f.size);
size_t start = s.find("<center>Splits:");
s = s.substr(start + 15);
size_t end = s.find("</center>");
s = s.substr(0, end+1);

if( s.find("none") != s.npos )
{
return &data;
}

while( s.find("<nobr>") != s.npos )
{
s.replace(s.find("<nobr>"), 6, "");
}
while( s.find("</nobr>") != s.npos )
{
s.replace(s.find("</nobr>"), 7, "");
}
while( s.find(",") != s.npos )
{
s.replace(s.find(","), 1, "");
}
while( s.find("-") != s.npos )
{
s.replace(s.find("-"), 1, " ");
}
while( s.find("[") != s.npos )
{
s.replace(s.find("["), 1, "");
}
while( s.find(":") != s.npos )
{
s.replace(s.find(":"), 1, " ");
}

//do a string.find and divide the result by 3 to get the month number.
string months = "000JanFebMarAprMayJunJulAugSepOctNovDec";

while( s.find(']') != s.npos )
{
SYSTEMTIME t;
memset(&t, 0, sizeof(SYSTEMTIME));
string month;
Split split;

strstream stream((char*)s.c_str(), s.size(), ios_base::out);

stream >> t.wDay >> month >> t.wYear >> split.to >> split.from;

if( t.wYear > 50 ) t.wYear += 1900;
else t.wYear += 2000;

t.wMonth = months.find(month) / 3;

split.time = systemTimeToTime_t(t);
split.time -= 8 * 60 * 60;

data.push_back(split);

size_t n = s.find(']');
s = s.substr(n+2);
}

return &data;
}

time_t systemTimeToTime_t(SYSTEMTIME st)
{
tm t;
memset(&t, 0, sizeof(tm));
t.tm_mon = st.wMonth-1;
t.tm_mday = st.wDay;
t.tm_year = st.wYear - 1900;

time_t ret = mktime(&t);
return ret;
}

[NinjaTrader_Dierk]

Thanks for your suggestion. We'll add it to the list of future considerations.

[rolfewiz]

Any word on this? This is a major flaw in the backtesting.

[NinjaTrader_Dierk]

Please see my post #15 below.

[rolfewiz]

Runtime Error: Stack Overflow! Infinite loop detected.

[NinjaTrader_Dierk]

Sorry, not sure I follow.

[NinjaTrader_Dierk]

We revisited the issue and will provide an updated implemented with next beta. This should work better.