TD Ameritrade Missing daily historical data

[scjohn]

When charting a new stock symbol, a daily chart will not have the previous days data. For example, a new daily chart for SNE will be missing the daily data for 01/22/09, with today's date being 01/23/09. I also have a linked chart that is set to 60-minutes. The 60-minute data for 01/22/09 appears. This is all before the market opens. After the market opens today's data does appear on the daily and 60-minute chart. However I am still missing the daily data for 01/22/09. The only way to get the missing previous day's data is to do Ctrl|Shift|R - Reload historical data. Now the missing day appears. It's a pain to have to a Ctrl|Shift|R

Note: I have a Market Analyzer running with 34 stock symbols. The Market Analyzer is Linked to 2 charts. A daily chart and a 60-minute chart. It's a pain to have to a Ctrl|Shift|R on 34 symbols to get the missing daily data.

Do not know if the problem is that TD Ameritrade is not sending down the data or NT's is dropping the bar of daily data. I tens to the latter since a Reload will result in the missing day showing up.

[NinjaTrader_Michael]

Hi SCJohn,


Do these charts open in a workspace?


If so I recommend creating a new workspace with a few daily charts, please let me know if you still have the irregularity with a new workspace.


Managing workspaces:

http://www.ninjatrader-support.com/HelpGuideV6/ManagingWorkspaces.html

[scjohn]

Yes. The Market Analyzer and the 2 charts are all part of workspace. And the workspace is not the 'default' workspace.

[NinjaTrader_Vincent]

Hi,

Try repairing your database under Tools--> Options--> Misc tab--> Repair DB.

[scjohn]

I repaired the DB as requested.

Started NT. I closed the existing Workspace and opened a New Workspace. Connected to the TD Ameritrade data feed. I created a new Chart and used a symbol that was NOT in the Market Analyzer in the just closed workspace. Symbol was PPDI. Friday's Historical data, 01/23/09 was on the chart.

Went back and opened the original Workspace. I looked a several symbols that, remember that the Market Analyzer is Linked to 2 chart, and Friday's data was there. I now proceeded to add 5 new symbols to the Market Analyzer. Note: the Market Analyzer has 2 columns that display indicators. The first indicator is calculated on daily data and the 2nd indicator is calculated on 60-minute data. The 1st 4 symbols that I added were all missing Friday's daily data - 01/23/09 (all 60-minute data was there). The 5th symbol - PPDI (the one I used in a new workspace) had Friday's daily data and the 60-minute data.

[NinjaTrader_Michael]

Hi SCJohn,



Please delete all current workspaces and create a new one as desired, after doing so please let us know if the backfill on the chart loads correctly.

To delete your workspaces:
• Please close NinjaTrader and go to your desktop Start menu
• Open My Documents
• Go to the NinjaTrader 6.5 folder
• Go to the Workspaces folder
• Right click on all XML documents and select Delete
• Now restart the application and it should start up normally

[scjohn]

Deleted all workspaces, repaired the DB - again, reset the instrument list, rebuilt the Workspace. Guess what? Added 3 new symbols to the Market Analyzer, (remember that the 2 charts - daily and 60-minute are linked to the Market Analyzer), clicked on the the 3 new symbols are yesterday's date, 01/27/09, are missing. 01/27/09 is also missing on the daily for a lot of other symbols in the Market Analyzer. Have not checked them all.

[NinjaTrader_Michael]

Hi,


Please send an e-mail to support at NinjaTrader dot com referencing this post with ATTN: Mike in the subject line so I can further investigate your inquiry.

Could you also add a screenshot as well?

To send a screenshot press Alt + PRINT SCREEN to take a screen shot of the selected window. Then go to Start--> Accessories--> Paint, and press CRTL + V to paste the image. Lastly, save as a jpeg file and send the file as an attachment.

[Mannygags]

Was wondering if a solution to this problem was found, as I am having the same issue - previous day's historical data is missing from the chart during the trading day. Although the data appears to be there when I re-open NT later in the evening.


Thanks
Manny

[scjohn]

After exchanging numerous email with Mike at NT support, they have not been able to replicate the problem. To the best of my knowledge they have only tried once to replicate the problem and that was with another support member. Not sure what 'support member' means.

I am still having the problem of the previous days data not showing up on the daily chart before the market opens. After the market opens, no problem.

My workspace is made up of 1 Market Analyzer and 2 charts. The 2 charts are Linked to the Market Analyzer. So when ever I click on a row in the Market Analyzer, the 2 charts will show show data for that symbol in the Market Analyzer. I never have any probelm with the 60-minute chart. It is always the daily chart.

I have repaired the data base numerous times, deleted and rebuilt the workspace, and refreshed the Instrument list. Nothing.

I have downloaded TD Ameritrade's StrategyDesk and their program does not have this problem.

At this point NT is blaming the problem on 3rd party indicator(s). I have deleted the daily chart and have rebuilt it only with NT supplied indicator - the EMA.

I suggest that anybody that is having a problem with daily data on daily charts please document your problem on this thread. At this point NT does not think it is there problem.

Edit: Forcing a Reload Historical data on the Daily chart will usually fix the problem.

[NinjaTrader_Jason]

Hello scjohn,

Can you please tell me what version of NinjaTrader you have installed. You can check this under Help-->About.

[scjohn]

Version 6.5.1000.7

[NinjaTrader_Jason]

Can you please install the latest version at the link below and check if the issue persists.
http://www.ninjatrader.com/webnew/download_trading_software.htm

[scjohn]

Jason

I upgraded to version 6.5.1000.8 last night, after the market closed.

Started NT this morning (before the market opens) and opened the default workspace. Default workspace contains 1 Market Analyzer and 2 charts ( daily and 60-minute). The 2 charts are linked to the Market Analyzer. The daily chart has no 3rd party indicators, just10 EMAs with varying periods. The Market Analyzer contained 25 symbols. I then proceed to connect to TS Amertridae. I let the Market Analyzer finish updating.

I add 5 new symbols to Market Analyzer. The symbols are: COGO, FDO, HAR, NCR and XRAY. To the best of my knowledge there are no historical data in the Database for these symbols.

At this point the market is still closed.

I click on the row, in Market Analyzer, that contains COGO. Yesterday's data, 02/05/09 is present on the daily chart.

I click on FDO. Yesterday's data is missing on the daily chart.

I click on HAR. Yesterday's data is missing on the daily chart.


At this point I wait for the maket to open. The maket opens.

I click on NCR. Yesterday's data is present along with todays data on the daily chart.

I click on XRAY. Yesterday's data is present along with todays data on the daily chart.

I know go back to the 1st 3 symbols.

I click on COGO. The daily chart is still correct.

I click on FDO. I get 2 error messages. See attachement.

I click on HAR. The daily chart is updated and the previous day's data is now present along with today's data.

I go back and click on FDO. This time there are no errors and the daily chart is updated with the previous day's data and today's data.

Note: TD Ameritrad's StrategyDesk was able to load all of the above five symbols, before the market opened, with yesterday's data on a daily chart.

Edit. The only difference I see after the upgrade is that I do need to force a Reload Historical data on the daily chart to get the missing data after the market opens.

[Mannygags]

Hi. I have been using v 6.5.100.8 and am still having similar issues. I am viewing daily charts and the previous day's bar is either missing, or the OHLC values are all screwy. Sometimes reloading the data can fix for a particular security, sometimes not. Just tried to reload for 'XLF' and NT appears to be hanging. A couple other symbols that I have been looking at that are giving problems are 'DIA', 'QQQQ', 'EEM' and 'EFA.'

Any suggestions? It's defintiely becoming frustrating to have to go back and check the data on the bars for each symbol that I attempt to look up.

Thanks

Manny

[NinjaTrader_Jason]

Hello scjohn,

The error in the attached screenshot is only displayed in our software. It seems a message from TD AMERITRADE stating they were unable to provide the requested historical data at that time. Does the the error persist upon requesting it a different time? If so, I sugggest to contact your broker regarding the historical data for this instrument.

It seems all other data is displayed as expected during market hours. I am not sure why yesterday's data was not present for the two stocks mentioned. Does this occur with any of the nstruments already present in the Market Analyzer?

I suggest to check if this behavior persists Monday morning before markets open.

[NinjaTrader_Jason]

Hello Manny,

I just tested a XLF, EEM and EFA daily chart on my end and it loaded the requested data including yesterday's bar. It took a few moments to load on my end.

I suggest to restart NinjaTrader and try once more. It could be your setup is requesting too much resources; try the stocks with only one chart opened and check if this will make a difference.

[scjohn]

Manny,

Regarding "hanging" when loading data on a chart. See this thread:
http://www.ninjatrader-support2.com/vb/showthread.php?t=13703

My experience has been that anything longer than 60 seconds and NT is hung up waiting for data that will never come. At this point the only way I can clear this condition is to restart NT.

[scjohn]

Jason,

The second time I went back to FDO, the daily chart was correct. No missing data.

Because of my use of Market Analyzer and link my charts to the Market Anlyzer, I feel that the historical data in the Database is either missing or inaccurate. See: http://www.ninjatrader-support.com/HelpGuideV6/helpguide.html?HowDoesNinjaTraderBuildChartBars .

I uses to have a column in the Market Analyzer that was an indicator based upon daily data and was calculated on Calculate on bar close = true. I would be missing yesterday's data and would have to force a Reload Historical data after the maker opened for the daily chart.

I will forward several emails that i sent to Mike to your attention at support.ninjatrader. and referencing this thread.

[Mannygags]

Hi Jason. I repaired the database, restarted, shut down the 3 market analyzers that had been running and tried bringing up 'XLF', 'EFA' and 'OIL.' The data for yesterday's bar was wrong, but this time I was able to reload the data and it gave me the correct data. 'DIA' and 'QQQQ' were correct without having to reload the data.

Do you think it can be a resource issue on my machine? My market analyzers are not linked to my chart. Is one chart and three market analyzers taxing on my system?

Thanks

Manny

[NinjaTrader_Jason]

Hello Manny,

That could be the case. Depending on how many instruments, columns and indicators the Market Analyzer contain, it can take up quite some resources.

Please see the following link for more instructions to improve the performance of NinjaTrader.
http://www.ninjatrader-support.com/HelpGuideV6/PerformanceTips1.html

[scjohn]

Manny,

I have 1 Market Analyzer linked to 2 charts - daily and 60-minute.

The Market Analyzer has 12 columns but only 1 column is an indicator. That indicator is running on a 60-minute interval with 'Compute on bar close' = true. I have had as many as 50 symbols in the Market Analyzer.

CPU utilization is under 12% about 90% of the time. Intel Dual Core 3.0 GHz with 4 GB ram. No problems.

Edit. Physical Memory usage is at 39% with 69 processes running.

[scjohn]

Tried an experiment this morning before the market opens. The experiment is to use IB instead of TD Ameritrade as the data feed.

Start NT and let the default workspace load and do it's thing. Workspace contains 1 Market Analyzer and 2 charts. The 2 charts are linked to the Market Analyzer.

Save the layout of the Market Analyzer.
Now delete the Market Analyzer.
Create a new Market Analyzer and load the previous saved layout.
Add 2 symbols to the new Market Analyzer: KG and MR. As far as I know these 2 symbols are not in the database.
Start IB's TWS and the connect to IB via NT.
Let the Market Analyzer update.
Link the Market Analyzer window to the 2 charts.
Click on the 2 rows/symbols to load data on the 2 charts.

After getting past the IB/NT pacing warnings, the daily chart(s) showed yesterday's data for both symbols. This would not have happened using the TD Ameritrade data feed.

You are left to draw your own conclusions.