25.11.10

Tableau, the Smartest BI going?

Tableau 6 can process a query of 67 million records and produce a beautifully clean chart in 2 seconds.

I have recently been spending a fair amount of time extracting health data from many government sites and am surprised at how far these sites haven't come in the past 5 years.  There is a lot more data available, but it is painful to extract and compile.  Most of the front end tools cannot handle large batches of data and even with smaller queries you wait, and wait.  And visually?  Well, they're clunky. The user has to work to understand the message.



Having worked inside government I know how data is managed and how expensive it is to do so well.  Government is in the business of providing services, not making a profit, so expending millions (and millions) on IT solutions is hard for any Ministry or department to rationalize.  They have a difficult time proving they can recoup the costs or that it will directly benefit Canadians on a cost-effectiveness basis.
There is a greater and greater gap between the systems used to manage complexity in the private sector and the same systems in the public sector. An average citizen carrying an iPhone has access to real-time "situational awareness" that outstrips military systems costing millions of dollars. ~Tim O'Reilly, O'Reilly Media
But now, on the BI side there's Tableau 6.  I can see how at least 40% of my time as an analyst would be freed up to produce results to help management make better decisions.  Now if my time is opened up like that, think of all the projects or initiatives I could be supporting with that extra time.  That's just my time - think about those Directors and Managers who can now make decisions quicker.  Who can't get jazzed about that kind of efficiency?
... at some point your data analysis is only as good as the story you tell. The BI space around it is very large. And having tried all the BI tools in the current space around it, the only tool that works – and you have 4 billion rows a month, and I want to put that on an actual US map, and I have four years of data, and Tableau can do it in four seconds?!  ~Abhishek Mehta, Managing Director for big data and analytics for Bank of America
How is Tableau 6 able to achieve such speed?  Well, I don't completely understand the technical aspect, but it appears that they are handling memory differently than current applications.  Abhishek Mehta explains it in the interview as "pushing the code to the data" using their own code VizQL, which has not been done until now.  That's pretty smart.




Here's a bunch of links for health data (not technically open data) - just fyi - if you need health data and are in a masochistic mood.

Public Health Agency Cancer Surveillance:  http://dsol-smed.phac-aspc.gc.ca/dsol-smed/cancer/index-eng.php
Public Health Agency Notifiable Diseases:  http://dsol-smed.phac-aspc.gc.ca/dsol-smed/ndis/index-eng.php
Public Health Agency Injury Surveillance:  http://dsol-smed.phac-aspc.gc.ca/dsol-smed/ndis/index-eng.php
Public Health Agency Chronic Disease Infobase:  http://www.phac-aspc.gc.ca/sti-its-surv-epi/hepc/hepc_pt-eng.php  


Labels: