I recently came across couple of articles in AllthingsD where two C level executives have put forth their point of view on what is more critical to business (More data or better algorithms?). I believe since this is a BI blog, we might as a group be more inclined towards the more data argument than the better algorithm argument. I would like to put forth my views on why better algorithms would do a world of good to business.
Its easy to say, we need both, more data and better algorithms. But very few companies have the resources and the skill to manage and deliver on both counts.So businesses might have to choose between them.
I am parroting both sides of the arguments below and would finally make my case for better use of algorithms in BI.
Argument towards More data
http://allthingsd.com/20120907/more-data-beats-better-algorithms-or-does-it/?refcat=voices
The argument put forth by the BlueKai CEO Omar Tawakol is that disparate data points are not useful to any business, but when the disparate data points are connected and a context related to them, the data becomes more richer and meaningful than ever.
The Future of Big Data
With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.
He argues that the degree of connectedness of the data which are laid beneath the analytical layer determines the relevancy and correlations that can be obtained from the data. Finally he points out that an isolated behavior in the data, when evaluated and connected, can produce unexpected value
Argument towards Better Algorithms
http://allthingsd.com/20121128/better-algorithms-beat-more-data-and-heres-why/
The argument that Mark Torrance is CTO of Rocket Fuel makes is that data might be rich and can be highly augmented but they mostly represent static information at any point of time and is less useful without good algorithms to sift through and find anomalies and business opportunities.
He believes that algorithms can extend the usefulness of the data assets and helps create significant and measurable improvements which cannot be obtained from more data. He does accept that more data can give better insights but only marginal gains compared to what better algorithms can.
The Case for better use of algorithms in BI
In BI we mostly structure the data in a manner useful for business to answer their questions.But we have probably ignored the use of better algorithms to help business gain useful insights into the business.
There are several advantages that algorithms can offer that more data simply cannot offer.
-
Algorithms can identify anomalies in the data faster and interrelate data to produce better results.
-
Unless the data is highly connected, it is near impossible to identify causation and correlation in the data. Algorithms can help business identify better connections between disparate data points even if the data is not rich enough.
-
Algorithms are highly useful in predictive behavior of the events and help take actions on the behavior.
-
Algorithms can react to changing business needs faster than data, and since data usually represent static points and or events in time, they fall short to address the changing nature of business models.
-
With the high growth and volume of unstructured data in the business and the consumer world, business would be better equipped if they focus on better algorithms than forcing structure to the data.
When will business invest time, energy and resources in using better Algorithms ?
I believe many businesses understand the need for better algorithms but don’t have the resources and the will to make use of the data. There would be point when they realize they have obtained all the insights there is through the structured data that they have assembled in their data warehouses. This might compel them to look at the connectedness of data and using algorithms to gain performance and identify new business opportunities.
