Skip to main content

Data & Intelligence

Big Data: Do you really need it? Findings and thoughts

Yeah, yeah, another post about Big Data. And, yes I know, for someone who complains about the number of posts on Big Data I’m only adding to the noise. But as I heard more and more about big data  I decided to build myself a little program that I could point at companies and track what people were saying about them. After a few months of data gathering I’ve found for a majority of companies most of the data I gathered was just noise. In fact in a large majority of cases (>85%) over half of what the social data out there was put out about a company was put out by the companies themselves or their employees, in some cases that percentage is as high as 98%.

Data Intelligence - The Future of Big Data
The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

If you filter that out you’re left a tiny bit of data and of that tiny bit only about half is of any use.  So now you’re essentially looking for needles in a haystack. It’s enough to make a man go home and be a family man. However, their rarity doesn’t necessarily make them valuable. Sometimes a needle is just a needle.  Which leads to an obvious question which Michael Wu made a really great post on:, “If 99.99% of Big Data is irrelevant, Why Do we need it?”

The answer? Maybe you do, maybe you don’t. Like I said in my little non-scientific study it was certainly true that most companies put out most of the noise about themselves. However, for certain companies (companies with large customer facing arms such as Hotels,  food, entertainment, etc) it certainly makes sense to make use the swell of information out there.  And certainly there’s big data that lies outside of the social sphere. Perhaps your company gathers data from the product base. That makes a lot of sense. However, for a lot of companies Big data might just not make sense yet.  In the post mentioned above Michael Wu postulates 3 scenarios to help ascertain if you’re ready for big data they are:

  1. If you have access to the talent and can do it cheaply. That includes the talents to extract and analyze the relevant data in order to derive insights and value from it
  2. If you are a DaaS provider and need the data to serve your customers
  3. If you have specific questions, then all you really need is just the “right” data, which is usually not big at all!

I could not agree more and my initial findings back this right up.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Ron Cruz

Ron Cruz is a Senior Solution Architect in Perficient's Oracle Apps group. He has over 14 years of experience in CRM and data warehousing across many industries including Software, Hardware, Telecommunications, Pharmaceutical, Medical Manufacturing, and many others. In his spare time Ron spends time with his wife and kids, works on music, writing or just naps.

More from this Author

Follow Us
TwitterLinkedinFacebookYoutubeInstagram