This is unmistakably the world of the “data scientist” and the “analytics guru”!
As growth of data outpaces our ability to process and even understand it in its totality, newer technologies and roles are emerging for handling the challenges posed by “big data”. More and more Retail, Financial, Insurance and Social media giants are treating data and business analytics as a computational science. More and more emphasis is on “in-memory” applications, private and public data clouds, and lots of innovation around big data and low-latency, real-time analytics.
For instance, consider these emerging BI scenarios:
- Retailers are increasingly focusing on “just-in-time” Business Intelligence.
- Huge numbers of corporations use Big Data technologies such as Hadoop and cloud computing to analyze massive amounts of application and transactional data.
- Emergence of public and private data cloud start-ups capable of handling petascale problems is defining the new BI world.
- It is amazing how data growth these days is being talked about more and more in terms of Petabytes, Exabytes, and Geopbytes!
Tremendous scale of innovation is taking place around next generation DW, in-memory analytics, and cloud computing. Therefore, if you have not heard of these new technologies – Data Appliances (SAP HANA), Hadoop, Cloud Databases– then it’s time to play catch-up. These all represent the next Generation in Data and Business Intelligence!
Some of the increasingly popular names in the Big Data space are:
SAP HANA: “This data appliance enables organizations to analyze business operations – based on large volumes of transactional and analytical data – and to instantly explore and analyze the data from virtually any data source in real time.” Data is primarily captured “in memory” as business unfolds, and flexible views expose analytic information rapidly.
Hadoop: Designed to process terabytes and even petabytes of unstructured and structured data. It breaks large workloads into smaller data blocks that are distributed across a cluster of commodity hardware for faster processing.
Columnar Databases: Columnar querying brings about performance efficiencies, and is unmatched by any row-oriented DB.
The companies to watch out for in this space are SAP, HP, Teradata, IBM, Splunk and Nothscale, just to name a few.
I agree with what you say about tremendous scale of innovation taking place around next generation DW, in-memory analytics, and cloud computing. We are taking the cloud computing route with bime (http://bimeanalytics.com), a BI software for the web, and another company to keep an eye on in this space.
Thank you for your comment, Kirsty. I will definitely check it out!
Hi Neetu,
Thanks for this article. In memory data processing is really intersting and fast in our traditional read-only data cache systems. But for this high volume transaction data, how will these products synchronize the data between in-memory and physical disk in a clusted environment ? I would like to know about these concepts and please provide some more detials.
Hi Bikram,
Unfortunately, I don’t have all of the details yet. This is actually on my plate to research further. I will be sure to share the info when I have it all figured out. Until then, if there is anything that you’ve got to share, please do so by responsing to this post!
Keep the discussion flowing…
Thanks & Warm Regards,
Neetu
Very informative. Do you have more details on SAP HANA?
Would definitely appreciate more info on SAP HANA
Very informative.
I am in the process of researching companies that have deployed SAP HANA for BETA purposes. I will be sure to share the results on this blog!
Excellent site you have here but I was curious about if you knew of any user discussion forums that cover the same topics discussed in this article? I’d really love to be a part of group where I can get feedback from other experienced individuals that share the same interest. If you have any suggestions, please let me know. Appreciate it!