Skip to main content

Data & Intelligence

Data Lakes, Not Just For Analytics Anymore

Data Lakes have been around since the early part of this decade as most Fortune 500 companies have a Data Lake or are building a Data Lake. The drive to lake data has predominately been driven by analytical use cases where Data Scientists can wrangle and prepare data for their study or model building.

Data Intelligence - The Future of Big Data
The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

However, Perficient is seeing a significant shift from just deploying Hadoop to support analytics use case to using Data Lakes for operational data processing use cases.   Companies are now able to move processing from expensive legacy mainframes and MPP data warehouses to Hadoop and Cloud-based systems. Although operational data processing has always been possible on Hadoop systems, the momentum has significantly accelerated due to a number of advances in in past few years.

These advances include:

  1. SQL-based transformation included in Spark has made Big Data ETL accessible to many firms not wishing to invest in expensive ETL tools,
  2. Cloud-based Data Warehouses that can offer similar scale and ease of use as traditional EDW systems at a fraction of the cost, and
  3. Security advancements of Hadoop and Cloud-based Big Data Offerings has reassured companies that their data assets are protected in the new data ecosystem.

The movement to perform more operational data processing on Data Lakes brings its own set of challenges that companies need to address. In my next blog post, I will investigate these challenges that companies are facing as Big Data becomes operational.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Bill Busch

Bill is a Director and Senior Data Strategist leading Perficient's Big Data Team. Over his 27 years of professional experience he has helped organizations transform their data management, analytics, and governance tools and practices. As a veteran in analytics, Big Data, data architecture and information governance, he advises executives and enterprise architects on the latest pragmatic information management strategies. He is keenly aware of how to advise and lead companies through developing data strategies, formulating actionable roadmaps, and delivering high-impact solutions. As one of Perficient’s prime thought leaders for Big Data, he provides the visionary direction for Perficient’s Big Data capability development and has led many of our clients largest Data and Cloud transformation programs. Bill is an active blogger and can be followed on Twitter @bigdata73.

More from this Author

Follow Us