Skip to main content

Data & Intelligence

Top 5 Lessons of Day 1 at Hadoop Summit #HS16SJ

by on June 29th, 2016 | ~ minute read

Perficient is at the Hadoop Summit in San Jose, CA and we’re tracking the best of the conference. Here’s the top 5 lessons from day 1:

  1. Apache Atlas for managing your business catalog is almost ready for prime time! It is not, however, ready to be a full fledged Records Management solution (no policy management, no proper disposition automation, although expiry is there… stay tuned).
  2. Microsoft ‘s Big Data and intelligence suite centered around HDInsight, MS R Server (formerly Revolution) and Power BI has a name, “Cortana Intelligence Suite” and they’ve got a lot to say about the future of the Connected Data Platform. Hortonworks is planning to prioritize HDInsight go-forward as their Premier Cloud Solution.
  3. Assemblies are the future. The next grand evolution in cluster management is coming soon (how soon is as yet unknown). We’ll see the marriage of Slider (long running apps on YARN) and Docker (containerized apps). This will enable the deployment of end-to-end data apps with only a few clicks into a YARN+HDFS cluster. The demo showed Credit Fraud Detection, Log Search and Metron. You will be able to deploy all parts of a logical App (eg. for Log Search, that’s Zookeeper, Solr and a UI) with ease through an Ambari integrated Assembly Store.
  4. Metron is going to be big. With early adopters seeing ROI for security threat detection less than two months from go-live. This is one of the great use-cases for the whole ecosystem, Storm, Spark+ML, Solr/Elastic, Hive, HBase and NiFi. I’m hearing stories about 6-8 months to implement, 3-6 weeks to see results, it’s truly remarkable and coming soon to the mainstream.
  5. Business Value. The ecosystem is trying to pivot, as is the conference, to tell the story of big data in terms of business value to business users. They will be the drivers of new innovation and items 1, 3 and 4 on my list are all signs of this change.
Data Intelligence - The Future of Big Data
The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

We’ll be following up on “Big Data and You (the enterprise IT Leader)” with more thoughts on Data Governance, Records Management and Metron. I’ll be back tomorrow with more from the conference.

BONUS: Check this out… Big Data is going to try and cure cancer through genomic research!

IMG_20160628_094539

Tags

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Eric Walk, Director

Eric Walk is the Principal for Enterprise Data Strategy at Perficient. He focuses on the intersection of strategy, data and technology, and business outcomes that drive growth. Eric has spent his career in consulting, taking advantage of opportunities to expand and grow. He started in Enterprise Document Management and Business Automation working with clients to modernize platforms and take advantage of the data trapped in their warehouses of virtual paper. He jumped at the opportunity to lead some early exploration of Big Data technologies with hybrid cloud architectures (Hadoop + AWS) and eventually found himself leading a segment of that practice at Perficient. Eric has since transitioned to lead Perficient’s Data Strategy capability across geographies and practices. In this capacity he serves as an advisor to executives both clients and internally on topics related to data discovery, availability, and trust. He serves as the editor-in-chief of thought leadership aligned to the firm’s Data + Intelligence pillar. Eric graduated from Vanderbilt in 2011. He holds a Bachelor of Engineering in Biomedical and Electrical Engineering with a minor in Engineering Management and currently resides in Cambridge, Massachusetts.

More from this Author

Categories
Follow Us