Skip to main content

Data & Intelligence

Top 5 Lessons of Day 1 at Hadoop Summit #HS16SJ

Perficient is at the Hadoop Summit in San Jose, CA and we’re tracking the best of the conference. Here’s the top 5 lessons from day 1:

  1. Apache Atlas for managing your business catalog is almost ready for prime time! It is not, however, ready to be a full fledged Records Management solution (no policy management, no proper disposition automation, although expiry is there… stay tuned).
  2. Microsoft ‘s Big Data and intelligence suite centered around HDInsight, MS R Server (formerly Revolution) and Power BI has a name, “Cortana Intelligence Suite” and they’ve got a lot to say about the future of the Connected Data Platform. Hortonworks is planning to prioritize HDInsight go-forward as their Premier Cloud Solution.
  3. Assemblies are the future. The next grand evolution in cluster management is coming soon (how soon is as yet unknown). We’ll see the marriage of Slider (long running apps on YARN) and Docker (containerized apps). This will enable the deployment of end-to-end data apps with only a few clicks into a YARN+HDFS cluster. The demo showed Credit Fraud Detection, Log Search and Metron. You will be able to deploy all parts of a logical App (eg. for Log Search, that’s Zookeeper, Solr and a UI) with ease through an Ambari integrated Assembly Store.
  4. Metron is going to be big. With early adopters seeing ROI for security threat detection less than two months from go-live. This is one of the great use-cases for the whole ecosystem, Storm, Spark+ML, Solr/Elastic, Hive, HBase and NiFi. I’m hearing stories about 6-8 months to implement, 3-6 weeks to see results, it’s truly remarkable and coming soon to the mainstream.
  5. Business Value. The ecosystem is trying to pivot, as is the conference, to tell the story of big data in terms of business value to business users. They will be the drivers of new innovation and items 1, 3 and 4 on my list are all signs of this change.
Data Intelligence - The Future of Big Data
The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

We’ll be following up on “Big Data and You (the enterprise IT Leader)” with more thoughts on Data Governance, Records Management and Metron. I’ll be back tomorrow with more from the conference.

BONUS: Check this out… Big Data is going to try and cure cancer through genomic research!


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Eric Walk, Director

10 years of experience, specializing in system architecture and planning, platform installations and upgrades, and system administrator training for a wide array of Content, Process and Data platforms. View My Certifications

More from this Author

Follow Us