Skip to main content

Data & Intelligence

10 Wishes for Hadoop’s Next 10 Years! #HS16SJ

Day 2 of Hadoop Summit 2016 ended with a big birthday bash for a big yellow elephant named Hadoop. IMG_20160629_185152246

On account of this milestone, the conference has spent a lot of time discussing what’s next for the platform and the ecosystem, here’s the top ten things on my wishlist for the next 1o years:

  1. Assemblies. Simplified deployment of complete applications.
  2. Records Management
  3. LLAP🖖. The next generation of Hive designed from the ground up for Data Warehouse Speed.
  4. Democratization of Machine Learning.
  5. Intelligent Security monitoring for the Hadoop Stack (Eagle) integrated with Intelligent Security Monitoring for the whole enterprise (Metron)
  6. A single API for data access that supports a variety of Domain Specific Query Languages and all the different storage and access paradigms
  7. Simplified integration with Enterprise Directory Servers, LDAP and KRB are mandatory.
  8. Policy manager (i.e. Ranger) is a prerequisite for all parts of the ecosystem.
  9. Containers on YARN, make any deployment scale elastically.
  10. Most of all, my wish for the big yellow elephant, is decorum in the zoo as the community, and the product, become teenagers.
Data Intelligence - The Future of Big Data
The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

Convergence toward a clear architecture for the Connected Data Platform is the definite theme. One of the biggest benefits of this is ease of communication. @benhammersly gave a great talk this morning to remind all of us that the proliferation of technology and data has not yet come with a proliferation of awareness and knowledge. We, as technologists, have a great responsibility that comes with our great power to change the world (the “Spidermanian school of politics” I think he called it…). We are responsible for ensuring that all people don’t just have the opportunity to become technologists, but that they are also given the right to understand the real impact of the technology on their lives. The gap is large, the path unclear, it is, however, the only way forward.

Here’s to you Hadoop! May the next 10 be as great as the first! (Also a big congrats to our partner Hortonworks on their 5 year anniversary!)

I’ll be back with the big conclusion to #HS16SJ tomorrow, stay tuned…

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Eric Walk, Director

Eric Walk is the Director of Enterprise Data Strategy at Perficient. As Director of Enterprise Data Strategy, he focuses on the intersection of strategy, data and technology, and business outcomes that drive growth. Eric has spent his career in consulting, taking advantage of opportunities to expand and grow. He started in Enterprise Document Management and Business Automation working with clients to modernize platforms and take advantage of the data trapped in their warehouses of virtual paper. He jumped at the opportunity to lead some early exploration of Big Data technologies with hybrid cloud architectures (Hadoop + AWS) and eventually found himself leading a segment of that practice at Perficient. Eric has since transitioned to lead Perficient’s Data Strategy capability across geographies and practices. In this capacity he serves as an advisor to executives both clients and internally on topics related to data discovery, availability, and trust. He serves as the editor-in-chief of thought leadership aligned to the firm’s Data + Intelligence pillar. Eric graduated from Vanderbilt in 2011. He holds a Bachelor of Engineering in Biomedical and Electrical Engineering with a minor in Engineering Management and currently resides in Cambridge, Massachusetts.

More from this Author

Follow Us