Data & Intelligence

10 Wishes for Hadoop’s Next 10 Years! #HS16SJ

Day 2 of Hadoop Summit 2016 ended with a big birthday bash for a big yellow elephant named Hadoop. IMG_20160629_185152246

On account of this milestone, the conference has spent a lot of time discussing what’s next for the platform and the ecosystem, here’s the top ten things on my wishlist for the next 1o years:

  1. Assemblies. Simplified deployment of complete applications.
  2. Records Management
  3. LLAP🖖. The next generation of Hive designed from the ground up for Data Warehouse Speed.
  4. Democratization of Machine Learning.
  5. Intelligent Security monitoring for the Hadoop Stack (Eagle) integrated with Intelligent Security Monitoring for the whole enterprise (Metron)
  6. A single API for data access that supports a variety of Domain Specific Query Languages and all the different storage and access paradigms
  7. Simplified integration with Enterprise Directory Servers, LDAP and KRB are mandatory.
  8. Policy manager (i.e. Ranger) is a prerequisite for all parts of the ecosystem.
  9. Containers on YARN, make any deployment scale elastically.
  10. Most of all, my wish for the big yellow elephant, is decorum in the zoo as the community, and the product, become teenagers.
Data Intelligence - The Future of Big Data
The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

Convergence toward a clear architecture for the Connected Data Platform is the definite theme. One of the biggest benefits of this is ease of communication. @benhammersly gave a great talk this morning to remind all of us that the proliferation of technology and data has not yet come with a proliferation of awareness and knowledge. We, as technologists, have a great responsibility that comes with our great power to change the world (the “Spidermanian school of politics” I think he called it…). We are responsible for ensuring that all people don’t just have the opportunity to become technologists, but that they are also given the right to understand the real impact of the technology on their lives. The gap is large, the path unclear, it is, however, the only way forward.

Here’s to you Hadoop! May the next 10 be as great as the first! (Also a big congrats to our partner Hortonworks on their 5 year anniversary!)

I’ll be back with the big conclusion to #HS16SJ tomorrow, stay tuned…

About the Author

10 years of experience, specializing in system architecture and planning, platform installations and upgrades, and system administrator training for a wide array of Content, Process and Data platforms.

More from this Author

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Subscribe to the Weekly Blog Digest:

Sign Up