Big Data is one of the enabling technologies for companies to digitally transform either their operations and/or customer interactions. However the open source world can be complicated, especially in the red hot Big Data arena. There are a myriad of technologies; some compete with one another, others overlap, some are complementary, and worse of all, some technologies both compete and are complementary (e.g. Cassandra can stand alone or run on top of Hadoop, thus it competes, and it is complementary.)
Like a Venn diagram, both Spark and MapReduce can coexist, but a number of their use cases do overlap. (Read more about Spark vs. Hadoop here). On Monday IBM, announced that it was throwing its weight behind Spark (see my blog post here) with little mention of Hadoop. It is easy to see how companies can one can feel like Dorothy in the Land of Oz. However, following a few simple recommendations a company can make good decisions in this seeming fantastical world (view my blog post here).
There is more good news. In this new digital, Big Data ecosystem the technology is pretty pluggable and is becoming more interchangeable. If the focus is on business value that data and analytics can bring to a company undergoing a digital transformation, then a good pragmatic business decision today will result in a good return on your technology investment.