Big Data is big deal. Every vendor has a strategy and a suite of products. Navigating the maze and picking the right Big Data platform and tools takes some level of planning and looking beyond techie’s dream product suite. Compounding the issue is the open source option vs. going with a vendor version of the open source. Like every other new technology, product shakedowns will happen sooner or later. So picking a suite now is like betting on the stock market, exercising caution and being conservative with long-term outlook will pay off.
Organizations tend to follow the safe route of sticking with the big vendor strategy but the downside is getting the funding and putting up with the procurement phase of waiting forever for the approval. The hard part is knowing the product landscape, assessing the strengths of each type of solution and prioritizing the short-term and long-term strategy.
I have seen smaller companies building their entire solution in open stack and don’t pay a penny for the software. 
Obviously the risk and the rewards plays out. Training the resources and hiring trained resources from the market place is a huge factor as well. Open source still has the same issues of version, bugs and compatibility, so having the knowledgeable team makes a big difference in managing the environment and the overall quality of the delivery.
But despite the confusion, there is good news. If you are in the process of figuring out how you want to play the Big Data game, big and small vendors alike are providing you with the sandbox or Dev environment almost free or for limited duration. Leveraging this option as part of the Big Data strategy will not only save money but also the learning curve. IBM Bluemix is an example of that. So does Cloudera, Datastax and the list is growing.
To maximize the benefit, follow the basic portfolio management strategy.
- Take an inventory of tools already available within the organization
- Identify the products which will play better with the existing tools
- Figure out the business case and the types of tools needed to get a successful POC
- Match the product selection with resource knowledge base
- Get as much help from external sources (a lot of them can be free, if you have the time) from training to POC
- Start small and use it to get the buy in for the larger project
- Invest in developing the strategy with POC to uncover the benefits and to build strong business case
Combining this strategy with little bit external help to narrow down the selection and avoiding the pitfalls based on the industry experience will add tremendous value in navigating the complex selection process. Time to market can be drastically cut down especially when you make use of the DevOps platform on the cloud.
The direct benefits in leveraging the try-before-buy options are:
- No Hardware / wait time or IT involvement for setting up the environment
- All the tools are available and ready to test
- Pricing and the product stack can be validated rather than finding out later that you need to buy one more product which is not in the budget
- Time to market is drastically cut down
- Initial POC and Business Case can be built with solid proof
- Throwaway work can be minimized
Looking at the all the benefits, it is worth taking this approach especially if you are in the initial stages and you want proof before asking for the millions which is hard to justify.
The payoff from joining the big-data and advanced-analytics management revolution is no longer in doubt. The output of modeling may be strikingly rich, but it’s valuable only if managers and, in many cases, frontline employees understand and use it. What’s needed are intuitive tools that integrate data into day-to-day processes and translate modeling outputs into tangible business actions like ViusualVue.When a plan is in place, execution becomes easier.So data visualization tools like this will help you make more informed decisions.