“The level of success is limited only by our imagination…”

This adage applies aptly to the world of Big Data and Analytics implementations. Having found early success in specific Big Data implementations applied to limited set of data analytics use cases, companies stick to the tried and tested problems only. They are often blindsided by the secondary benefits of the tool, so much so that the real value of Big Data is left behind. The processes governing the use of the tool are often departmentalized with information silos that restricts itself to answering same set of business questions.

The emergence of new job roles to sustain the information silos, for ex. Apache Hive Specialist or Apache Kafka Consultant, has blurred the vision of broad analytics ecosystems where every piece of data is tied up together.

Fig. Big Data Ecosystem

The premise on which Big Data ecosystem came into being was the unrestricted access to all raw data for unconstrained exploratory analysis.  The Big Data concept helped organizations to answer business questions that were never answered before and to improve the reliability of answers derived from limited data in the past. Traditionally, the BI Tools were used to act as a ‘rear-view’ mirror – providing insights to events which happened in the past. With the ever-growing inclusion of new tools in the Big Data ecosystem, there are no tool barriers or technological constraints to limit the businesses from having an insight into the future.

This will only be possible if we limit our limitations on tools and processes for Big Data use. We need to allow users to access any data and apply any form of analytics using any of the tools of their liking. This approach will help decision-makers gain new insights and foster creativity, enabling them to realize the true potential of Big Data.