Hadoop has been holding the limelight in the big data world for quite a few years or more than a decade we may suppose. When a certain technology is used for such a long time a discussion about its gradual fall from relevance is inevitable. Hadoop as a software suit is going through that period currently. You will easily find hundreds of articles on the web documenting peoples’ anticipation of Hadoop’s demise. But let us be a little more thoughtful about this. Let us look at some facts and decide for ourselves whether getting a big data Hadoop certification is still a good way to get in to the analytics industry.
Is the demand really falling?
Hadoop had been shorthand for big data for a good many years. But the demand for Hadoop in organizations was pretty low in 2015. The low is used of course in respect to the high demand Hadoop tools had enjoyed earlier. A survey by Infoworld disclosed that 54% of the companies that participated in the survey had no plans of implementing Hadoop. 18% wanted to deploy Hadoop within a year or two. The point is that these stats are visibly low for a software suit that had been so happening in the past.
New Players enter the field
Hadoop is not so much a tool for analysis as it is a medium of infrastructure. And infrastructure can be replaced with new technologies. The problem never was with the functionality of Hadoop, it was really with an acute lack of trained professionals who could handle multiple Hadoop systems. The long learning curve and apparently falling demand discouraged students to take up Hadoop. It is definitely one of the simpler tools. The NoSQL tools like MongoDB and Cassandra have come into play and they do not fit into the Hadoop ecosystem; these data base systems are replacing Hbase in many cases. Spark at the other hand has removed the relevance attached to MapReduce.
Hadoop is far from being out of business though
The major Hadoop vendors are actually making a decent amount of money. They have made their peace with Apache Spark and they are also embracing other technologies that can complement the Hadoop functionalities. Hadoop is far from out of business. It can still boast of a growing market in multiple regions of the world.
It still runs the industry in India
Even though the world had come to believe in around 2015 that Hadoop cannot deliver in terms of revenue it retains its position as the de-facto industry standard in India. Hadoop has undergone certain changes for the better which have actually spurred its cause. We will get into those changes a little later in the article. The point that matters most is that there is still a strong demand for skilled Hadoop operators in India. The job portals operating in India are pretty much crowded with opportunities for Hadoop professionals. The one important change we can notice in the trend is that employers are more interested in employing candidates with multiple and varied skills sets.
They will rather employ someone with both Hadoop and NoSQL skills than ones with expertise in any one. Since Hadoop is open source and does have a wide range of top notch functionalities it is often kept as a possible option; companies are happy to have people with these skills.
The Changes that took place
- There has been a shift to online processing from batch processing – Real time analytics has grown in popularity through the years. Online processing allows real time analytics with less delays and allowance to interactive dialogues. This shift has spurred the cause of Hadoop.
- An attachment with Spark has triggered increased popularity for Hadoop. Spark evidently has a way better processing speed and accuracy than MapReduce. This shift too has been quite natural.
- In house development and deployment of Hadoop has surely pushed things in a better direction.
- Emergence of IoT and its growth has brought Hadoop very much back into the scene. When it comes to data warehousing Hadoop is still the popular choice in India.
The future
As the word goes data lake platforms are the new thing and the future. But in the Asia pacific region, Indian sub continent and some parts of America, this is still a remote idea. Big data Hadoop certification can still get you a job and earn you a pay hike. So, it is not yet time to bid farewell to good old Hadoop.
