Hot Job Prospects In Big Data

Most of you may have noticed that there is  a surge of  frequent  job openings for Big Data on job portals these days.

Seemingly, hot job prospects in Big Data!

But what is this trendy phrase “Big Data”?

Let’s explore: Big data  is extremely large amounts of structured and/or unstructured data too big for analysis in traditional databases and database management tools. This data can come from sensors, click streams, posts to social media sites, multimedia data (images, video), transactions, log files, real-time GPS data and more. IBM estimates 2.5 quintillion bytes of data are created every day from a variety of sources including sensors, social media, and mobile devices around the world.

Before delving deep inside the domain, it is better to understand what industry experts say about prospects in Big Data. Big Data is reshaping business IT. Thanks to cheap storage, massive processing power, and tools like Hadoop, organizations are now able to mine terabytes of information and derive useful business intelligence from it.

According to research by McKinsey’s Global Institute and McKinsey’s Business Technology Office, the amount of data in our world has been exploding, and analyzing large data sets—so-called big data—will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus.

IDC estimates the market for “big data” technology and services will grow at an annual rate of nearly 40 percent to reach $16.9 billion by 2015.

Is Big Data Worth Shifting a Career For?

In a recently concluded Big Data and Analytics panel at the MIT Sloan CIO symposium, following reasons have been cited for people looking for a career shift in Big Data:

  • New tools are putting structure on so-called “unstructured” data like email and tweets.
  • Big data is big business. In the US health care industry alone, the value of big data is $300 billion. Of that, $200 billion is in reduced expenses.
  • The field is hot and demands a crop of skilled workers. The gap between supply and demand for data scientists in the US could reach 190,000 by 2018. GE is investing more than $2 billion in an analytics function for its industrial services and products, and is looking to fill hundreds of analytics positions that demand skills with business, statistics and big data tools.
  • Analytics is moving out of the IT function and into research and development, into strategy, into marketing departments. Distributed analytics is on its way.
  • Although big data promises significant value, it is a business trend that remains “emergent.” There are no best practices yet, if for no other reason than that the technology is so new. Sharing data also requires significant culture change in many organizations.

(To know more about the trends, read MIT Sloan Management Review)

Jobs Options In Big Data

In a quick scan of job boards you will find job titles such as: Big Data Engineer, Data Mining Engineer or Data Mining (Big Data) Engineer.

Of late,  the title that has really become hot  is Data Scientist. However,  big data jobs can come with titles such as Business Analyst, Business Intelligence Analyst, Data Analytics Engineer, or Data Architect.

Data Scientist


The  data revolution is also creating a new breed of hybrid business IT jobs, ones that blend business knowledge and powerful IT tools to the benefit of tech-savvy line-of-business professionals – and the possible detriment of IT pros oblivious to the Big Data trend.

In addition, apart from technical and analytic skills, must not forget that there is also a need for people with a good  knowledge of big data to market and sell big data products and services.  So opportunities are there for Big Data and Hadoop Product Manager, and Big Data Sales Representative jobs.

Skills Needed To Expand Big Data Career

Big data jobs typically require a  wide range of skills – from the technical to the inferential.Many of the jobs do not require hard-core programming skills but rather require business or other job-specific knowledge, strong analytical skills, and knowledge of analytical tools.

Some specific skills that can help you expand your career in Big Data are :

  • Data mining and machine learning techniques
  • Data visualization tools
  • Data warehousing
  • ETL (extract, translate, load)
  • Hadoop (Hadoop is an Apache project to provide an open-source implementation of frameworks for reliable, scalable, distributed computing and data storage.)
  • Predictive modeling
  • Statistical modeling with tools such as R, SAS, or SPSS
  • Structured and unstructured databases

Institutions To Acquire Big Data Skills

In the US, a few colleges offer  master’s programmes in data analytics. University of North Carolina one such institution, where the Institute of Advanced Analytics has been offering a Master of Science in Analytics.

Northwestern University is among the first top universities in the US to start a master’s degree programme in business analytics. These courses will select topics from statistics, machine learning, optimisation, distributed computing and related fields to prepare specialists who can develop or manage technologies around big data.

Stanford University has started an advanced project based course. Students will work on data mining and machine learning algorithms for analyzing very large amounts of data. Both interesting big datasets as well as computational infrastructure (large MapReduce cluster) will be provided by course staff. (To know more about this course, click here)

IBM conducts training programmes on predictive and business analytics for faculty in universities, and these faculty members will be qualified to teach students how analytics can be applied to their field. IBM, which hires more data scientists than anybody else, has skill development programmes with 500 Indian universities.

EMC offers  an open course to unleash  the power of Big data.

To learn on-the-job, Big Data University is a great place to learn more about big data and to start to acquire some of the necessary skills. Many of the courses are free.

Indian universities and B-schools are also starting to offer courses in data science.The Indian Institute of Science (IISc), Bangalore is among the first to start a master’s degree programme on business analytics, and its first set of students will graduate next year.

Indian School of Business ( ISB), Hyderabad has an Asia Analytics Lab that prepares its students to analyse data. The Indian Institutes of Technology (IITs) have started similar courses. IIT-Bombay, for example, has started a part-time programme in business analytics.

Pay Package In Big Data

“Data scientists would get at least a 30% to 40% premium in compensation benefits when compared with software engineers,” says Arnab Chakraborty, Director, HP Global Analytics.

In the US, exceptional compensation packages are available in the $300,000 to $500,000 range for the appropriate technical and leadership experience Some headhunters have gotten creative, emailing reporters who cover the beat looking for recommendations.

Future Trends In Big Data

The Big Data market is on the verge of a rapid growth spurt that will see it top the $50 billion mark worldwide within the next five years.

To know more about Big Data Market Trends, read the following articles:

Adapted from following articles:

 With inputs from Dharmvir.

Speak Your Mind