Quora Question: How Will Data Science Change in the Next 5 Years?

Quora Questions are part of a partnership between Newsweek and Quora, through which we'll be posting relevant and interesting answers from Quora contributors throughout the week. Read more about the partnership here.

Answer from Anthony Goldbloom, co-founder and CEO of Kaggle:

How will data science change in the next five years? In answering this question, I'm going to focus less on what I expect to happen at the cutting edge of data science and more on how data science continues its progression toward becoming mainstream and ubiquitous.

When thinking about where data science is going in the next five years, it's useful to reflect on how data science has evolved over the past five years. When Kaggle started in 2010, the term 'data science' wasn't common yet. Members of our community referred to themselves as doing advanced analytics, statistics, machine learning, bioinformatics, econometrics or one of the various other disciplines that involve working with data and statistical techniques. Companies also referred to the departments that did data-related work by their functions: marketing analytics, risk, underwriting, chemical informatics etc.

The phrase data science really took off after O'Reilly's Strata Conference in 2011. That conference brought 1,500 "data scientists" together. It gave individuals with different job titles a single way to refer to their skill-set. It told senior management that data professionals in different departments actually have the same skill-set.

So if O'Reilly's Strata conference was the first innings, I believe we've now moved into the second innings. We're now seeing many companies consolidating their data scientists into a single, large data science organization. The most effective structures involve the data science organization seconding data scientists out to the business units (marketing, risk etc). This structure works well because the data science organization learns how to attract and recruit data science teams, but allows data scientists to work closely with those who have context on the problems they're working on. Airbnb is a great example of a company using this structure effectively.

As companies derive more value out of their existing data science teams, those teams will continue to grow. Ultimately I think the central data science organization goes away and each business unit will have large dedicated data science teams.

Data science is really succeeding when it becomes the primary decision making tool inside organizations. When there's a decision to be made and management's first instinct is to ask "what does data science say?"

Tackling this question from a different direction, I believe data science will be bigger than software engineering in the next decade. If we define a data scientist as somebody using R or the Python Data tools, there are probably 1.5 million to 3 million* data scientists in the world (compared with 20 million software engineers). Meanwhile, there are ~8 million SAS users and ~120 million Excel users. I believe that SAS slowly declines and SAS-heavy jobs adopt R and Python. That many jobs that require heavy Excel use also switch to using R and Python.

*Triangulating around Kaggle's userbase (650K) and Jupyter Project users (they estimate 3MM).

How will data science change in the next 5 years? originally appeared on Quora - the knowledge sharing network where compelling questions are answered by people with unique insights. You can follow Quora on Twitter, Facebook, and Google+. More questions:

• Data Scientists: What tools can make data scientists more productive?
• Deep Learning: Will deep learning make other machine learning methods obsolete?
• Jobs and Careers in Data Science: What are the best ways for data scientists to collaborate on work?