Report- The State of Data Engineering

I recently came across an excellent report by Stitch on the state of data engineering.

Today, there are 6,500 people on LinkedIn who call themselves data engineers. In San Francisco alone, there are 6,600 job listings for this same title. The number of data engineers has doubled in the past year, but engineering leaders still find themselves faced with a significant shortage of data engineering talent.

The need for data talent is born from a fundamental shift: tech companies are now data companies. Uber, AirBnB, Spotify–these companies build data products, and as a result, are scrambling to hire (and hold onto) the people that build and maintain data systems. Josh Wills, Data Engineer at Slack, half-joked, half-pleaded at DataEngConf 2016, “Please don’t hire my data engineers, they are all here now.” Even Slack, one of the hottest tech companies in the valley, is worried about holding onto this valuable talent.

This report takes an in-depth look at the following:

• The number of data engineers in the market today
• Their backgrounds and core skills—information that is particularly valuable for leaders thinking about how to transition software engineers into data engineering roles.
• Employment information that can help make the case for investing in this often expensive skill set.

Answers to these questions are paired with input from engineering leaders at Stripe, MIT, Looker, and more; who share their strategies for finding and retaining talent, developing data engineering talent in-house, and prioritizing a data engineering team’s projects. This report presents a clear snapshot of the current state of data engineering.

View the full report

_{*Source: Stitch}