Building data teams. Tips from Erik Bernhardsson’s journey at Spotify and beyond

On February 24 we released Episode 9 of The Right Track, an interview with Erik Bernhardsson. It was a strange day to release anything. I want to use this opportunity to send love with the wind to everyone impacted by the current affairs. 

—

I remember when I read Erik Bernhardsson’s Building a data team at a mid-stage startup: a short story. It speaks so authentically from experience and left me with a feeling of incredible trust towards the author; that they really know what they’re talking about. It's a good mix of humble vs opinionated and realistic vs visionary. In general, Erik poses thoughtful questions to the data community, and is neither afraid to share his opinions, or be convinced and change them.

Erik was early at Spotify and built out their data team. He originally joined Spotify to work on music recommendations, but ended up prioritizing more important things. He learned what so many of us data professionals learn the hard way: Before we can work on the cool stuff, we need to fix the data fundamentals; the data plumbing. He needed to address that the company’s ability to answer the “basic” questions about Spotify’s user experience was generally lacking, and it was more important to get that foundation right—like consistent event logging. Because, ultimately, data should empower better decision making for product and market strategies. This matches exactly my experience at QuizUp, and the experience of so many other data scientists that join product organizations early, where we have to make the decision to work on the boring stuff before we can get to the cool stuff. As he put it, we need to pass the marshmallow test. While it’s common that data teams get stuck on the plumbing level, Erik eventually did get to build Spotify’s core recommendation system—making all our lives better (shout out to Discover Weekly and Autoplay Similar Songs). Along the way they built Luigi, one of the first major open source workflow schedulers, which we also used at QuizUp. He later joined Better.com as CTO and employee #8, where he built a tech team of eventually 300 people—including the data team. In five years, they went from 8 people to thousands, and were recognized as #1 on LinkedIn’s top 2020 startups. Now he’s working on something new at Modal Labs, and in February announced funding from the great team at Amplify Partners. I think it’s safe to say we’re all excited for what Erik and co build for data teams.

Spotify’s org structure was an inspiration for how we structured our teams at QuizUp, including how we thought about the data team. So it was a great pleasure to have Erik on The Right Track to talk about his experience building data cultures. 

We covered a lot of ground. Including the mind-blowing experience when you try your first cohort analysis and find something unexpected, how Erik discovered survival analysis and applied it to Spotify’s user journey, why advanced machine learning should typically wait until you’ve fixed the data fundamentals and data plumbing, and how and when you should prioritize that data plumbing you can’t ignore.

We also discussed data team recruiting and org structures. Hiring the right people to your early stage data team is the most important factor for success. We talked about what to look for in those early data team roles, which he described as something like a technical data journalist. As he put it, they’re “people who, sure, they’re good with numbers, they’re ok with stats, ok in software engineering, but above all, they’re driven by a pursuit of the truth and making an impact on the business”. I’ve often talked about the importance of data journalism so I loved and related to this characteristic description. He also shared tips on how to recruit and interview for those roles.

Listen in for Erik’s thoughts on strategy when building and scaling a data team and data culture, and sit back and enjoy letting him convince you that simple things are usually far more impactful than complicated things.

For more inspiring interviews on building data cultures, visit The Right Track, and hit subscribe on your favorite podcast streaming service to make sure you don’t miss out on new releases.