Patreon gets better team alignment, efficiency, and fewer analytics implementation errors with Avo

"With shelter-in-place, Patreon has become a great source of income for 140,000 creators. But as we’ve scaled, with every new release we spent up to 4 days implementing or fixing analytics. Now Avo ensures we’re implementing in less than an hour, capturing our metrics accurately, and collaborating on our tracking in real-time. What has historically been a source of friction across multiple teams is now seamless.”

Jason Byttow
‍Engineering Manager

Patreon is an international platform used by artists ranging from writers to musicians, gaming creators, and Youtube content creators. It provides membership tools for artists that help them obtain funding from their fans, or patrons, and to make a sustainable and predictable living off of their creative output. Patreon can be accessed on mobile apps (iOS), Android, regular mobile), as well as a desktop. With its mission to fund the creative class, Patreon aims to view the world through the lens of creators, shift culture, and the way art is funded.

Maura Church leads Patreon’s full-stack data science team, which works across the company to support its product analytics, business analytics, core research and machine learning, and business intelligence functions. Maura’s role is to ensure that data is available, acceptable, accurate, and high-quality. One of their biggest challenges is making sure that they are measuring the right things, have the right metrics, and that the metrics are being used by others to improve internal decision-making and their product.

Jason Byttow manages engineers on two of Patreon’s product teams as well as their infrastructure team. Focusing on the intersection of people, technology, team orchestration, and building, Jason is continuously looking for ways to make their tech stack easier to navigate and work in for their developers. His teams work closely along with Patreon’s data scientists to ensure that analytics are correctly implemented and running and that their workflow runs smoothly.

Patreon’s tech stack is React on the front-end (JavaScript) and Python on the backend. The team uses Python for statistical analysis, Amplitude for behavioral analytics, and it powers most of its analysis with a Redshift database. Maura’s data science team also uses Tableau and Mode.

As Patreon’s technology and product have scaled and grown in complexity, the process of orchestrating across teams and platforms has become more error-prone. The team looked to Avo to help them consolidate and standardize their analytics tracking, make event implementation efficient for their engineering and data science teams, and support data democratization across the company.

Easier and efficient analytics implementation

One of the challenges Patreon faced was an inefficient workflow when implementing events. Prior to adopting Avo, if a new feature was ready to ship and needed to be tracked, there might be several rounds of feedback between their developers and data scientists, and errors that needed to be re-worked or fixed. This slowed everyone down and prevented teams from getting clean analytics.

“Now, a data scientist can open up a branch in Avo, send that branch to a developer, who can pull in those changes, Avo will validate them, and they can be QA’ed from there,” Maura said.

“Overall, people are excited that Avo removes what tends to be a very error prone interplay between engineering and data science and launching, and does it through a simple UI, “ Jason added.

With fewer instrumentation errors and greater efficiency and alignment, Maura and Jason’s teams can ship faster and focus their efforts elsewhere.

“We’re very data and experiment-oriented. Providing tools that enable our engineers to easily implement event tracking with confidence, and ensuring that we have a good relationship with data science throughout that process, is one of my core focuses as an engineering manager.”

Jason Byttow
‍Engineering Manager

Standardized events and properties

According to Maura, Avo has helped Patreon standardize their event workflows and validate their taxonomy, which consists of more than 1,800 events and many shared properties across their platforms and products. Previously, there might have been two events that shared the same event property, but one event property was sent as a boolean and in another place as a string.

Standardizing event names and properties is now making it easier for team members to get started with analytics, and it alleviates hassle and the back-and-forth needed between data scientists and engineers when implementing their tracking code.

"Instrumentation process used to take 1-4 days but now take 30-120 minutes.

Maura Church
‍Director of Data Science

Education and data democratization

For Patreon, providing teams across the company with access to the data and the context needed so that data is used responsibly is a high priority. Avo facilitates this effort. Prior to adopting Avo, Maura’s team was spending time every week educating product managers, designers, and developers about analytics, event samples, what they mean, where they fire from, and what their properties are.

Now, Avo can serve as that source of education for Patreon’s teams, whether that’s through a pop-up modal that a product manager sees when clicking on an event to know exactly what’s firing, or viewing the list of all events with better descriptions and comments. “If it’s someone’s first week at Patreon and they’re interested in user behavior, they can look at Avo and see what’s possible for them to analyze,” said Maura.

With more accurate data available in context and accessible across their teams, decisions can be more easily made by everyone. “The more that the basic questions are democratized and answerable by everyone, the faster the company is learning,” Maura said.

PMs are no longer afraid they will slow down product development by asking devs to add analytics.

“The more the basic questions are democratized and answerable by everyone, the faster the company is learning.”

Maura Church
Director of Data Science

Recommendations

Both Maura and Jason recommend that other teams think ahead when it comes to good behavioral analytics instrumentation and start as early as possible to avoid challenges later on as a company or product grows in complexity.

“Whether you’re a data scientist or an engineer or even a product manager, you should be thinking about this problem early on, because it’s only going to get more difficult as you scale,“ Jason said. “You don’t know how much you’re missing out of in terms of the speed and value of insights if you have an unstandardized way of tracking things.”

Maura added, “It can feel very scary to be strict about standards and meaning, but what I’ve learned is that it’s so much faster to have insights when you have clean data.”

“It’s so much easier to have clean insights when you have clean data. ”

Maura Church
Director of Data Science