How Pendo uses Data Sync to unify teams with product data

Published Aug 21, 2023
John (a Data Engineer at Pendo) will never forget the day he saw the Slack message from Lead Product Manager, Dana: “Hey, Data Sync is ready—let’s get it up and running.”

This was a huge moment for Dana and John. They knew how much of a win Data Sync would be for their teams—and for all of Pendo and their customers. 

You see, as a data engineer overseeing Pendo’s internal data pipeline, it’s John’s job to pull data from Pendo—along with all other business-critical data sources—into Pendo’s data warehouse (BigQuery). That data is then queried and eventually surfaced in Pendo’s business intelligence (BI) tool, for the entire organization to analyze. This was great news for Dana, too. Data Sync would allow her team’s hard work to be fully represented in the business’ analyses. So now, the data that drives her team’s success can be seen by—and help support—other teams, too.

“Heck yes!”John replied. Memories of Slack conversations can grow hazy over time. In truth, his response may have been even more emphatic (and slightly more crass). He knew that Data Sync would be a huge upgrade over his current solution for pulling Pendo data into BigQuery. 

The existing pipeline relied on Pendo’s Aggregations API, which is a great solution for asking very specific questions of your Pendo data, but less effective in systematically pulling out data in bulk. Data from the API is always aggregated to the hour—making it easy to query, but losing the granularity that many teams want with event data in their warehouse. With Data Sync, John knew he would gain visibility into every single event, timestamped to the millisecond—making it great for answering any question. Plus, Data Sync transfers the data into Avro files (i.e. they’re flexible and easy for all teams to utilize), and the ETL data transfer process that connects the “data pipes,” is super easy to establish. Win, win, win.

It was then that Shelby, a data analyst at Pendo, saw the same Slack message. Already, her mind was racing with possibilities. 

On her roadmap from Pendo’s product-led growth (PLG) team was a big ask for full visibility into the end-to-end user journey—starting with top-of-the-funnel marketing content engagement, leading to in-app usage in Pendo, and culminating in upgrades from Pendo Free to paid plans. She’d had some ideas on how to unleash this level of visibility, but thought that the aggregated data she had access to was something she needed to work around. 

With Data Sync, Shelby knew she could unify and pinpoint the insights she needed, allowing her to see every event in the warehouse and stitch together a full user journey—combining marketing events from the team’s marketing automation tool, in-app events from Pendo, and revenue data from the CRM. 

Shelby also knew from previous conversations that Data Sync would support retroactive processing, right out of the box—which means she can instantly access data from as far back as the data’s earliest export date. This was previously a constant thorn in her side, when comparing data in the warehouse (pulled via the API) to data directly in Pendo. Previously, any time a page or feature was added or updated, Pendo automatically applied that update to all of the historical data in Pendo.

Now, with Data Sync, updated data files are sent by Pendo automatically, ensuring that the data in Pendo and the data in the warehouse is always synced up. Music to Shelby’s ears!

The end-to-end user journey wasn’t the only thing on her analytics roadmap that Data Sync could help with. Her team had been working on an AI-powered churn risk model during a hackathon project. Getting the full breadth of usage data into the warehouse as a source would ensure that her team was working with the highest fidelity usage and sentiment data imaginable—ensuring their AI-powered decision-making has the best input data available.

After just a few weeks using Data Sync, Dana was being looped into more strategic conversations with leadership to determine where the company needed to invest next. John had built a shiny new data pipeline—which was getting updates every day with new and historical data. And Shelby was demoing the end-to-end user acquisition funnel to executive leaders—giving the team deep insight into what was working and what could be improved.

To learn more about Data Sync and see how it can help your team unlock the full value of your product data, check out this page.

Pendo implemented their ETL and open sourced it.