Think about the last time you used a vending machine. Did your snack of choice get stuck? Did the machine throw out weird error codes? If you were hit with barriers, odds are you felt frustrated, helpless, and hangry.
This exact thing happened to Kameron Canbaz, Principal Product Manager at Yahoo DSP. A failed vending machine interaction (he really wanted those Donnettes) got him thinking about how friction affects user journeys, and launched Yahoo DSP on a transformative product journey with Pendo Analytics.
Canbaz works on Yahoo’s demand side platform (DSP), which helps companies buy advertisements. Rather than going to individual websites and buying one ad at a time, advertisers input their budget and goals into Yahoo DSP, which finds the best ad placements to purchase at the right time.
Meeting user needs drives product growth
One of the most important activities users do in Yahoo DSP is executing and managing campaigns—it’s how advertisers get their ads in front of their target audiences and how Yahoo DSP earns revenue. As with any complex platform, Yahoo DSP occasionally experienced challenges that had the potential to impact customers if not promptly addressed.
The product team was proactively looking for ways to quickly identify and address these challenges to ensure a great experience for advertisers. They wanted a solution that empowered them to identify and resolve problems quickly, not use too many resources, and craft a great customer experience. Pendo Analytics equipped them with the tools to gather a wealth of data, enabling valuable insights into optimizing performance.
Building a foundation with democratized data
Canbaz created a simple three-step framework to guide the product team as they worked on addressing customer challenges:
- Quantify the problem with data
- Plan and execute a strategy
- Measure, modify, and repeat
Before using Pendo, the product team would have to rely on the engineering team to mine data and build complex dashboards, which could take multiple days or weeks depending on the request. But with his knowledge as an avid Pendo user, Canbaz knew that Track Events could provide the “missing link” for getting the data he needed without pulling engineers away from critical work.
An added benefit of using Pendo is that the data is democratized. Every product manager had access to the data they needed to run reports and make informed decisions.
Pendo Analytics uncovered two key areas where the user experience could be enhanced— page performance and error handling. Canbaz went deep into each problem to find how they could make strategic updates to quickly improve the customer experience.
Using Track Events to pinpoint performance improvement opportunities
To work through the first challenge, Canbaz’s team placed a Track Event on every page of the platform, focusing on the data concerning three areas:
- Current user path
- Previous user path
- Total page load time
These data points gave Canbaz the information he needed to determine what to prioritize to make the most impact. He gathered detailed metrics using the Pages section of Pendo to review the slowest pages and utilized Data Explorer to view their load times. He prioritized updating the top 10 pages with load times over five seconds.
Deep dive into error messages with single-page data
To tackle the errors in the platform, Canbaz again used Track Events. Looking at errors through the lens of Track Events allowed Canbaz’s team to see and quantify what their users were struggling with in the platform. He set a Track Event to fire whenever an error message occurred within the platform. He tracked four things:
- Error type—whether UI or API
- Error source—how the user experiences the error
- Error message presented—did it make sense for the user
- The current and previous path—where was the user in their journey when the error occurred
First, Canbaz used a global errors report to find the pages with the most errors. Then, he dug deeper into each page to look for any patterns. By looking at data on a single-page level, Canbaz could determine if there were notable trends with the types of visitors or accounts receiving the error message. He also could look at the error message to determine if it was unclear or confusing.
An example he saw within the platform was very similar to his vending machine debacle, where the error code he received gave zero context to the actual problem. The error message users received didn’t explain what they were doing wrong or how they could fix the problem. They would end up attempting a workaround or abandoning the action entirely. Understanding the root cause of the error message required knowledge of the user and the timeframe where the error occurred.
Leveraging reports to create fast insights and improve performance
With all the data in hand, Canbaz needed to deliver the right data to the engineering team to concentrate their efforts. To enhance performance in the platform, running reports to pinpoint specific pages gave engineering a clear place to start. As engineers implemented improvements, the load times for the slowest pages got shorter. The product team continued to measure and monitor load times to ensure pages continued to get faster and no new slow pages appeared.
To fix error messages, the product team created two reports—one to find the error on a specific day and a second to narrow that error down to a specific timeframe. This level of specificity provided the engineers with the information they needed to tackle the problem quickly.
“Getting error occurrences narrowed down to a specific timeframe turned out to be the most helpful data that we could provide to engineers,” Canbaz said. “That data allows them to pinpoint that error within our own internal systems.”
Platform improvements = happier customers
After initial implementation from engineering, the top ten slowest pages had their load times shortened by up to 80%. Customers started talking about how ‘snappy’ the platform felt. “For anyone that works on a B2B product, you know how challenging it can be to get unsolicited positive feedback about your products,” notes Canbaz.
Their error occurrences were drastically reduced, with Canbaz saying, “What used to be a lot of errors happening at once went to effectively zero.” He continues to roll out the error reporting process to more and more product managers within the organization to decrease errors within the DSP.
Measure, modify, repeat: Using Pendo to reduce risk
This process was not a one-and-done fix. Using Pendo data, Canbaz continuously tracks performance and error data. The power of Pendo data reduces the back and forth between product and engineering, streamlining their processes and speeding time to action.
Canbaz built the reports to be easily replicable so any product manager can simply adjust filters as needed to find the data they’re looking for and package for engineering. Pendo continues to help Canbaz and team deliver data, action improvements, and monitor success.
“Using Pendo allowed us to get a quick handle on the full scope of the problem,” explains Canbaz. “When we develop fixes to the problems, we continue to follow our framework and adjust as needed over time to ensure ongoing progress. This equates to more time for us to spend on continuing to make our platform incredibly user-friendly.”
