How IHS Markit uses Product Engagement Score to make more outcomes-based product decisions

As intelligence firm IHS Markit’s Associate Director of User Experience, Ellie O’Connor is responsible for conducting deep product analysis and communicating customer feedback to the company’s product management team across a portfolio of products.

When working on one of those products, Engineering Workbench, O’Connor often needs to combine quantitative and qualitative data to tell a story that helps PMs prioritize where to invest resources into building new features or improving parts of the product. That’s where Pendo comes in.

One metric to rule them all

O’Connor has been using Pendo for some time, but the platform’s new Product Engagement Score (PES) has been a game-changer. PES is a composite metric that combines adoption, stickiness, and growth to provide a holistic, at-a-glance overview of product health. 

Prior to PES, O’Connor’s team would come up with a long list of different metrics and try to patch them together to paint some sort of picture of product health. “I would have to cobble together a whole slew of metrics like stickiness, feature usage, and others on a monthly and quarterly basis,” she said. PES packages all of that together neatly and reports it as a single metric on a rolling basis.

“PES gave me a calculation and a formula that I could really sink my teeth into quickly! What we found really valuable was the combination of the three different metrics that speak to multiple teams’ needs and that indicate product success or product friction.”

She was finally able to discard all those cumbersome spreadsheets for good.

Measuring what actually matters

The Product Engagement Score has been a crucial tool for O’Connor, but implementing it required deep thought and cross-team collaboration between research, product, and customer success.

To identify the data that would be fed into the PES formula, the teams had to work together to define the key workflows of IHS Markit’s customers and identify the most important behaviors in Engineering Workbench that led to successful user outcomes, the platform’s Core Events. The hard part? They had to choose just 10.

“We asked ourselves: What are the key workflows that indicate success in our product? What is the customer trying to achieve in the product?” O’Connor said. “Just the idea of having to take a 60+ year business with over 650,000 users and say these are the 10 key things we think a user should be doing in our product was eye-opening. We were able to bring a little bit of realism and focus into what really matters in our product.”

Those behaviors were then designated as the Core Events in Pendo and became the building blocks of their PES.

Better data, healthier customers

With the Product Engagement Score up and running, IHS Markit had a new, consistent way to make more data-driven decisions. Beyond the product function, sales reps and customer success managers can now drill into each of their accounts and quickly gauge how healthy they are when entering renewal conversations and proactively take measures to prevent churn.

If a customer’s score shows low adoption, for instance, the team can examine their progress toward experiencing Core Events and evaluate whether their onboarding and training sessions have been effective or need to be changed.

View of a key account’s PES score used to help monitor account “health”

When time-to-first-use data from Core Events revealed that users were taking nearly 8 days from first login to access some key features within Engineering Workbench, O’Connor and the team were able to bring that into discussions about how to redesign onboarding to get users to those moments faster. PES and Core Events proved to be useful tools in bringing common ground to those discussions.

Overcoming analysis paralysis

Having tons of rich data to analyze is one thing. Knowing what to do with the results is an entirely different beast. PES has provided the team with a single source of truth and helped them take more decisive action, more quickly.

“It’s a huge value-add, where I’m not having to spend all my brainpower and mental capacity just gathering, cleaning, and reporting data,” O’Connor said. Instead, teams are spending much more time addressing problems, figuring out various triage paths, and finding ways to keep customers happy.

Ultimately, PES is helping IHS Markit rely less on “gut feeling” and more on hard data when considering account health. It provides a common language about solving problems like low adoption or tackling onboarding or retention challenges. 

“PES is all about helping our teams focus on bringing new value to customers whether it’s creating different guides, allocating new resources, or enabling greater discoverability of features. PES makes us smarter and more collaborative when making outcome-based decisions about our products,” O’Connor said.