Thomas Supplier Directory
Increasing buyer engagement on Thomas' core B2B product through A/B testing.
Company: Thomas
Project Date: 2023
Role: Senior UX designer responsible for discovery, UI design, and test planning
Team: VP of Product, Data Lead, Engineering Lead, Design Manager, Research Lead
Project Overview
Thomasnet.com is the internet's biggest B2B search directory for industrial suppliers and manufacturers. It was falling short on its promise to deliver audience exposure and leads to paying subscribers, leading to rising churn. Thomas needed to increase buyer activity in order to increase suppliers' return on ad spend. I conducted user interviews to learn how their experience could be improved and A/B tested several solutions. My team and I found 2 simple changes to implement right away and made a plan to continue iterating on improvements.
How the Supplier Directory Works
The Thomas supplier directory contains 500k+ suppliers in 70k+ categories ranging from commodity distributors to custom equipment manufacturers. Suppliers and manufacturers paid a subscription fee to be listed and expected to gain leads for their business. Thomas' detailed categories and filtering tools attracted in-market buyers who contacted suppliers about quoting and purchasing. Most buyer sessions started at the search results page of the directory, which was the focus of this project.
Buyers discover suppliers on the search results page.
Supplier directory user flow diagram
Problem
Subscriber churn increased last year because suppliers experienced a decline in exposure to and inquiries from potential clients. Churned suppliers felt that Thomas had not contributed to their business' growth so they did not renew their subscription.
The product team had little insight into how buyers experienced the platform because buyer research was 10+ years out of date. Thomas was also experiencing organizational changes after its acquisition and was in the middle of a design system update. Therefore we were limited in the amount of front-end changes we could make at the time.
“...last year we looked at what we had paid versus how many RFQs came in, and the ROI just wasn’t there.”
Objective
We aimed to increase the following buyer engagement metrics that suppliers used to measure ROI of their subscription:
Inquiries from potential buyers
Visits to supplier profiles
Engagement with supplier listing, such as visits to their website, phone number reveal, and video or catalog views
Additionally, we needed to document the buyer experience and update our research repository.
Discovery Research
Working with a researcher, I planned and conducted interviews with 10 buyers from Thomas and similar users from usertesting.com. Partnering with the design manager, I reviewed playback sessions on Quantum Metrics to observe user behavior on screen. We learned their needs, goals, and pain points that contributed to buyer disengagement.
I documented the buyer’s procurement process in a journey map, which included Thomas touch points and activities performed outside of the platform. The industrial buying journey could take months to complete.
The industrial buyer’s procurement journey, with steps that take place on and off Thomas
Testing Strategy
After gaining research insights, I helped my team sort and prioritize buyer problems based on impact on suppliers. We were constrained by the scope of UI changes we could make due to ongoing design system and front-end updates. I decided to start by testing solutions that did not involve complex UI changes in order to deliver value to customers as quickly as possible. After the design system update, engineering would be able to spin up and test more variations.
Intervention 1: Match Button Copy with User Intent
Problem
We learned from user interviews that the main “Quote and Buy” call to action did not match user intent while they browsed suppliers. During supplier discovery, users are researching suppliers’ capabilities and are likely not ready to ask for pricing or make a purchase. It was more important for them to contact a supplier to further discuss their project before quoting.
Hypothesis
Changing the call to action copy from “Quote and Buy” to “Contact Supplier” could better capture the buyer’s intent to send a general inquiry even if they were not ready to buy yet. This could introduce more leads to suppliers’ sales funnels who are earlier in the buyer journey.
Test
We A/B tested 2 versions of the button in a 50-50 split of “Quote and Buy” and “Contact Supplier”.
Metrics: Clicks on the button and inquiries sent from buyers
A/B Test: CTA Button Copy
A: CTA with “Quote and Buy” - Original
B: CTA with “Contact Supplier” - Implemented ✅
Result
The "Contact Supplier" segment had a 36% lift in clicks on the button and 11% increase in inquiry sent. The test was a success and the new version was easy to implement.
Intervention 2: Help Buyers Evaluate More Suppliers Each Session
Problem
The search results page opened up supplier profiles in the same tab by default, which slowed down buyers from evaluating multiple suppliers and distracted them from returning to the search results if they wanted to see more suppliers. By observing and interviewing buyers, we learned that it was common for them to manually open multiple supplier profiles in a new tab on desktop.
After analyzing the search results experience of other listing websites, I noticed that 6/16 opened up profiles in new tabs on desktop, so it must be a common design pattern for users to encounter.
“I would open up suppliers in a new tab or write down their names in a notebook.”
Hypothesis
Buyers could efficiently evaluate more suppliers profiles if they opened in a new tab without navigating away from the search results page, which could lead to more profile visits and engagement activities per session.
Test
I worked with the data and product team to plan a simple A/B test with a 50-50 split in default supplier profile open behavior. The control would have the existing behavior and open profiles in the same tab, navigating away from the search results page. The test half would open profiles in a new tab with the profile page in focus.
Metrics: Supplier profile views and engagement activities per session
A/B Test: Opening Profile in a New Tab
A: Profile opens in the same tab - Original
B: Profile opens in a new tab - Implemented ✅
Result
Opening profiles in new tabs resulted in a 25% lift in supplier profile views per session and a small lift of 1-2.5% in clicks on website and phone number respectively. This solution was also deemed a success and implemented right away without needing UI design changes.
Intervention 3: Encourage Buyers to Use Filters
Problem
Buyers we interviewed barely noticed the filters when asked to demonstrate how they got more specific. Usage data indicated that buyers were not using the filters to narrow down suppliers that fit their most important criteria (location, capability, company type, and certifications).
This caused a 2-way problem: Buyers sent inquiries to suppliers who were a poor fit, so their messages were ignored. This discouraged buyers from contacting Thomas suppliers in the future, contributing to fewer leads and decreased ROI for suppliers.
“The worst part I would say is finding the exact thing that I need. Sometimes it’s, you know, in five minutes, sometimes it takes like days to find the part that I’m looking for.”
Hypothesis
Buyers skipped the filters because the list was long and did not show the most important criteria at the top. A layout that displayed the most relevant filter categories at the top of the page could increase filter usage and help buyers find and contact relevant suppliers.
Test
I designed 2 new filter layout variants to test against the existing control in an A/B/C test and developed a testing plan with the product, data, and engineering teams.
The first variant had the same vertical layout as the existing filters list, but the filters were reordered to have the most important criteria at the top.
The second only had the 4 most important criteria in drop-downs at the top of the screen. Additional filters were available in a modal so buyers could focus on selecting filters in a space designated for the task.
Metrics: Overall filter usage, inquiries sent, and inquiry responses
A/B/C Test: Moving Most Important Filters to Top of the Page (Not Run Yet)
A: Filters on the left that users did not use - Original
B: Original filter design, reordered with most important filters at top of list
C: New drop-down design displaying the most important filters at the top of the page and the rest hidden in ‘More Filters’
Result
This test is being built and has not been run yet.
Impact
We were able to quickly boost some buyer engagement even though we only ran 2 low-code tests without major UI work. Since engineering implemented these solutions right away, suppliers saw some metrics improve before our team progressed with testing more complex solutions.
Opening supplier profiles in new tabs increased supplier profile views (+25%) and slightly increased engagement activities (+1 to +2.5%). Changing the CTA to “Contact Supplier” lifted inquiry submissions (+11%).
Adding fresh discovery research to our knowledge base kick-started the process of improving the buyer experience in order to increase ROI for suppliers. The team will dive deeper into other problems that were discovered during the research but were out of scope in this project.
Since the business cycle in this industry takes months to a year, it will take a while before we can measure the impact on supplier churn.
2 simple, low-code changes resulted in moderate lift in metrics for suppliers
Next Steps
Run Test on Filters
The engineering and design team will finish building the A/B/C test for filter order and location, and the data team will measure its impact on filter usage and inquiries to and from suppliers.
Test Alternative Solution for Evaluating More Suppliers
The product team floated the concept of a 2-panel design of the search results page that could help buyers view and evaluate more supplier profiles without navigating away from the list. This design would take more time to build and test, so it was cut from the scope of this project. I designed some early mocks of a 2-panel design for the team to use as a starting point. This design would be tested against the new tab experience for supplier profile views and engagement activities per session.
Low-fi sketch of a 2-panel layout with supplier profile preview on the right.
High-fi mock of a 2-panel layout with supplier profile preview on the right.
Address Platform Leakage
9 out of 10 buyers we spoke to discovered and evaluated suppliers on Thomas but contacted them off platform by email, phone, or the supplier’s website. While this was an effective way for buyers and suppliers to do business, Thomas did not get credit for facilitating those interactions. This led to a larger conversation about whether Thomas should force users to change their behavior and keep communication (and transactions) on platform or rethink how to measure ROI for subscribers. Since this was Thomas’ first study on buyer behavior, more work is needed in this area.
Buyers and suppliers who connected on Thomas take their business off platform