Find out how Catch boosted on-site search conversion rates more than 40%

At-a-glance

• More than 40% lift in search conversion rate

• Scalable search on more than 2 million products

• More than 500 products fully updated and searchable every second

‍

Catch wins with world-class search

Catch.com.au is Australia’s largest online marketplace selling more than two million products across food, liquor, clothing, footwear, furniture, sporting goods, electronics and more. The fast growing ecommerce giant wanted to display more relevant search results to its 1.5 million loyal customers, an A/B test was arranged to compare the existing search solution with Search.io's next-generation, machine learning-powered search. Discover how Catch enhanced search relevance and significantly improved sales and profitability using Search.io's machine learning-powered search.

‍

‍

‍

Seeking relevance: assessing Search.io for onsite search excellence

Australia’s leading online marketplace Catch.com.au wanted to explore how they could display more relevant products through onsite search to lift conversion and deliver more revenue. The online retail giant put their existing solution and Search.io's search to the test, measuring both solutions in an A/B test across multiple e-commerce metrics to see if their onsite search could generate more revenue.

Catch had been using a tie-breaking based search solution from Algolia. This type of search engine is highly transparent and businesses can easily add business rules to modify the search experience. For Catch this initially worked well, but once hundreds of rules had been written, many in conflict with each other, the limitations of this approach became very clear.

“We had pushed the pre-sorted, tie-breaker algorithm to its limits. There was no upside left.” explained Catch.com.au’s Chief Product Officer Dr. Liron Nehmadi.

Catch wanted to use its own rich business data - like product conversion rate and discount percentages - to influence internal site search rankings and lay the foundation for a better shopper experience.  

Catch pitted Search.io against their existing solution in a strict A/B test to determine which search solution could deliver a better conversion rate from Catch.com.au shoppers.

Catch found three major benefits to using Search.io, which included:

‍

• Less manual intervention - Breaking away from maintaining hundreds of manual query rules, each with hard to predict side-effects. Instead using a combination of machine learning powered and internal business expertise to determine the best patterns to optimize towards.

• Supporting quick iteration of search relevance A/B testing - Flexibly testing different relevance strategies in real-time. Search.io allows new query strategies to be tested without needing to duplicate entire indexes.
  

• Personalizing results for different customer segments -  Using machine learning to adaptively improve search personalization. For example, boosting discounted member-only products if a customer is a member.
  

‍

43% of e-commerce sales involve an onsite search

Onsite search is a key tool to satisfy e-commerce customers and grow sales revenue. Around 43 percent of all transactions on any given e-commerce site will be influenced by onsite search - most online shoppers head straight to a search box to find what they want to buy. But determining the best products to show for a given query is challenging.

“Search is always a challenge for ecommerce businesses - while shoppers know what they want to buy, the language they use to search for products can be ambiguous and the intent behind the same query varies from person to person,” says Search.io Chief Product Officer Jens Schumacher.

“If someone searches for “apple” it would make sense to assume they are searching for Apple branded products, but often they end up buying non-Apple branded accessories. With pre-sorted ranking everything is in hard preference order, like sorting the columns in an excel spreadsheet, so this kind of ambiguity becomes difficult to deal with. There are many reasons a product should rank well for a particular query, there is no silver bullet and forcing priorities means some products and searches will suffer.”

Search.io gets around this problem by allowing nuanced relationships between queries and product data to more intelligently rank results based on behavioural trends. Using machine learning, Search.io develops an understanding of which product data fields best predict a positive outcome, like a click or a sale, for a specific query. The relevance weighting of that field is then boosted when the query is entered by a user.

Dr. Nehmadi commented: “Managing such a large range of products using pre-sorted ranking meant the business had applied all sorts of rules for specific queries. When someone left the business no one knew why a rule was there, if it mattered and if we could remove it. Search.io has allowed us to use our business intelligence to experiment with different search approaches without having to manage hundreds of specific rules.”

‍

"Search.io has allowed us to use our business intelligence to experiment with different search approaches without having to manage hundreds of specific rules."

Catch.com.au Chief Product Officer, Dr. Liron Nehmadi

‍

‍

Smarter search with conditional boosting

Prior to the A/B test, Catch had a particular problem with their incumbent search solution around the search query “TVs”. Catch stocked smart TVs, TV box sets and TV accessories like wall mounts and antennas. The online retail giant knew that most people typing in the query “TVs' ' were looking for a smart TV, but the current system’s pre-sorted ranking approach meant TV accessories and TV box sets were being displayed ahead of smart TVs.

After Catch implemented Search.io's machine learning capabilities, the ecommerce team could determine which product categories were more likely to lead to a sale for a particular query - the team could then boost products within the category accordingly using a model created by Sajari’s machine learning systems.

At Search.io, we call this conditional boosting, which is giving a relevance boost to certain products based on conditions such as product metrics, or customer data – or any other ‘condition’ – to display the most relevant result for any given query.

Today when customers search on Catch.com.au, smart TVs outrank other TV-related products, resulting in higher conversions and more sales revenue for these big ticket items.

‍

‍

Catch.com.au’s legacy search displaying less relevant TV results (left) side-by-side with relevant, conditionally boosted Search.io results (right).

What’s more, conditional boosting also enabled Catch to display special member products or discounts, based on the searching customer’s Catch.com.au membership status.

“Our members pay us a subscription fee to access discounted products and free delivery, with Search.io we can boost results that are relevant to them so they get the most value from their membership,” explains Dr. Nehmadi.

‍

Rapid A/B testing delivers immediate growth

Experimenting with different relevance approaches was key to unlocking the power of onsite search and higher conversion rates. Search.io allows rapid real-time tests to define the best relevance algorithm for a given circumstance or data point.

“A/B testing new search relevance approaches with pre-sorted, tie-breaker algorithms means running duplicate indexes for each test variant - that just wasn’t feasible.” Dr. Nehmadi said.

“Search.io allows us to immediately run many A/B tests without index duplication. Since we started working with Sajari we’ve learnt a lot about what’s useful in our own data.”

Catch is now running multiple relevance approaches concurrently. Catch.com.au’s users are bucketed into different testing groups with each group shown different results based on different relevance approaches. Some experiments boost discounted products more strongly, others boost products with faster delivery times and some rely more heavily on Search.io's reinforcement learning.

This means Catch can now measure which relevance strategies are most effective for conversion and session profit. Once a new relevance factor is established, Catch then uses it as a baseline to launch further experiments.

‍

"Search.io has allowed us to use our business intelligence to experiment with different search approaches without having to manage hundreds of specific rules."

Catch.com.au Chief Product Officer, Dr. Liron Nehmadi

‍

Machine learning search adds millions in increased sales

During various A/B test runs over six months, Catch and Search.io were able to increase search conversion or session profitability by more than 16 percent. Sajari also enabled the ecommerce giant to simultaneously combine profit and conversion goals by delivering a six percent increase in conversion rate along with a six percent session profit lift.

Leaving out the cost of no longer manually implementing hundreds of business rules to make search work, Search.io's search performance directly translated to millions of dollars in incremental revenue for Catch with a return on investment (ROI) exceeding 100x.

“When we set out on this project we knew there was an opportunity to use our investment in data to make our search smarter. With Search.io we’ve definitely delivered. Our results are more relevant to our customers and we’re able to sell more of the products that matter to us, which has resulted in huge improvements to our bottom line.”

‍

"Search.io has allowed us to use our business intelligence to experiment with different search approaches without having to manage hundreds of specific rules."

Catch.com.au Chief Product Officer, Dr. Liron Nehmadi

‍