How to Measure Effectiveness of Your E-commerce Search

How to Measure Effectiveness of search engine for eCommerce sites?

Measuring how effective is your site search engine for eCommerce sites can be tricky. High-level metrics such as revenue and conversion rate are nearly impossible since many other factors influence these. After enhancing search or making configuration changes, there can be an expectation of increased revenue or conversion rate. But then, let’s say neither of these metrics changes. How do you investigate further?
Implicit in such discussions is that there is data available for a before-after comparison. What if it’s a new site and there is no prior data available?
In this post, you’ll find out how to determine the effectiveness of the search engine for eCommerce sites:

  • When there is no change in revenue or conversion rate
  • When you do not possess the data for making a before-and-after comparison for other metrics.

Other low-level metrics can still signal changes in search quality. The nice thing about such metrics is that you do not need to have any data before deploying a search to determine how much benefit search is having for your site. Once collected, they are also a great way to track search quality henceforth.
So let us dive in!

Search Contribution to Transactions

IOW, the percentage of transactions that are taking place through search. This metric indicates how many transactions are taking place when visitors find products of interest through search. The other ways users can find products are:

  • through an SEO/SEM landing page,
  • those that are taking place through a visitor’s manual browsing of the product catalog/categories
  • Email marketing campaigns, etc.

All of the above will combine to give you a 100% score. This score will tell you where most of your transactions are coming from. It is an indicator of whether the search is a driver of commerce.

How to Measure Effectiveness of Your E-commerce Search
High Conversions from Search

Suppose the percentage of conversions from search is high, as in the pie chart above. In that case, we can say that search is a significant contributor to your revenue, and you need to pay a lot more attention to the quality and accuracy of search results.

How to Measure Effectiveness of Your E-commerce Search Low Conversions from Search

In contrast, if conversions from search are low, we need to determine the reason behind the down conversions from search. It can be because of the poor relevance of search results to unattractive prices on the search results page. Let’s try finding out how the search quality looks like.

Search Vs. Browse Percentages


Once a visitor is on your site, there are two things he can do[Note 2]. He can either look for products of interest through the search engine or browse through your product catalog.  At this step, we need to look at what %age of visitors ends up using search for locating products versus those that end up simply browsing through the product catalog.

How to Measure Effectiveness of Your E-commerce SearchSearch Queries and Product Ranks

When the visitors are spending more time searching for products, the chart will look something like the image above. If the conversion source chart earlier shows that the conversions from search are low even when visitors are spending more time searching, it is a definite indication that there is something wrong with the search results. It could be due to irrelevant search results or a UI/UX issue with the results page. We’ll investigate this with Click-through rates in the later section.

How to Measure Effectiveness of Your E-commerce SearchLow Search Activity over Browsing or Catalog Navigation

On the other hand, if the search activity is far less than the browsing or catalog navigation, you can rest assured that the conversions from search are low simply because your visitors are not naturally inclined to use search while looking for products of interest.

Click-through Rate on Search Results Page

A click on a product in the list of products on a search results page is one click-through. If a user ends up clicking on at least one product after every search query, you will observe a click-through rate of 100%. This metric will tell us what percentage of visitors click on the products displayed in the search results.
If the number is high, it means that the quality of search results is high.
If the number is low, either the results we show to users are not relevant enough, or another factor like product price is driving visitors away from clicking on the product.
To discover something about the search quality, let’s take a look at the next metric.

Average First-Click Rank

Click Rank was the rank of the product when shoppers clicked on the search results page. First-Click Rank is the rank of the product that the shopper clicks first on the search results page.
E.g., for two search queries, the order of displaying the results on a results page is:

How to Measure Effectiveness of Your E-commerce Search
Search Queries and Product Ranks

For query 1, A’s rank is 1, B’s rank is two, and C’s rank is 3. If the user clicks on Prod C first, then the First-Click Rank of this search query is 3.
For query 2, E’s rank is 1, F’s rank is two, and G’s rank is 3. If the user clicks on Prod E first, then the First-Click Rank of this search query is 1.
Using these two data points, the Average First-Click Rank of the site will be:
( First-Click Rank for Query 1+ First-Click Rank for Query 2 ) / No. of Search Queries = 2
For the Average First-Click Rank, lower is better. If the average value for this metric is lower, it will mean that the first clicks across the site after the search take place on the most relevant result. If the value is closer to 1, we can safely assume that the search results are highly suitable for the site.
If the value is low, we’ll know that the search engine is throwing up the relevant results. On the other hand, if the value is high, it is an indicator that either due to search quality or other reasons:

[Note 1], your visitors do not like the products displayed on the top of the search results page.

[Note 2] Other reasons can range from the high prices displayed for each product on the results page to out-of-stock products, which are still relevant from a search perspective but not from a conversion one.
[Note 3] There may be other actions the user can take but for this investigation, let’s assume there are only two, searching and catalog navigation using the left side menu.

Now that you know how to measure the effectiveness of the search engine for eCommerce sites. The question is do you want to go through all that hassle by yourself. Book a site audit with us.

More To Explore