Onsite search optimization has often been a low priority for many businesses and website owners. However, optimizing the search experience not only allows your website visitors to find relevant information quickly, but it also helps in improving engagement and conversions.

A study by Forrester Research shows that a high percentage of website users (up to 43%) head straight to do a search after landing on a website. According to another study by Weblinc, onsite searchers are 216% more likely to convert than regular users. So, if you are convinced that onsite search needs attention, keep on reading.

This article gives general suggestions on how to optimize your web-content for site search, but also includes suggestions on how to make best use of search analytics of Sajari Site Search or other site search product.

Getting Started

Once you have completed setting up your site search and have accumulated some search data, such as how, what, and when users search, you can begin optimizing onsite search experiences for your users. We recommend accumulating search data for at least two weeks before beginning site search optimization.

The goal of the optimization is to increase engagement and improve the search experience. Search click-through rate (CTR) is a robust metric which can be used to track whether you have improved engagement and search experience. Click-through rate for a search session is the percentage of results clicked for a particular search session. For example, if 50 visitors use search on your site and 25 of them click on a result, then your CTR will be 50%.

Based on our experiences with setting up site search for a wide range of clients, here are a few practices we think work best to help you improve your site search and return relevant results to your users.

1. Getting it right: Content structure and meta-information

After helping hundreds of customers with search installations, one of the clearest insights we've learned is that is that most site search installations don’t work in an optimal way primarily because of insufficient meta information or poorly defined website structure.

You can use the Site Search Health Report tool we’ve built to help you discover issues in your website content and structure.

Some key things to consider are:

• Logical structure - The website’s directory structure should be logical and follow a hierarchical order. A logical website structure will allow crawlers to detect categorization in your website and also enable you to set boost rules effectively. This article by Yoast goes into a lot more detail about site structures.
• Title & description tags - Ensure the title and description tags accurately describe the purpose and content of the page.
• Headings & paragraph content - Ensure the main keywords of the page are included in the headings and paragraphs.
• Duplicate content - Check if there is duplicate content on your pages and if that might negatively impact your website search results. (Run a Search Health Report to determine pages with duplicate content)
• Canonicals - In case there are multiple versions of a page (e.g. different URLs for each color of the same shirt), set the canonical to identify the original page. (Learn more about canonicals here)
• Redirects - A common problem that often goes unnoticed is not setting up redirects to the preferred version of your website URLs. Visitors use different variations of URLs to access your website, and if you don’t have server-level redirects set up, these URL variations will not resolve to a single correct URL and result in duplicate content in your search results which can confuse users. For example, if your website successfully resolves to two different URLs but they are the same page, the only difference being one has HTTP and the other has HTTPS at the beginning, then it would result in duplicates in the search results. Similar cases where redirects are crucial include: trailing slashes at the end of URLs, ‘www’ or ‘non-www', URLs not being sensitive to capitalization.

This guide might help you in understanding more about how to set up redirects and resolve these issues.

2. Focusing on Ineffective Searches

One of the benefits of adding a modern website search solution like Sajari Site Search to your website is the ability to identify which search terms have a low click-through rate through search analytics.

Sajari customers can log into the Console (view image below), navigate to Content Issues section under analytics and view the ‘Ineffective Searches’ section to find the search terms that are not performing well. To investigate, start by searching for the same terms on your website and check the search results.

Below are a few suggestions that might help to improve CTR% for queries that are underperforming:

• Add a Synonym - If users are using a slightly different term to find content that is referred in a different way (e.g. searching for “pricing”, which is mentioned on your “plans” page), you can add a synonym rule to address this problem. Acronyms are often used as synonyms; GOT for Game of Thrones, SF for San Francisco.
• Review meta information - the Another way to improve your CTR% is to rewrite or update the meta title and meta descriptions of web pages. When you are looking at search terms with low CTR, try to identify whether you have content that is relevant to those queries or not. Sometimes, the search terms will give you an insight into wording and language that you might want to use for your website copy. You can then rewrite your titles, meta descriptions and page content using the same language that your users are searching with. We've noticed that improving meta title and description is one of the most effective ways to optimize internal site search, as it clearly communicates the purpose of the page and allows the user to determine if the page serves their intent. This article on Search Engine Watch gives very useful examples of good/bad titles and descriptions. If you don’t have any content that matches their queries then you know you need to write some new content in a hurry!
• Fix Crawling/Indexing Issues - If some URLs are not showing up in results when expected, it might be possible that the pages have not been crawled or indexed. Head to the diagnose section to investigate if the page has been indexed or not. If it's not indexed, you will find relevant information as to why it wasn't indexed.

3. Optimize Content for Top Queries

• Identifying Top Queries with low CTR% - Another way to optimize your website search experience is to target top queries that have a low CTR%. View the list of top queries by logging in to your Sajari console and navigate to the “Analytics” section.
• Testing out the Top Queries - Use the 'Browse' section or simply search on your website to test the top queries and view the search results. Alternatively, you can also test by searching these queries on your website. Identify any results that do not rank highly for the query but which you believe are relevant.
• Improve Title and Description - Improvements in the meta information (title and descriptions) of these search results would be the best way to improve CTR%. Review the title and meta description for these pages as they will be among the most crucial pages on your website. An easy tweak is to add the search term at the beginning of the title and include it in the description for the relevant page if feasible.
• Use a Boost Rule - Sometimes it makes sense to boost certain categories on your website. Consider a website that lists recipes both from featured chefs and also from other website users. You notice that the results for your top query “chocolate” don’t show any featured chef recipes. You can use a boost rule to promote recipes submitted by chefs if your website is structured in a hierarchical manner. If users begin to click on the chef recipes, it will get promoted as machine learning updates the relevance for that result.

4. Keeping a Close Look on Search Trends

An exclusive feature of Sajari’s analytics platform is ‘Unexpected trends observed yesterday’. This section helps you to detect significant changes in user search behavior. For instance, if a search query was searched 5 times in the last few weeks, but it was searched 20 times this week, then the query will appear in this section.

Looking at this data you can quickly see if there has been a change in search behavior, and then you can decide whether to make changes or additions to your website content.

Step 1: Find out details about the variation in the data by hovering over the query

Step 2: If some business event, activity or external variable caused this change, then:

• Find out if the necessary and relevant content is available on the website, and if it requires updating.
• If there is no relevant content currently available on the website, can an update or content be added to the website to address this change?

5. Using Insights from the Learning Tool

Another tool at your disposal is the Learning tool. It shows how your users have interacted with your search results and whether our machine learning models believe the result is a good match for a query. It takes a rather different approach to look at the behavioral data from the user rather than search query data.

For example, you run a driving school business and your search data shows the term “parking” has a 40% CTR and is also listed as an ineffective search term. You want to learn more and investigate why “parking” has a low CTR.

You suspect that people who search “parking” on your page are looking for parking information when they arrive at the school.

You know that you have some parking information listed under “Getting here” page and also on the “Contact Us” page. You add the term “parking” to the learning page and get the following result.

The confidence for ‘Contact us’ and ‘Getting here’ pages is lower than the confidence for "Parking Tips and Tricks" and "Park and Ride".  This means that the user behavior is different to what you expected earlier! People are clicking on the“park and ride” event page and “parking tips and tricks” page more than the pages with parking information.  

If the title and descriptions of results do not accurately match the content on the page, users can be confused, causing repeated search sessions and frustration. For example, if you notice that the description of the “park-and-ride” result contains the keyword “parking information”, it would lead people to assume that they would find parking information there instead of the event details. Instead, rewording it to “Park-and-ride event information and details” would make it clearer for the user.

Ending Notes:

• One common mistake is starting off without enough data, so if you have just installed site search on your website, wait at least two weeks or till you gather a few hundred queries before you start optimizing.
• Remember to iterate in small increments, and wait for data to accumulate before making more changes. Frequent and drastic changes (particularly with boost rules) would severely skew the search ranking algorithm and have a negative impact on the user search experience.

Hopefully, after reading this you've learned a bit more about how to use Sajari Site Search features to optimize onsite search. Remember that optimization is an ongoing process, and as demonstrated above, it can be both an art and a science. The key to the process is knowing your customers and being led by how they're using your website, not the other way around.

‍