Occasionally something comes along that blows your mind. It feels like magic and just working on it makes you smile.

Today we are releasing Neuralsearch® into beta which is a hybrid combination of traditional keyword based search and neural hashes to deliver the worlds first instant AI search. By instant we mean single digit millisecond queries, by AI search we mean real AI retrieval based on mathematics and not keywords (think synonym suggestions, dictionaries and language rules).

Below is an introduction and some background on why this is going to be massively impactful.

What is instant search?

Instant search is defined as less than 50 msec from pressing a key to search results being returned. The human eye refreshes around 50 times a second. This is why when wheels spin at certain speeds they look like they’re rotating backwards. It’s also why movies were traditionally done at 24 frames per second, it’s slightly janky if you concentrate and look for it, but generally it is perceived as continuous.

So 50 msec is 1/20th of a second, almost the same as the film frame rate. Below 50 msec most humans won’t notice any delay. Above 50 msec and we start to!

The impact of instant search

Amazon once famously tested incremental increases in search latency to see what impact latency had on conversion and revenue. The impact was profound.

Approximately a 1% revenue loss for every 100 msec of added latency.

Slow search kills revenue

“Big deal,” you might say, people have been doing instant search for a while. Yes they have, but no one has done proper dense AI retrieval instantly. We have now, though, and that is a big deal. AI search is pretty amazing — you basically don’t need synonyms and language rules, concepts are understood, you can answer questions and much, much more…

So instant search is great, but AI instant search just landed and the old keyword based instant search is already looking outdated.

The next two sections are somewhat technical, but I think it’s worth explaining what we mean by “world’s first instant AI search” and why we’re so excited to introduce this new service. If you prefer, however, skip to the end to understand what it means for your site.

What is AI search?

AI search goes beyond keywords and instead represents text as mathematics which automatically embeds the meaning of language and thus can deliver highly relevant results no matter what keywords were used.

An analogy to keyword search would be asking if you are in Los Angeles and the answer is a binary yes or no. Technically Long Beach is a different city and not in Los Angeles, so maybe you need a rule to make that be a yes instead of a no. Greater Los Angeles is made up of 90+ cities though… ugh this means lots of rules and exceptions will be needed.

AI-based search is different and instead uses mathematics to measure proximity, so the analogy here would be using a latitude and longitude instead. So unlike asking “if” (binary), you instead ask “how close” (proximity) you are to Los Angeles. Suddenly Long Beach is very close, as are the other 90+ cities and bingo! You don’t need all those rules anymore.

As per above, when search uses keywords, they either match or they don’t. Normally if you see someone search for “usbc”, “usb-c” or “usb c” that would need to match exactly with the text on the product. That’s not good; often it will fail unless they enter it exactly. Enter synonyms and rules. These allow simple issues like this to be fixed, but

1. you need to write the rules and
2. rules often have exceptions. This is subtle but costly in many ways.

For example: What if you wrote a rule to convert queries containing “usb c” to “usb-c”. It fixed your issue and you headed home for the night. The boss then calls to let you know a supplier is complaining his cables have disappeared from search. You suddenly realize “usb cable” is being turned into “usb-cable” and now returning zero results as a consequence. This example is contrived but illustrative of a rules-based world.

Unintelligent search kills revenue and drains resources by requiring manual intervention to make basic expectations work

AI-based retrieval is different. You basically don’t need synonyms at all — language that has the same meaning works as you would expect it to. It’s smarter and more human in every way. And best of all you get better outcomes with infinitely less effort.

The power of AI search

So basically everyone agrees AI is the future of search and it’s easy to see why. So why are search providers still suggesting synonyms, rules, and hacks and selling it as AI search?

Because delivering AI search is really hard and either they can’t figure it out or they just think you won’t know the difference. Real AI search is computationally very expensive and also uses lots of RAM. So to deliver proper AI search instantly you need to do one of several things:

1. Spend big $$$ on infrastructure (you need high revenue per search to justify this, few businesses can justify it)
2. Accept massive drops in accuracy (reduced search result quality by compressing AI models and such)
3. Use it as a fall back only (this relies on keywords, and only on failure to match calls out to do the more expensive query. It’s better than keywords alone, but still expensive to deliver)
4. Find a way to make it 100x faster and cost effective enough to scale up and be a realistic solution

Both #2 and #3 are becoming common as search providers desperately try to find cheaper models that will scale up, or they tack on vector databases like Milvus and Faiss to their open source keyword-based search, but both approaches are a hack and somewhat compromised from the outset.

#4 is the future and that’s where we focused accordingly. After more than a year of R&D, much testing and experimentation, we’ve managed to solve this and add neural indexes into our existing search index. This is special because unlike bolting on a third party vector database, these “neural indexes” live inside our standard indexes and are part of our overall existing data structure. This isn’t a bolt on, it’s a core part of Search.io. Amazingly this means index upserts (also unique to Search.io) literally add and delete neural indexes as you modify records! You don’t need to wait for a vector index to be built, your changes are live in search as soon as you make them!

If you’re familiar with HNSW and other vector based approximate nearest neighbors techniques the key differences are:

• You can insert, update and delete easily (deletions are mostly unsupported in other methods. Graph based structures like HNSW make deletion very complex)
• The RAM needed to replicate accuracy of vector ANN is roughly 10% of the size (can be smaller with reduction in accuracy), which helps to make neural indexes extremely cost effective
• Sharding neural indexes scales up linearly with more shards (HNSW performance drops off non-linearly with more shards)
• The neural indexes are stored in very similar structure to keyword indexes so hybrid retrieval is simple and easily testable in contrast to systems bolting on some kind of vector based ANN database.

The magic of instant and AI search together

Slow search is bad for business. Unintelligent search is still the common standard today, but will soon be unacceptable as it is bad for business as well. Fast and intelligent search is the future, but you don’t need to wait for it as you can experience it already today!

For Search.io customers, Neuralsearch provides unmatched search quality and performance. It is:

• On par with the fastest keyword search available today
• Noticeably more accurate due to utilizing both AI and keyword retrieval together
• Much easier to retrain exceptions thanks to a one-click teaching module we’ve added to the UI

Neuralsearch is available today in a beta release. When combined with our built-in reinforcement learning and dynamic boosting, you can offer your visitors incredible, self-optimizing results that boost conversions, improve customer experience, and lower support costs.

Contact our search experts for a custom demo and free trial of Neuralsearch to see how it can impact your business.