Gone are the days when search engines primarily are keyword driven. Modern search has become more intuitive thanks to the capabilities of natural language processing (NLP). Perficient’s Director of Search and Content, Eric Immerman spoke on a webinar hosted by Coveo to discuss the power of NLP in modern search.
The Pitfalls of Traditional Search
Traditionally in the early days of web search, search engines were very keyword based. Meaning that if you type in a word in a search engine, the engine looks for how frequent it is in a result, what total percent is in the makeup, and how closely related the words you searched in your quarry are.
The downside to traditional search is that it relies on keyword matching and keyword matching is not intuitive for humans. We expect search engines to look for the concept of the keyword we searched, rather than the keyword itself.
The Solution: NLP in Modern Search
NLP strays away from straight index approaches and uses deep learning algorithm models to become more conceptual. The algorithm model continuously gets better and better as it gets more data. This allows search engines and platforms, with NLP embedded, to retrieve the most relevant results based on context and the searcher’s intent.
NLP algorithms use a mix of analytical signals:
- Who searches for what?
- Who clicks on what?
- How do people interact?
Eric Immermann explains that “these are content understanding signals that use natural language processing, knowledge graph, and other technologies to help the search platform or search engine better understand the content that’s being searched for”
Choosing the Best Method of Search Implementation for You
When it comes to implementing NLP search capabilities there are two approaches, building your own NLP search engine or investing in an NLP search platform to do the work with you.
Building Your Own NLP Search Engine
You can create a search engine using Solr or ElasticSearch and build the NLP capabilities and natural query understanding models on top of that. Eric Immermann describes this approach as “assembling the LEGO block of your search capabilities” on your own. However, this will require developers that understand how to do this.
This method is great if you want to own your search capabilities from the ground up and tweak it for every intricacy of your business. However, there is a significant investment required in implementation, development, testing, tuning, and ongoing product-level management to keep your search product running well.
Investing in an NLP Search Platform
You can work with a company like Coveo who has taken all of these technologies and integrated them into a holistic platform that is ready out of the box and makes it available as a SASS solution. They have done the tuning and testing already. Additionally, they have built a user-friendly business-facing UI on top so that business users can go in and look at the analytics, make tweaks to relevancy, do A/B testing, and get feedback without the need for a developer’s help.
Essentially this decision boils down to whether you want to take the time to build your own search engine from the ground up or do you want to work with a company like Coveo and outsource your relevancy capabilities to a team that specializes in search so you can focus on the rest of your business.
Learn More
Eric Immerman runs the search practice at Perficient and has a team of 50 people who implement search across the board, from commerce, website, services, and internal search. Eric has been involved with 100+ search implementations and counting! Learn more about how our team can help you provide a relevant search experience with Coveo!
