Experience Management

Three Advantages of Cloud-Based Search Engines

IMG_2801

photo by chad johnson on flickr


Traditionally, enterprise search engines have been firmly planted on the ground. Now, along with almost everything else in enterprise IT, search engines are quickly moving into the sky. But why? What factors are prompting the growth of cloud-based search engines? Given the immense IT security (and mindset) challenges associated with indexing sensitive content in the cloud, why are vendors continuing to march full-speed in this direction?

1. Follow the Content

Covid 19
COVID-19: Digital Insights For Enterprise Action

Access Perficient’s latest insights into how you can leverage digital technologies to not only respond to the pandemic, but drive your operations forward and deliver experiences your customers need.

Get Informed

Almost all of our enterprise search clients are indexing at least one cloud-based content repository, such as Box, Salesforce, Google Drive, Office 365, etc.  This historically involved writing connectors that download all of the information back to an on-premise search engine. As more content sources move to the cloud, it is becoming increasingly inefficient (and absurd?) to download all of the content back to the ground so it can be indexed.
With cloud-based search engines, the idea of cloud-to-cloud indexing is introduced.  Typically, some form of connector or adapter is still required (see my recent post for a suggestion to make this easier), but as long as the connectors are also deployed in the cloud, it is an architectural improvement.  Some vendors are even considering internal peering arrangements that reduces the cost of intra-cloud data movement, reducing the cost to index first-party content in the cloud, since the bits are all flying around within the same cloud.

2. We Need More Power!

KL_CoreMemorySearch applications increasingly demand advanced features like text analytics, language translation, optical character recognition (OCR), document previews, and machine learning ranking algorithms, not to mention increased indexing and serving capacity for an ever-increasing amount of content and number of users.  On-premise search engines often struggle to keep up with the demand for these features, particularly on larger repositories.  Physical limitations of CPU and RAM and storage are a cat and mouse game with traditional on-premise search engines.  The competition between indexing-time resources and serving-time resources is always a delicate balance.
Cloud-based search engines have fewer limits when it comes to computational power and resources.  Yes, there are still ultimately limits, but they can be mitigated more easily in the cloud, and typically at a lower financial cost.  OCR and language translation can be spun off to queues and processed by farms of worker nodes.  Document previews can be stored in vast cloud storage buckets.  Machine learning models can consume enormous amounts of RAM without drawing attention.

3. Hide the Complexity

Untitled drawingDeploying a load-balanced, fault-tolerant, performant search engine on-premise typically requires deploying multiple, replicated copies of the search index and all associated components.  The Google Search Appliance, for example, was a fast machine, but it still required multiple nodes (at increased cost) to handle extreme indexing or serving loads.  Autonomy IDOL and Microsoft FAST architectures for large environments often looked like the ironwork of the Eiffel Tower with all the interconnected layers and nodes.
Properly designed cloud-based search engines can abstract that complexity for the customer.  Search vendors can architect the indexing and serving layers to scale dynamically to meet variable demand.  Products like SolrCloud, Elasticsearch and SearchBlox make it easy to spread load across multiple nodes, but they still require the customer to physically manage their own instances and capacity.  True cloud-based search engines, like Amazon CloudSearch, Azure Search, Coveo Cloud, and Swiftype, are taking this concept to the next level, offering search-as-a-service that eliminates the customers need to worry about physical infrastructure or scale.
The advantages of moving search engines to the cloud are real.  The challenges are real, too.  Next week I plan to address some of the criticisms of cloud-based search engines and the possible mitigations.

About the Author

Chad is a Principal of Search and Knowledge Discovery at Perficient. He was previously the Director of Perficient's national Google for Work practice.

More from this Author

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Subscribe to the Weekly Blog Digest:

Sign Up