Expand your search using AWS native services to identify, comprehend and securely store documents.

The document debacle

Companies continue to fight the battle of the age-old problem: paper documents. Adapting to document modernization to expand the ability to search, catalog and protect HIPAA\PII data is paramount. Perficient continues to help businesses speed up the time it takes to digitize documents for further integration in other sectors. In this article, we will cover how Perficient can help integrate a server-less pipeline within AWS to tackle the document debacle!

Scenario

In the below solution architecture, we will cover data being securely migrated from an on-premise data center to the AWS Cloud. Networking components such as AWS Direct Connect are used to ensure data securely traverses the networking fabric to its destination. The assumption that the data is in a report style format, raw text, Adobe PDF or image based (.PNG, .JPG). This solution can be implemented as a one-time use of a forklift of data, or as replication system in batches over time.

The end-user does not need to be concerned with the process to convert the data, as the server-less pipeline handles all of the data ETL (extract, transform & load). Elasticsearch, when paired with Kibana offers an immensely powerful tool for searching large datasets. It is based on the Apache Lucene engine and is suitable for large document indexing and search capabilities.

Solution overview

Revolutionize Your Business With Generative AI

From product design and software development to virtual agents, content creation, and reporting, GenAI is transforming business. Our AI experts help you unlock GenAI’s full potential and drive growth.

Let’s Get Started

Components

1. Data resides on-premise and is in a format supported for conversion. AWS DataSync is deployed to a conventional operating system and is levied to export the data securely.
2. Data traverses an AWS Direct Connect to ensure the transit remains private and does not traverse public internet space.
3. The VPC endpoint is the ingress point of the VPC, facilitating the secure path.
4. The Amazon DataSync service is configured, with agents running in private subnets within the VPC. The DataSync agent will receive the data and process it. In this case, it will be sent to the destination Amazon S3 bucket for processing.
5. Data is sent within the VPC (privately) to the Amazon S3 bucket. An Amazon S3 endpoint is used to ensure traffic does not leave the VPC. Objects are encrypted in-flight set by the Amazon S3 bucket policy, while the stored S3 bucket objects are encrypted using AWS KMS encryption at-rest.
6. An Amazon Lambda function(s) run to process the data in batches, that have landed in the Amazon S3 bucket. Multiple AWS components facilitate the analysis of the data.
7. An Amazon Lambda function(s) run to extract the data in batches, now sent from the previous function. Multiple AWS components facilitate the extraction of the data.
8. Amazon ElasticSearch service stores the extracted data in an encrypted (at rest and in transit) index. This data now can be used to be searched internally using the Elasticsearch API or Kibana. Amazon Cognito is used to secure the login process, along with integrating SSO if required.
9. Kibana is used to overlay Elasticsearch and provides user friendly search expressions, dashboards and tools.🔍 Employees can now retrieve records & documents much more easily, while using a single interface!

Are you ready to empower your organization’s document search experience?

Perficient prides itself as an Advanced Consulting Partner in the Amazon Partner Network. Our Solutions Architects are subject-matter experts in the search & discovery fields. We are here to help ease the burden of digital document transformation along with expanding search capability.

Contact us today to find out more information about how Perficient can help you get the most out of document search!