Pete – “This year we have the budget to replace our existing data replication technology.”
Harry – “Good to hear. Have you evaluated any tools or services?”
Pete – “I recently attended an Amazon Web Services (AWS) seminar hosted by our cloud services team. They showcased many managed and core compute services. Harry – You are a certified AWS developer. Do you know of any AWS offering that might come in handy? Our organization’s IT goals are geared towards cloud adoption; better we choose a cloud based solution and ensure that our department’s IT strategy is in alignment with that of our organization.”
Harry – “I might just know of an AWS managed service that is worth evaluating – AWS Data Migration Service (DMS)”
Brief
Like Pete and Harry many in our departments face a similar dilemma when evaluating data transfer technology. Coupled with costs associated with the new databases, these discussions lead to complex and lengthy migration plans with prolonged downtime. AWS’ DMS is a data migration service that helps with quick and secure data migration with minimal downtime to the consuming application. The service can be used to migrate data from on premise to AWS and vice versa or even between two on premise data bases. The service supports both homogeneous migrations (same source and destination dbs) as well as heterogeneous migrations (source and destinations dbs are different). The service can also move data straight to the Redshift data warehouse, where it could be used for SQL based data analysis.
In case of heterogeneous migrations, a free of cost Schema Conversion Tool (SCT) can be used to automate the schema mappings. The tool automatically migrates the source to the destination schemas. There could be instances where certain elements could not be mapped and the tool flags them for manual intervention and rectification. Once configured, DMS’s replication task can read data from the source and moves it to the target database. It will continue to do so even for the data that has recently changed after migration. Replication task can be configured to continue moving new or changed data to the target database in real-time.
As with other AWS offerings, DMS is most economical when used with other AWS offerings. If the target database is Aurora, Redshift or DynamoDB, then, in a single AZ (Availability Zone) environment the service use is free for six months with a possible extension of another three months upon request. This preludes to cost effectiveness when the organization plans further adoption of the AWS’ echo system. For example, AWS Aurora, an InnoDB based storage management system which costs one tenth the price of Oracle, if chosen as the target database can not only save you cost in migration but also in storage. DMS is a low-cost service where the consumer only pays for the underlying compute resources, storage to manage replication logs and data transfer if the outbound data is not stored on the same AZ or AWS region or resides outside of AWS. For a typical use case where an on premise data is moved to AWS, DMS will cost as much as $3.00 in service charges.
Sources for DMS
Source databases could be both on premise and EC2 based instances – Source could be Oracle, MySQL, MS SQL Server edition, MariaDB, PostgreSQL, MongoDB and SAP ASE. In short if your organization uses the most common database systems and are not stuck in versions from the prehistoric IT era, in all probability your source database is a good candidate for DMS.
DMS also supports full load when using Azure SQL database as source. Good to know if your organization decides to migrate over from Azure to AWS.
AWS RDS supported databases are Oracle, MySQL, MS SQL, MariaDB, PostgreSQL and Amazon Aurora with MySQL compatibility.
DMS supports full data load and CDC (Change Data Capture) when using S3 as source.
Please refer DMS documentation HERE for exact versions supported.
Targets for DMS
Irrespective of the source storage system, DMS creates the MySQL-compatible target table as an InnoDB table by default.
Supported on premise and EC2 based engines are Oracle, MySQL, MS SQL Server edition, MariaDB, PostgreSQL and SAP ASE.
AWS RDS supported databases are RDS Oracle, RDS MySQL, RDS MS SQL, RDS MariaDB, RDS PostgreSQL, Amazon Aurora with both MySQL and PostgreSQL compatibility, Redshift, S3 and DynamoDB.
Please refer DMS documentation HERE for exact versions supported.
Security
DMS supports security for data that is in transit or at rest. AWS KMS can be used to generate and manage encryption keys. It support SSL and require authentication through IAM as an authorized user.
Summary
AWS DMS is a highly resilient data migration service. The service continuously monitors the source and target databases alongside network connectivity. Any interruption that results in halting of replication task is automatically rectified and the process resumes from where it was halted. For higher availability, the process can be deployed in a multi AZ environment. If your organization is looking for AWS based replication solution, then DMS could be your answer.
If you’re interested in learning more, reach out to one of our specialists at sales@perficient.com and download our Amazon Web Services guide for additional information.
