In this fast moving Tech world, everyone wants to use best technology with max performance and minimum cost. AWS Cloud was introduced with the same notion. In 2016, AWS introduced its tool DMS (Database Migration Service) to help customers migrate their Databases to AWS.
Through this blog, we would try to understand AWS provided different methods to migrate data from On-Prem/Non AWS Cloud to AWS(Amazon Web Service). Our Primary focus is DMS which is almost free. For more details on pricing, Ref: https://aws.amazon.com/free/migration/ I shall walk you through below topics: a. What is DMS? b. List of Pre-requisites to use DMS.
Lets start: There are two native migration options provided by Amazon:
Schema Conversion Tool (SCT) This is basically a tool which one can use if want to convert database schemas from one engine to another. If you want to know more about SCT, refer below: https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Welcome.html
Database Migration Service (DMS) DMS is a database migration service provided by AWS to help customers move their databases to AWS cloud.
NOTE: Before migration, AWS DMS checks for below on the target:
If above do not exist then AWS DMS creates them and then actual data transfer is initiated.
It does not create schemas on Target table. Schema should already exist if data has to be moved in a specific schema. It does not do conversions neither Schema nor code . If that is needed, we need to use tools like: SQLDev, MySQL Workbench, SCT.
DMS is generally an EC2 server in the cloud VPC(Virtual Private Cloud) which runs software called Replication. It can be used for migration of below databases:
For extensive list, Refer: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Introduction.Sources.html
Security group( with outbound rule set to let all traffic on all ports leave (egress) the VPC )
Below different Network configuration can be used but need to be chosen prior which one you want to go with: Refer: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.VPC.html
a: Network configuration to a VPC using AWS VPN or Direct Connect
Allowed Source details: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Introduction.Sources.html
Allowed Target details: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Introduction.Targets.html
Below is the generic information required:
NOTE: DMS does not creates any unnecessary objects on target which are not required for migration for eg non primary key constraints, secondary index, data defaults are not created.
Case: Homogeneous migration:
This is when both source and target are of same engine type Here schema can be exported and imported using Native tool of your engine also
b. Cached changes:
When you are doing a full load in AWS and your source is still active then there could be data change at source end. These changes are not applied by DMS till full load is complete.
Once full load is complete cache changes are applied.
NOTE: If we want to migrate to different database engine then SCT service need to be used in conjunction with DMS.
Through this blog we tried to understand DMS, its architecture and pre-requisites. In next Part we shall see how we can Configure Ongoing Replication from RDS to Redshift. Stay tuned.
Use the Feedback tab to make any comments or ask questions. You can also start a conversation with us.