An online retail company has more than 50 million active customers and receives more than 25,000 orders each day. The company collects purchase data for customers and stores this data in Amazon S3. Additional customer data is stored in Amazon RDS.
The company wants to make all the data available to various teams so that the teams can perform analytics. The solution must provide the ability to manage fine-grained permissions for the data and must minimize operational overhead.
Which solution will meet these requirements?
A. Migrate the purchase data to write directly to Amazon RDS. Use RDS access controls to limit access.
B. Schedule an AWS Lambda function to periodically copy data from Amazon RDS to Amazon S3. Create an AWS Glue crawler. Use Amazon Athena to query the data. Use S3 policies to limit access.
C. Create a data lake by using AWS Lake Formation. Create an AWS Glue JDBC connection to Amazon RDS. Register the S3 bucket in Lake Formation. Use Lake Formation access controls to limit access.
D. Create an Amazon Redshift cluster. Schedule an AWS Lambda function to periodically copy data from Amazon S3 and Amazon RDS to Amazon Redshift. Use Amazon Redshift access controls to limit access.
Sure, here's why the other options are not as suitable:
A. Migrate the purchase data to write directly to Amazon RDS. Use RDS access controls to limit access.
This option would not meet the requirement to minimize operational overhead. Migrating all purchase data to write directly to Amazon RDS could be a significant task, and managing access controls in RDS could also be complex and time-consuming.
B. Schedule an AWS Lambda function to periodically copy data from Amazon RDS to Amazon S3. Create an AWS Glue crawler. Use Amazon Athena to query the data. Use S3 policies to limit access.
While this solution could work, it doesn't provide the ability to manage fine-grained permissions for the data as effectively as AWS Lake Formation does. S3 policies are not designed for fine-grained access control.
D. Create an Amazon Redshift cluster. Schedule an AWS Lambda function to periodically copy data from Amazon S3 and Amazon RDS to Amazon Redshift. Use Amazon Redshift access controls to limit access.
This solution could also work, but it might not minimize operational overhead because managing an Amazon Redshift cluster and scheduling AWS Lambda functions for data transfer can be complex tasks. Moreover, Redshift is a data warehousing solution and might be overkill for this use case if the primary requirement is just to perform analytics on the data.
The solution that will meet these requirements is:
C. Create a data lake by using AWS Lake Formation. Create an AWS Glue JDBC connection to Amazon RDS. Register the S3 bucket in Lake Formation. Use Lake Formation access controls to limit access.
This solution allows the company to make all the data available to various teams for analytics, manage fine-grained permissions for the data, and minimize operational overhead. AWS Lake Formation simplifies the process of setting up, securing, and managing data lakes. AWS Glue can connect to Amazon RDS using a JDBC connection, and you can register an Amazon S3 bucket in Lake Formation as a data source. Then, you can use Lake Formation's access controls to manage permissions for the data.