Description

Build a realistic AWS data engineering project that ingests healthcare data from Amazon RDS PostgreSQL into an Amazon S3 data lake using AWS Glue.You will work through the project as a data engineer—from understanding the business context and source systems to defining the pipeline contract, making design decisions, setting up AWS, implementing the solution, validating the output, and scheduling repeatable runs.In this project, you will:Prepare the source: Set up a PostgreSQL database in Amazon RDSBuild the Glue job: Ingest multiple source tables using AWS Glue and PySparkLoad incrementally: Use watermarks to process new and updated recordsStore the data: Write the output to Amazon S3 as Parquet filesTrack each run: Add ingestion metadata for auditing and validationCatalog the data: Use AWS Glue Crawlers and the Glue Data CatalogValidate the output: Query the ingested data using Amazon AthenaSchedule the pipeline: Use Glue Workflows and Triggers for repeatable runsMore than isolated service demonstrationsYou will build the project inside your own AWS account using the provided source code, healthcare sample data, configuration files, and infrastructure setup scripts.The focus is not only on getting the pipeline to run. You will understand why each AWS service is used, how the components work together, and how the pipeline handles new and updated source records during subsequent runs.By the end of the course, you will have a complete AWS data engineering project that you can practice, adapt for your portfolio, and explain clearly during interviews.This course contains a promotion.