Why it matters

Glue for serverless ETL. Understanding shapes AWS data pipelines.

Advertisement

The architecture

Crawlers: discover schemas.

Catalog: unified metadata.

Jobs: Spark-based ETL.

Glue architectureCrawlersschema discoveryData Catalogunified metadataGlue JobsSpark ETLGlue Studio for visual authoring; PySpark for custom
Glue components.
Advertisement

How it works end to end

Catalog: shared with Athena, Redshift Spectrum, EMR.

Jobs: Spark + Python shell.

Glue Studio: visual authoring.

Cost: DPU-hours.