Manual reports take days to refresh — how do you keep dashboards live without emailing spreadsheets?
Per Wikipedia: “Report automation schedules ETL pipelines to refresh fact and dimension tables and rebuild downstream reports on a cron or event trigger.”
Like an alarm that fills your fridge before breakfast — pipeline runs nightly so dashboard is fresh in morning.
- Orchestrator triggers ETL at schedule
- Extract pulls from sources
- Transform joins facts/dims
- Load into warehouse star schema
- BI tool refreshes dashboard
- Reduces manual error
- Requires idempotent loads
- Needs monitoring for late data
- Often built with Airflow
from airflow import DAG
with DAG('daily_report'): pass # schedule ETL- Built from: ETL Pipeline — automation wraps ETL
- Built from: Star Schema — loads into star schema
- Related: Apache Airflow — orchestrator
- Related: Data Warehouse — warehouse is target
- Backfills overwrite if not partitioned
- Silent schema drift breaks loads