Apache Airflow
DAG-based workflow orchestration
What it does
Airflow defines task dependencies in Python and uses schedulers and executors to run directed acyclic graphs.
When to use it
For a batch pipeline spanning several sources, inspect execution history, retries, and downstream tasks in its web interface.
