Kestra
Declarative orchestration for data, infrastructure, and business workflows
Apache Airflow
Python-defined workflows with scheduling, dependencies, and execution monitoring
Meet Apache Airflow
Apache Airflow is an open-source platform for building, scheduling, and monitoring workflows. Developers describe tasks and their dependencies in Python, making it possible to version pipeline definitions alongside other code and generate workflows dynamically. Operators and provider packages connect those tasks to cloud services, databases, and existing infrastructure.
The web interface shows running and completed tasks, execution status, and logs, helping teams investigate failures and manage recurring work. A data team might use Airflow to move source data, run transformations in the right order, and prepare datasets for reporting or machine learning. Its extensible architecture supports custom operators and distributed workers as the number of workflows grows.
Discussion
Tried it? Share a useful thought or ask a question.
Good conversations start with a hello.
Sign in to commentProduct details
- Website
- airflow.apache.org
- Status
- Launched
- Category
- Data & analytics
- Listed
Explore related topics
Similar products
Dagster
Build reliable data pipelines with asset-based orchestration
Meltano
Manage data pipelines as code with reusable open-source connectors
dltHub
Build Python data pipelines and run them on managed dltHub infrastructure
Matillion
Build and manage warehouse pipelines through visual tools and the Maia platform
Airbyte
Move data between sources and analytical systems