Skip to main content

Orchestration

Scheduling​

Arch supports schedules for pipelines and transformations. After creating either of these tasks you can select a schedule using an interval (e.g. daily, weekly, etc.) or for more complex cases you can using cron syntax.

Interval​

Interval

Cron​

Cron

Schedule Dependencies​

The ability to define dependencies (e.g. run a transform after each successful pipeline run.) between tasks is currently on the roadmap.

External Orchestration​

For more complex orchestration needs, the Arch backend has a well documented publicly accessible API that can be orchestrated externally from standalone orchestrators like Airflow, Dagster, etc. if needed. Reach out to the Arch support team to get access.