问题描述
是否有一种方法可以对整个DagRun使用 depends_on_past
,而不仅仅是将其应用于Task?
Is there a way in airflow of using the depends_on_past
for an entire DagRun, not just applied to a Task?
我每天都有DAG,星期五DagRun在第四个任务上出错,但是星期六和星期日DagRun仍按计划运行。使用 depends_on_past = True
可以使DagRun在相同的第4个任务上暂停,但是前三个任务仍然可以运行。
I have a daily DAG, and the Friday DagRun errored on the 4th task however the Saturday and Sunday DagRuns still ran as scheduled. Using depends_on_past = True
would have paused the DagRun on the same 4th task, however the first 3 tasks would still have run.
我可以在DagRun数据库表中看到一个 state
列,其中包含星期五DagRun的失败
。我想要的是一种配置DagRun的方法,以使其在先前的DagRun失败时不启动,直到发现以前失败的Task才启动并运行。
I can see in the DagRun DB table there is a state
column that contains failed
for the Friday DagRun. What I want is a way configuring a DagRun to not start if the previous DagRun failed, not start and run until finding a Task that previously failed.
有人知道这是否可能吗?
Does anyone know if this is possible?
推荐答案
在您的身边第一个任务,设置 depends_on_past = True
和 wait_for_downstream = True
,组合将导致仅运行当前的dag运行如果最后一次运行成功。
At your first task, set depends_on_past=True
and wait_for_downstream=True
, the combination will result in that current dag-run runs only if the last run succeeded.
因为在当前dag-run设置第一个任务会等待之前的
(depends_on_past)和所有任务(wait_for_downstream)成功
Because by setting the first task at current dag-run would waits for previous (depends_on_past) and all tasks (wait_for_downstream) to succeed
这篇关于整个DAG的气流取决于_过去的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!