本文介绍了气流:PythonOperator:为什么要包含'ds'arg?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

在定义一个函数以供以后用作python_callable时,为什么将'ds'作为函数的第一个参数?

While defining a function to be later used as a python_callable, why is 'ds' included as the first arg of the function?

例如:

def python_func(ds, **kwargs):
    pass

我查看了Airflow文档,但找不到任何解释。

I looked into the Airflow documentation, but could not find any explanation.

推荐答案

这与 provide_context = True 参数有关。根据Airflow文档,

This is related to the provide_context=True parameter. As per Airflow documentation,

ds 是这些关键字参数之一,并以 YYYY-MM-DD。对于在文档中标记为(模板)的参数,可以使用’{{ds}}’默认变量来传递执行日期。您可以在此处阅读有关默认变量的更多信息:

ds is one of these keyword arguments and represents execution date in format "YYYY-MM-DD". For parameters that are marked as (templated) in the documentation, you can use '{{ ds }}' default variable to pass the execution date. You can read more about default variables here:

(已淘汰)

PythonOperator没有模板化参数,所以要做类似

PythonOperator doesn't have templated parameters, so doing something like

python_callable=print_execution_date('{{ ds }}')

不起作用。要在PythonOperator的可调用函数中打印执行日期,您将必须执行以下操作:

won't work. To print execution date inside the callable function of your PythonOperator, you will have to do it as

def print_execution_date(ds, **kwargs):
    print(ds)

def print_execution_date(**kwargs):
    print(kwargs.get('ds'))

希望这会有所帮助。

这篇关于气流:PythonOperator:为什么要包含'ds'arg?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

08-05 10:30