Airflow: PythonOperator: why to include 'ds' arg? -
while defining function later used python_callable, why 'ds' included first arg of function?
for example:
def python_func(ds, **kwargs): pass
i looked airflow documentation, not find explanation.
this related provide_context=true
parameter. per airflow documentation,
if set true, airflow pass set of keyword arguments can used in function. set of kwargs correspond can use in jinja templates. work, need define **kwargs in function header.
ds
1 of these keyword arguments , represents execution date in format "yyyy-mm-dd". parameters marked (templated) in documentation, can use '{{ ds }}'
default variable pass execution date. can read more default variables here: https://pythonhosted.org/airflow/code.html?highlight=pythonoperator#default-variables
pythonoperator doesn't have templated parameters, doing python_callable=print_execution_date('{{ ds }}')
won't work. print execution date inside callable function of pythonoperator, have as
def print_execution_date(ds, **kwargs): print(ds)
or
def print_execution_date(**kwargs): print(kwargs.get('ds'))
hope helps.
Comments
Post a Comment