Airflow: PythonOperator: why to include 'ds' arg? -


while defining function later used python_callable, why 'ds' included first arg of function?

for example:

def python_func(ds, **kwargs):     pass 

i looked airflow documentation, not find explanation.

this related provide_context=true parameter. per airflow documentation,

if set true, airflow pass set of keyword arguments can used in function. set of kwargs correspond can use in jinja templates. work, need define **kwargs in function header.

ds 1 of these keyword arguments , represents execution date in format "yyyy-mm-dd". parameters marked (templated) in documentation, can use '{{ ds }}' default variable pass execution date. can read more default variables here: https://pythonhosted.org/airflow/code.html?highlight=pythonoperator#default-variables

pythonoperator doesn't have templated parameters, doing python_callable=print_execution_date('{{ ds }}') won't work. print execution date inside callable function of pythonoperator, have as

def print_execution_date(ds, **kwargs): print(ds)

or

def print_execution_date(**kwargs): print(kwargs.get('ds'))

hope helps.


Comments

Popular posts from this blog

aws api gateway - SerializationException in posting new Records via Dynamodb Proxy Service in API -

asp.net - Problems sending emails from forum -