没有这样的文件或目录/airflow/xcom/return.json

问题描述

创建一个包含 /airflow/xcom/return.json
的图像，所有子目录
上均带有chmod + x找不到文件或目录（尝试chmod + x）

created an image contain /airflow/xcom/return.json with chmod +x on all sub-dirSince the log show it cannot find file or directory(tried chmod +x)

strtpodbefore = KubernetesPodOperator(namespace='rguonew',
                                    image="mydockerimage",
                                    name="fail",
                                    image_pull_policy="Always",
                                    task_id="failing-task",
                                    get_logs= True,
                                    xcom_push=True,
                                    dag=dag
                                    )'

这是日志

[2019-03-18 20:32:07,007] {logging_mixin.py:95} INFO - [2019-03-18 
 20:32:07,007] {pod_launcher.py:166} INFO - Running command... cat 
/airflow/xcom/return.json
[2019-03-18 20:32:07,007] {logging_mixin.py:95} INFO - 
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - [2019-03-18 
20:32:07,026] {pod_launcher.py:173} INFO - cat: can't open 
 '/airflow/xcom/return.json': No such file or directory
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - 
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - [2019-03-18 
 20:32:07,026] {pod_launcher.py:166} INFO - Running command... kill -s 
 SIGINT 1
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - 
[2019-03-18 20:32:07,067] {models.py:1788} ERROR - Pod Launching 
failed: Failed to extract xcom from pod: fail-e18e3dac

因此尝试过这种方法，但它意味着它是从外部而不是从图像中分配xcom json

So tried with this way it works, but it means its assigns the xcom json from outside but not from the image

 return_value = '{"foo": "bar"\n, "buzz": 2}'
 strtpodbefore = KubernetesPodOperator(namespace='rguonew',
                                image="python:3.6.6-stretch",
                                cmds=["bash", "-cx"],
                                name="fail",
                                task_id="failing-task",
                                arguments=['echo \'{}\' > 
 /airflow/xcom/return.json'.format(return_value)],
                                get_logs= True,
                                xcom_push=True,
                                dag=dag
                                )

所以我尝试通过额外的参数尝试了最终解决方案，但仍然无法正常工作，第一个命令没有得到这样的目录return

so i tried the final solution with doing an extra argument but still doesnt work, the first command get no such directory return

strtpodbefore = KubernetesPodOperator(namespace='rguonew',
                                    image="myimages",
                                    name="fail",
                                    image_pull_policy="Always",
                                    cmds=["bash", "-cx"],
                                    arguments=['echo \'{}\' > 
/airflow/xcom/return.json'.format(return_value)],
                                    task_id="failing-task",
                                    get_logs= True,
                                    xcom_push=True,
                                    dag=dag
                                    )

推荐答案

设置xcom_push = True时，将创建一个sidecar容器以及您的执行者容器。边车容器将读取执行者的 /airflow/xcom/return.json ，因此您实际上需要像第二个示例中那样从执行者容器中写入内容。

When you set the xcom_push=True, a sidecar container will be created along with your executor container. The sidecar container will read the executor's /airflow/xcom/return.json, so you actually need to write to this from the executor container as you have done in the 2nd example.

请参见此处：

在处理pod xcom时，还需要设置一些与RBAC相关的问题：

When we were working on our pod xcom, there were also some issues related to RBAC that you need to set: Airflow k8s operator xcom - Handshake status 403 Forbidden

这篇关于没有这样的文件或目录/airflow/xcom/return.json的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！