Azure数据工厂:处理直到/活动中的内部故障

问题描述

我有一个包含直到活动的Azure数据工厂v2管道.

I have an Azure data factory v2 pipeline containing an until activity.

在直到内是一个复制活动-如果失败，则记录错误，与本文中的记录完全一样，并且我希望循环继续进行.

Inside the until is a copy activity - if this fails, the error is logged, exactly as in this post, and I want the loop to continue.

Azure数据工厂管道出现故障"

尽管处理了内部复制活动的错误，但由于内部活动已失败，因此直到活动被视为失败.

Although the inner copy activity’s error is handled, the until activity is deemed to have failed because an inner activity has failed.

当内部活动失败时，是否有任何方法可以配置直到活动才能继续?

Is there any way to configure the until activity to continue when an inner activity fails?

推荐答案

解决方案

将错误处理步骤放在自己的管道中，然后从ExecutePipeline活动中运行它们.您需要传入外部管道所需的所有参数.

Put the error-handling steps in their own pipeline and run them from an ExecutePipeline activity. You'll need to pass-in all the parameters required from the outer pipeline.

然后，您可以使用ExecutePipeline中的完成(蓝色)依赖项(而不是成功(绿色))，因此尽管存在内部错误，外部管道仍可以继续运行.

You can then use the completion (blue) dependency from the ExecutePipeline (rather than success (green)) so the outer pipeline continues to run despite the inner error.

请注意，如果您希望外部知道内部发生了什么，那么当前无法将数据从ExecutePipeline传递到其父级( https://feedback.azure.com/forums/270578-data- factory/suggestions/38690032-add-ability-to-customize-output-fields-from-execut ).

Note that if you want the outer to know what happened in the inner then there is currently no way to pass data out of the ExecutePipeline to its parent (https://feedback.azure.com/forums/270578-data-factory/suggestions/38690032-add-ability-to-customize-output-fields-from-execut).

要解决此问题，请在ExecutePipeline中使用sp活动将数据写入到SQL表中，该表由管道运行ID标识.可以在管道内部使用@pipeline().RunId引用.

To solve this, use an sp activity inside the ExecutePipeline to write data to a SQL table, identified with the pipeline run id. This can be referenced inside the pipeline with @pipeline().RunId.

然后，您可以在管道外使用运行ID在SQL表中进行查询，以获取正确的行.

Then outside the pipeline you can do a lookup in the SQL table, using the run ID to get the right row.

健康警告:

出于某些奇怪的原因，ExecutePipeline的输出不是作为JSON对象而是作为字符串返回.因此，如果您尝试选择类似@activity('ExecutePipelineActivityName').output.something这样的输出属性，则会出现此错误:

For some weird reason, the output of ExecutePipeline is returned not as a JSON object but as a string. So if you try to select a property of output like this @activity('ExecutePipelineActivityName').output.something then you get this error:

字符串"类型的值不支持属性选择

因此，要从外部获取ExecutePipeine的运行ID，您需要:@json(activity('ExecutePipelineActivityName').output).pipelineRunId

So, to get the ExecutePipeine's run ID from outside you need:@json(activity('ExecutePipelineActivityName').output).pipelineRunId

我在任何地方的Microsoft文档中都找不到该文档，因此在此处发布了血腥的细节.

I couldn't find this documented in Microsoft's documentation anywhere, hence posting gory details here.

这篇关于Azure数据工厂:处理直到/活动中的内部故障的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！