问题描述
我想将Spring与将在Google Cloud Data Flow Runner上运行的Apache Beam结合使用.在执行管道步骤时,数据流作业应该能够使用Spring Runtime应用程序上下文.我想在我的Apache Beam管道中使用Spring功能来实现DI和其他功能.在Google上浏览了数小时后,我找不到任何显示Spring在Apache Beam中集成的帖子或文档.因此,如果有人尝试过使用Apache Beam进行弹簧加工,请告诉我.
I want to use Spring with Apache Beam that will run on Google Cloud Data flow Runner. Dataflow job should be able to use Spring Runtime application context while executing the Pipeline steps. I want to use Spring feature in my Apache Beam pipeline for DI and other stuff. After browsing hours on google, I couldn't find any post or documentation which shows Spring integration in Apache Beam. So, if anyone has tried spring with Apache beam, please let me know.
在主类中,我已经初始化了spring应用程序上下文,但是在执行管道步骤时不可用.对于自动装配的豆,我得到了空指针异常.我想问题是,在运行时上下文不适用于工作线程.
In main class i have initialised the spring application context but it is not available while execution of pipeline steps. I get null pointer exception for autowired beans. I guess the problem is, at runtime context is not available to worker threads.
public static void main(String[] args) {
initSpringApplicationContext();
GcmOptions options = PipelineOptionsFactory.fromArgs(args)
.withValidation()
.as(GcmOptions.class);
Pipeline pipeline = Pipeline.create(options);
// pipeline definition
}
我想将spring应用程序上下文注入每个ParDo函数.
I want to inject the spring application context to each of the ParDo functions.
推荐答案
这里的问题是ApplicationContext在任何工作程序上都不可用,因为 main
方法仅在构造作业时调用,并且不在任何工作计算机上.因此,永远不会在任何worker上调用 initSpringApplicationContext
.
The problem here is that the ApplicationContext is not available on any worker, as the main
method is only called when constructing the job and not on any worker machine. Therefore, initSpringApplicationContext
is never called on any worker.
我从没尝试过在Apache Beam中使用Spring,但是我猜想在静态初始值设定项块中移动 initSpringApplicationContext
会导致您期望的结果.
I've never tried to use Spring within Apache Beam, but I guess moving initSpringApplicationContext
in a static initializer block will lead to your expected result.
public class ApplicationContextHolder {
private static final ApplicationContext CTX;
static {
CTX = initApplicationContext();
}
public static ApplicationContext getContext() {
return CTX;
}
}
请注意,仅此一项不应被视为在Apache Beam中使用Spring的最佳实践,因为它在Apache Beam的生命周期中无法很好地集成.例如,当在应用程序上下文初始化期间发生错误时,错误将出现在使用 ApplicationContextHolder
的第一位置.因此,我建议从静态初始化程序块中提取 initApplicationContext
,并就Apache Beam的生命周期进行显式调用.设置
这篇关于Spring与Apache Beam的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!