本文介绍了Google Dataflow 作业和 BigQuery 在不同区域失败的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我有一个 Google Dataflow 作业失败:
I have a Google Dataflow job that is failing on:
BigQuery job ... finished with error(s): errorResult:
Cannot read and write in different locations: source: EU, destination: US, error: Cannot read and write in different locations: source: EU, destination: US
我开始工作--zone=europe-west1-b
这是管道中唯一可以使用 BigQuery 执行任何操作的部分:
And this is the only part of the pipeline that does anything with BigQuery:
Pipeline p = Pipeline.create(options);
p.apply(BigQueryIO.Read.fromQuery(query));
我正在阅读的 BigQuery 表的详细信息如下:Data Location EU
The BigQuery table I'm reading from has this in the details: Data Location EU
当我在本地运行作业时，我得到:
When I run the job locally, I get:
SEVERE: Error opening BigQuery table dataflow_temporary_table_339775 of dataset _dataflow_temporary_dataset_744662 : 404 Not Found
如果我只是在读取数据，我不明白为什么它会尝试写入不同的位置.而且即使需要创建临时表，为什么要创建在不同的区域?
I don't understand why it is trying to write to a different location if I'm only reading data. And even if it needs to create a temporary table, why is it being created in a different region?
有什么想法吗?
推荐答案
我建议验证:
- 如果 Google 数据流的暂存位置在同一区域中.
- 如果 Dataflow 中使用的 Google Cloud Storage 位置也在同一地区.
这篇关于Google Dataflow 作业和 BigQuery 在不同区域失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！