本文介绍了addng ReadAllFromText转换时管道失败的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我正在尝试在Apache Beam中运行一个非常简单的程序,以尝试其工作原理.
I am trying to run a very simple program in Apache Beam to try out how it works.
import apache_beam as beam
class Split(beam.DoFn):
def process(self, element):
return element
with beam.Pipeline() as p:
rows = (p | beam.io.ReadAllFromText(
"input.csv") | beam.ParDo(Split()))
运行此程序时,出现以下错误
While running this, I get the following errors
.... some more stack....
File "/home/raheel/code/beam-practice/lib/python2.7/site-packages/apache_beam/transforms/util.py", line 565, in expand
windowing_saved = pcoll.windowing
File "/home/raheel/code/beam-practice/lib/python2.7/site-packages/apache_beam/pvalue.py", line 137, in windowing
self.producer.inputs)
File "/home/raheel/code/beam-practice/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py", line 464, in get_windowing
return inputs[0].windowing
File "/home/raheel/code/beam-practice/lib/python2.7/site-packages/apache_beam/pvalue.py", line 137, in windowing
self.producer.inputs)
File "/home/raheel/code/beam-practice/lib/python2.7/site-packages/apache_beam/transforms/ptransform.py", line 464, in get_windowing
return inputs[0].windowing
AttributeError: 'PBegin' object has no attribute 'windowing'
任何主意这里有什么问题吗?
Any Idea what is wrong here ?
谢谢
推荐答案
ReadAllFromText
希望从文件的PCollection中读取而不是将其作为参数传递.因此,在您的情况下,应该是:
ReadAllFromText
expects to read from a PCollection of files instead of passing it as an argument. So, in your case, it should be:
p | beam.Create(["input.csv"])
| beam.io.ReadAllFromText()
这篇关于addng ReadAllFromText转换时管道失败的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!