我正在尝试模拟某些文档的流传输,并在流进的其他文档上更新LSI。我发现此错误:
Traceback (most recent call last):
File "gensimStreamGen_tutorial5.py", line 57, in <module>
for vector in corpus_memory_friendly: # load one vector into memory at a time
File "gensimStreamGen_tutorial5.py", line 44, in __iter__
lsi = models.LsiModel(corpus, num_topics=10) # initialize an LSI transformation
File "/Users/Desktop/gensim-0.12.0/gensim/models/lsimodel.py", line 331, in __init__
self.add_documents(corpus)
File "/Users/Desktop/gensim-0.12.0/gensim/models/lsimodel.py", line 388, in add_documents
update = Projection(self.num_terms, self.num_topics, job, extra_dims=self.extra_samples, power_iters=self.power_iters)
File "/Users/Desktop/gensim-0.12.0/gensim/models/lsimodel.py", line 126, in __init__
extra_dims=self.extra_dims)
File "/Users/Desktop/gensim-0.12.0/gensim/models/lsimodel.py", line 677, in stochastic_svd
q, _ = matutils.qr_destroy(y) # orthonormalize the range
File "/Users/Desktop/gensim-0.12.0/gensim/matutils.py", line 398, in qr_destroy
qr, tau, work, info = geqrf(a, lwork=-1, overwrite_a=True)
ValueError: failed to create intent(cache|hide)|optional array-- must have defined dimensions but got (0,)
流文档和更新LSI模型的代码:
class MyCorpus(object):
def __iter__(self):
for document in documents:
# Stream-in documents and build TF-IDF model to construct new_vec
yield new_vec
corpus.append(new_vec)
tfidf = models.TfidfModel(corpus)
corpus_tfidf = tfidf[corpus]
lsi = models.LsiModel(corpus_tfidf, num_topics=2)
corpus_lsi = lsi[corpus_tfidf]
lsi.print_topics(2)
for doc in corpus_lsi:
print(doc)
corpus_memory_friendly = MyCorpus()
for vector in corpus_memory_friendly:
print(vector)
语料库每次迭代都会获得一个新的new_vec。 new_vec对每个产量进行不同的迭代:
[]
[(0, 1)]
[(1, 1), (2, 1), (3, 1)]
[(3, 2), (4, 1), (5, 1)]
[(2, 1), (6, 1), (7, 1)]
[]
[(8, 1)]
[(8, 1), (9, 1)]
[(9, 1), (10, 1), (11, 1)]
该错误出现在第一次迭代中(预期的new_vec中的第一行)。其余的是new_vec的预期输出。
最佳答案
我认为因为您的文件中的数据有空白
尝试添加
if(document!=[]and document!=[[]])
关于python - Gensim:ValueError:无法创建intent(cache | hide)| Optional数组-必须具有已定义的尺寸,但得到(0,),我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/31512853/