python - 无法从 NLTK 库导入 Bigrams

让我困惑的快速问题。我已经安装了 NLTK，它一直工作正常。但是，我正在尝试获取语料库的 bigrams 并基本上想使用 bigrams(corpus) .. 但它说当我“从 nltk 导入 bigrams”时未定义 bigrams

与三元组相同。我错过了什么吗？另外，我怎么能手动从语料库中获取 bigrams。

我也想计算二元三元组和四元组的频率，但我不确定如何解决这个问题。

我在开头和结尾适本地用 "<s>" 和 "</s>" 标记了语料库。到目前为止的程序:

 #!/usr/bin/env python
import re
import nltk
import nltk.corpus as corpus
import tokenize
from nltk.corpus import brown

def alter_list(row):
    if row[-1] == '.':
        row[-1] = '</s>'
    else:
        row.append('</s>')
    return ['<s>'] + row

news = corpus.brown.sents(categories = 'editorial')
print len(news),'\n'

x = len(news)
for row in news[:x]:
    print(alter_list(row))

最佳答案

我在 virtualenv 中对此进行了测试，并且可以正常工作:

In [20]: from nltk import bigrams

In [21]: bigrams('This is a test')
Out[21]:
[('T', 'h'),
 ('h', 'i'),
 ('i', 's'),
 ('s', ' '),
 (' ', 'i'),
 ('i', 's'),
 ('s', ' '),
 (' ', 'a'),
 ('a', ' '),
 (' ', 't'),
 ('t', 'e'),
 ('e', 's'),
 ('s', 't')]

这是您遇到的唯一错误吗？

顺便说一下，至于你的第二个问题:

from collections import Counter
In [44]: b = bigrams('This is a test')

In [45]: Counter(b)
Out[45]: Counter({('i', 's'): 2, ('s', ' '): 2, ('a', ' '): 1, (' ', 't'): 1, ('e', 's'): 1, ('h', 'i'): 1, ('t', 'e'): 1, ('T', 'h'): 1, (' ', 'i'): 1, (' ', 'a'): 1, ('s', 't'): 1})

对于单词:

In [49]: b = bigrams("This is a test".split(' '))

In [50]: b
Out[50]: [('This', 'is'), ('is', 'a'), ('a', 'test')]

In [51]: Counter(b)
Out[51]: Counter({('is', 'a'): 1, ('a', 'test'): 1, ('This', 'is'): 1})

这种按词拆分显然是非常肤浅的，但根据您的应用程序，它可能就足够了。显然，您可以使用 nltk 的标记化，它要复杂得多。

为了实现您的最终目标，您可以执行以下操作:

In [56]: d = "Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum."

In [56]: from nltk import trigrams
In [57]: tri = trigrams(d.split(' '))

In [60]: counter = Counter(tri)

In [61]: import random

In [62]: random.sample(counter, 5)
Out[62]:
[('Ipsum', 'has', 'been'),
 ('industry.', 'Lorem', 'Ipsum'),
 ('Ipsum', 'passages,', 'and'),
 ('was', 'popularised', 'in'),
 ('galley', 'of', 'type')]

我修剪了输出，因为它不必要地大，但你明白了。

关于python - 无法从 NLTK 库导入 Bigrams，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/13080301/