我知道如何使用nltk得到bigram和trigram的搭配,并将它们应用到我自己的语料库中。代码如下。
但是,我不确定(1)如何得到一个特定单词的搭配?(2)NLTK是否有基于对数似然比的配置度量?
import nltk
from nltk.collocations import *
from nltk.tokenize import word_tokenize
text = "this is a foo bar bar black sheep foo bar bar black sheep foo bar bar black sheep shep bar bar black sentence"
trigram_measures = nltk.collocations.TrigramAssocMeasures()
finder = TrigramCollocationFinder.from_words(word_tokenize(text))
for i in finder.score_ngrams(trigram_measures.pmi):
print i
最佳答案
尝试此代码:
import nltk
from nltk.collocations import *
bigram_measures = nltk.collocations.BigramAssocMeasures()
trigram_measures = nltk.collocations.TrigramAssocMeasures()
# Ngrams with 'creature' as a member
creature_filter = lambda *w: 'creature' not in w
## Bigrams
finder = BigramCollocationFinder.from_words(
nltk.corpus.genesis.words('english-web.txt'))
# only bigrams that appear 3+ times
finder.apply_freq_filter(3)
# only bigrams that contain 'creature'
finder.apply_ngram_filter(creature_filter)
# return the 10 n-grams with the highest PMI
print finder.nbest(bigram_measures.likelihood_ratio, 10)
## Trigrams
finder = TrigramCollocationFinder.from_words(
nltk.corpus.genesis.words('english-web.txt'))
# only trigrams that appear 3+ times
finder.apply_freq_filter(3)
# only trigrams that contain 'creature'
finder.apply_ngram_filter(creature_filter)
# return the 10 n-grams with the highest PMI
print finder.nbest(trigram_measures.likelihood_ratio, 10)
它使用可能性度量,并过滤掉不包含“生物”这个词的ngrams。
关于python - 针对特定单词的NLTK搭配,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/21165702/