问题描述
我正在尝试从荷兰文字中提取命名实体.我使用 nltk-trainer 来在conll2002荷兰语料库上训练标记器和分块器.但是，来自分块器的parse方法未检测到任何命名实体.这是我的代码:
I am trying to extract named entities from dutch text. I used nltk-trainer to train a tagger and a chunker on the conll2002 dutch corpus. However, the parse method from the chunker is not detecting any named entities. Here is my code:
str = 'Christiane heeft een lam.'
tagger = nltk.data.load('taggers/dutch.pickle')
chunker = nltk.data.load('chunkers/dutch.pickle')
str_tags = tagger.tag(nltk.word_tokenize(str))
print str_tags
str_chunks = chunker.parse(str_tags)
print str_chunks
该程序的输出:
[('Christiane', u'N'), ('heeft', u'V'), ('een', u'Art'), ('lam', u'Adj'), ('.', u'Punc')]
(S Christiane/N heeft/V een/Art lam/Adj ./Punc)
我希望克里斯蒂安妮被视作一个命名实体.有帮助吗?
I was expecting Christiane to be detected as a named entity.Any help?
推荐答案
conll2002语料库同时具有西班牙语和荷兰语文本，因此应确保使用fileids参数，如python train_chunker.py conll2002 --fileids ned.train所示.西班牙语和荷兰语的培训都不会取得很好的效果.
The conll2002 corpus has both spanish and dutch text, so you should make sure to use the fileids parameter, as in python train_chunker.py conll2002 --fileids ned.train. Training on both spanish and dutch will have poor results.
默认算法是基于Tagger的Chunker，在conll2002上效果不佳.而是使用像NaiveBayes这样的基于分类器的分块器，这样完整的命令可能看起来像这样(并且我已经确认生成的分块器确实将"Christiane"识别为"PER"):
The default algorithm is a Tagger based Chunker, which does not work well on conll2002. Instead, use a classifier based chunker like NaiveBayes, so the full command might look like this (and I've confirmed that the resulting chunker does recognize "Christiane" as a "PER"):
python train_chunker.py conll2002 --fileids ned.train --classifier NaiveBayes --filename ~/nltk_data/chunkers/conll2002_ned_NaiveBayes.pickle
这篇关于NLTK在荷兰命名实体识别的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！