问题描述
好的，我有以下代码来训练来自 OpenNLP 的 NER 标识符
Ok, I have the following code to train the NER Identifier from OpenNLP
FileReader fileReader = new FileReader("train.txt");
ObjectStream fileStream = new PlainTextByLineStream(fileReader);
ObjectStream sampleStream = new NameSampleDataStream(fileStream);
TokenNameFinderModel model = NameFinderME.train("pt-br", "train", sampleStream, Collections.<String, Object>emptyMap());
nfm = new NameFinderME(model);
我不知道我是不是做错了什么或者是缺少什么，但是分类不起作用.我假设 train.txt 是错误的.
I don't know if I'm doing something wrong of if something is missing, but the classifying is not working. I'm supposing that the train.txt is wrong.
出现的错误是所有的token都被归为一种类型.
The error that occurs is that all tokens are classified to only one type.
我的 train.txt 数据类似于以下示例，但具有更多变化和条目数量.另一件事是，我每次都从文本中逐字分类查找，而不是所有标记.
My train.txt data is something like the following example, but with a lot more of variation and quantity of entries. Another thing is that I'm classifind word by word from a text per time, and not all tokens.
<START:distance> 8000m <END>
<START:temperature> 100ºC <END>
<START:weight> 50kg <END>
<START:name> Renato <END>
谁能告诉我我做错了什么?
Somebody can show what I doing wrong?
推荐答案
你的训练数据不正常.
您应该将所有实体放在一个句子中的上下文中:
You should put all entities in a context inside a sentence:
At an altitude of <START:distance> 8000m <END> the temperature of boiling water is less than <START:temperature> 100ºC <END> .
The climber <START:name> Renato <END> is carrying <START:weight> 50kg <END> of equipment.
如果您的训练数据来自真实世界的句子并且与您正在分类的句子具有相同的风格，那么您将获得更好的结果.例如，如果您要处理新闻，您应该使用报纸语料库进行训练.
You will have better results if your training data derives from real world sentences and have the same style of the sentences you are classifying. For example you should train using a newspaper corpus if you will process news.
此外，您还需要数千个句子来构建模型！也许你可以从一百个开始引导并使用糟糕的模型来改进你的语料库并再次训练你的模型.
Also you will need thousands of sentences to build your model! Maybe you can start with a hundred to bootstrap and use the poor model to improve your corpus and train your model again.
当然你应该对句子的所有标记进行分类，否则将没有上下文来决定实体的类型.
And of course you should classify all tokens of a sentence, otherwise there will be no context to decide the type of an entity.
这篇关于我如何在 OpenNLP 中训练命名实体识别器标识符?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！