openNLP对内容进行归类始终返回第一类

问题描述

我正在使用openNLP库进行测试，以实现内容分类的自动化，但遇到了麻烦.我正在使用此代码，它始终返回训练数据中传递的第一类，我正在从任何新闻站点传递全文.

I'm testing with openNLP library to implemented automation in categorizing content but i have trouble. I'm using this code and it returns always the first category that i have in my training data which i'm passing full article from any news site.

    public void trainModel() {
        try {
            InputStreamFactory inputStreamFactory = new MarkableFileInputStreamFactory( new File("C:\\Users\\emehm\\Desktop\\data\\training_data.txt") );
            ObjectStream<String> lineStream = new PlainTextByLineStream(inputStreamFactory, "UTF-8");
            ObjectStream<DocumentSample> sampleStream = new DocumentSampleStream(lineStream);

            DoccatModel model = DocumentCategorizerME.train("en", sampleStream, TrainingParameters.defaultParams(), new DoccatFactory());
            DocumentCategorizerME myCategorizer = new DocumentCategorizerME(model);
            double[] outcomes = myCategorizer.categorize(  new String[]{ this.getFileContent() });
            String category = myCategorizer.getBestCategory(outcomes);
            Map<String, Double> map = myCategorizer.scoreMap(new String[]{ this.getFileContent() });
            System.out.println(category);
        } catch (IOException e) {
            // Failed to read or parse training data, training failed
            e.printStackTrace();
        }
    }

    public String getFileContent() throws IOException {
        InputStream is = new FileInputStream("C:\\Users\\emehm\\Desktop\\data\\statija.txt");
        BufferedReader buf = new BufferedReader(new InputStreamReader(is));
        String line = buf.readLine();
        StringBuilder sb = new StringBuilder();
        while (line != null) {
            sb.append(line).append("\n");
            line = buf.readLine();
        }
        buf.close();
        return sb.toString();
    }

培训数据: http://pastebin.com/ZhxswkvJ

我正在使用的文章: http://pastebin.com/xtABGcbh

它总是返回列表中的第一个类别，我想知道我所缺少的是什么?当我调试它时，所有这些都返回0.25分，并且出于某种原因选择它们中的第一个.当我测试一个单词时，我猜它可以工作，但不适用于文章.

it always returns the the first category from the list and i want to know what i'm missing? when i debug it it returns 0.25 score for all of them and picks first of them for some reason. when i test one word it works i guess but it's not working with an article.

推荐答案

输入需要分为单个单词，即按空格分隔.

Input needs to be divided into individual words, ie split by spaces.

更改此内容:double[] outcomes = myCategorizer.categorize( new String[]{ this.getFileContent() });

对此:double[] outcomes = myCategorizer.categorize( this.getFileContent().split(" ") );

之后，您应该会得到更好的结果.重要的是要注意有效性与模型的质量有关.

After, you should have better results. It's important to note the effectiveness is tied to the quality of the model.

这篇关于openNLP对内容进行归类始终返回第一类的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！