使用 SimLex-999 和 wordsim353 评估 word2vec

问题描述

我已经使用 SimLex-999 和 wordsim353 评估了我的模型，但我不知道结果是否正常?

I have evaluated my model with SimLex-999 and wordsim353 but i don't know if the result is ok or not?

wordsim353 结果

wordsim353 result

 Pearson correlation coefficient against C:\ProgramData\Anaconda3\lib\site-packages\gensim\test\test_data\wordsim353.tsv: 0.4895
2019-08-27 08:30:06,655 : INFO : Spearman rank-order correlation coefficient against C:\ProgramData\Anaconda3\lib\site-packages\gensim\test\test_data\wordsim353.tsv: 0.4799
2019-08-27 08:30:06,656 : INFO : Pairs with unknown words ratio: 7.1%

((0.4894983099817645, 3.6324947252392034e-21), SpearmanrResult(correlation=0.4798812637344527, pvalue=2.6991867797169835e-20), 7.0821529745042495)

SimLex-999 结果

SimLex-999 result

 2019-08-27 15:43:13,000 : INFO : Pearson correlation coefficient against C:\ProgramData\Anaconda3\lib\site-packages\gensim\test\test_data\simlex999.txt: 0.3138
    2019-08-27 15:43:13,001 : INFO : Spearman rank-order correlation coefficient against C:\ProgramData\Anaconda3\lib\site-packages\gensim\test\test_data\simlex999.txt: 0.2992
    2019-08-27 15:43:13,002 : INFO : Pairs with unknown words ratio: 1.2%
    ((0.31381174440491943, 5.375150591505246e-24), SpearmanrResult(correlation=0.29915866880742126, pvalue=7.433265418805336e-22), 1.2012012012012012)

推荐答案

只有您可以决定结果是否适合您的目的.在将一个模型与另一个模型进行比较时，这些类型的分数最有意义，作为关于其他变化(新参数、新预处理、更多/不同数据)是有益还是有害的粗略指南.

Only you can decide whether a result is adequate for your purposes. These kinds of scores are most meaningful when comparing one model against another, as a rough guide as to whether other changes – new parameters, new preprocessing, more/different data – are helping or hurting.

您可以查看介绍您正在使用的评估数据集的论文，以了解如何解释分数:

You could look at the paper introducing the evaluation dataset you're using to see how to interpret the scores:

https://arxiv.org/abs/1408.3456v1

您还可以下载一些现成的词向量集，检查它们的评估分数，与您的进行比较.

You could also download some off-the-shelf word-vector sets, checking their evaluation scores, to compare with yours.

这篇关于使用 SimLex-999 和 wordsim353 评估 word2vec的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！