我正在使用 NLTK 测试情绪分析模型。我需要在分类器结果中添加一个混淆矩阵,如果可能的话,还需要添加 Precision、Recall 和 F-Measure 值。到目前为止,我只有准确性。 Movie_reviews 数据有 pos 和 neg 标签。然而,为了训练分类器,我使用了与通常的(句子、标签)结构不同的“特征集”。在通过“特征集”训练分类器后,我不确定是否可以使用 sklearn 中的混淆矩阵
import nltk
import random
from nltk.corpus import movie_reviews
documents = [(list(movie_reviews.words(fileid)), category)
for category in movie_reviews.categories()
for fileid in movie_reviews.fileids(category)]
random.shuffle(documents)
all_words = []
for w in movie_reviews.words():
all_words.append(w.lower())
all_words = nltk.FreqDist(all_words)
word_features = list(all_words.keys())[:3000]
def find_features(document):
words = set(document)
features = {}
for w in word_features:
features[w] = (w in words)
return features
featuresets = [(find_features(rev), category) for (rev, category) in documents]
training_set = featuresets[:1900]
testing_set = featuresets[1900:]
classifier = nltk.NaiveBayesClassifier.train(training_set)
print("Naive Bayes Algo accuracy percent:", (nltk.classify.accuracy(classifier, testing_set))*100)
最佳答案
首先,您可以对所有测试值进行分类,并将预测结果和黄金结果存储在一个列表中。
然后,您可以使用 nltk.ConfusionMatrix 。
test_result = []
gold_result = []
for i in range(len(testing_set)):
test_result.append(classifier.classify(testing_set[i][0]))
gold_result.append(testing_set[i][1])
现在,您可以计算不同的指标。
CM = nltk.ConfusionMatrix(gold_result, test_result)
print(CM)
print("Naive Bayes Algo accuracy percent:"+str((nltk.classify.accuracy(classifier, testing_set))*100)+"\n")
labels = {'pos', 'neg'}
from collections import Counter
TP, FN, FP = Counter(), Counter(), Counter()
for i in labels:
for j in labels:
if i == j:
TP[i] += int(CM[i,j])
else:
FN[i] += int(CM[i,j])
FP[j] += int(CM[i,j])
print("label\tprecision\trecall\tf_measure")
for label in sorted(labels):
precision, recall = 0, 0
if TP[label] == 0:
f_measure = 0
else:
precision = float(TP[label]) / (TP[label]+FP[label])
recall = float(TP[label]) / (TP[label]+FN[label])
f_measure = float(2) * (precision * recall) / (precision + recall)
print(label+"\t"+str(precision)+"\t"+str(recall)+"\t"+str(f_measure))
您可以检查 - 如何计算 精度和召回 here 。
您还可以使用: sklearn.metrics 进行这些使用 gold_result 和 test_result 值的计算。
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
print '\nClasification report:\n', classification_report(gold_result, test_result)
print '\nConfussion matrix:\n',confusion_matrix(gold_result, test_result)
关于scikit-learn - 混淆矩阵 - 测试情感分析模型,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/38541644/