使用jaccard相似度的Python Pandas距离矩阵

问题描述

我已经实现了使用jaccard相似度构造距离矩阵的功能:

I have implemented a function to construct a distance matrix using the jaccard similarity:

import pandas as pd
entries = [
    {'id':'1', 'category1':'100', 'category2': '0', 'category3':'100'},
    {'id':'2', 'category1':'100', 'category2': '0', 'category3':'100'},
    {'id':'3', 'category1':'0', 'category2': '100', 'category3':'100'},
    {'id':'4', 'category1':'100', 'category2': '100', 'category3':'100'},
    {'id':'5', 'category1':'100', 'category2': '0', 'category3':'100'}
           ]
df = pd.DataFrame(entries)

和带有scipy的距离矩阵

and the distance matrix with scipy

from scipy.spatial.distance import squareform
from scipy.spatial.distance import pdist, jaccard

res = pdist(df[['category1','category2','category3']], 'jaccard')
squareform(res)
distance = pd.DataFrame(squareform(res), index=df.index, columns= df.index)

问题是我的结果看起来像这样，这似乎是错误的:

The problem is that my result looks like this which seems to be false:

我想念什么?例如，0和1的相似性必须最大，而其他值也似乎是错误的

What am i missing? The similarity of 0 and 1 have to be maximum for example and the other values seem wrong too

推荐答案

查看文档，实现是jaccard 不相似，而不是相似性.这是使用jaccard作为度量标准时计算距离的常用方法.这样做的原因是，要成为度量标准，相同点之间的距离必须为零.

Looking at the docs, the implementation of jaccard in scipy.spatial.distance is jaccard dissimilarity, not similarity. This is the usual way in which distance is computed when using jaccard as a metric. The reason for this is because in order to be a metric, the distance between the identical points must be zero.

在您的代码中，应最小化0和1之间的差异.在不相似的情况下，其他值也看起来是正确的.

In your code, the dissimilarity between 0 and 1 should be minimized, which it is. The other values look correct in the context of dissimilarity as well.

如果您要相似而不是不相似，只需从1中减去不相似即可.

If you want similarity instead of dissimilarity, just subtract the dissimilarity from 1.

res = 1 - pdist(df[['category1','category2','category3']], 'jaccard')

这篇关于使用jaccard相似度的Python Pandas距离矩阵的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！