问题描述
[已解决]
我的错误是我没有意识到如果所有熵都是一种类型,那么熵就是0。因此,如果全部为正,则熵为0,如果全部为负,则熵也为零。如果等量的正负相等,则熵将为1。
My mistake was that I did not realise that entropy is 0 if all are of one type. Thus if all are positive, entropy is 0 and if all are negative it is zero as well. Entropy will be 1 if equal amount are positive and negative.
一个人获得负信息增益没有意义。
It does not make sense that one would get negative information gain.
但是根据此示例,我获得了负信息增益。
However based on this example I am getting a negative information gain.
以下是数据:
here is the data:
如果我计算了Humidity属性的信息增益,我会得到
And if I calculate the information gain on the Humidity attribute I get this:
很显然我在这里错过了一些东西。
Obviously I am missing something here.
编辑:
阐明我的理解。
To clarify how I understand it.
整个系统的熵定义为:
在这种情况下为:
每个属性的信息增益定义为:
And the information gain per atribute is defined as:
对于湿度,我计算得出:
Which for humidity I calculate to:
系统的熵-(1 / 4)湿度熵正常-(3/4)湿度熵高
Entropy of system - (1/4)Entropy of Humidity Normal - (3/4)Entropy of Humidity High
根据此Libre Office Calc:
As per this Libre Office Calc:
或者是我对属性信息获取公式的理解
Or is my understanding of the formula for information gain for an attribute incorrect?
推荐答案
首先,我假设您的 S 变量为 EnjoySport 。 (我想您可以更清楚地表达文字,顺便说一句。)
To begin with, I'm assuming your S variable is EnjoySport. (I think you could phrase the text more clearly, BTW.)
因此, S 的熵为0.8113,但这是最后一部分我同意。
So the entropy of S is 0.8113, but that's the last part with which I agree.
给定 Normal 的 S 的熵为0,因为它是确定性的。
The entropy of S given Normal is 0, as it is deterministic.
给定 High 的 S 的熵为0.91829583405448945,但是您需要将其乘以0.75,因为这是 Normal的概率。这样就可以得到0.68872187554086706。
The entropy of S given High is 0.91829583405448945, but you need to multiply that by 0.75, because that is the probability of Normal. So that gives you 0.68872187554086706.
差异与预期的一样是非负的。
The difference is non-negative, as expected.
请注意,信息收益是,并且期望需要考虑条件事件的概率。
Note that the Information gain is the expected difference in Entropy, and the expectation needs to take into account the probability of the conditioned event.
这篇关于为什么我获得负面信息收益?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!