我在二进制分类问题上没有得到期望的输出。
问题是使用二进制分类将乳腺癌标记为：
-良性，或
-恶性
它没有提供所需的输出。
首先，有一个函数可以加载返回测试并训练形状数据的数据集：
x_train is of shape: (30, 381),
y_train is of shape: (1, 381),
x_test is of shape: (30, 188),
y_test is of shape: (1, 188).
然后是用于逻辑回归分类器的类，该类可预测输出。
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np
def load_dataset():
cancer_data = load_breast_cancer()
x_train, x_test, y_train, y_test = train_test_split(cancer_data.data, cancer_data.target, test_size=0.33)
x_train = x_train.T
x_test = x_test.T
y_train = y_train.reshape(1, (len(y_train)))
y_test = y_test.reshape(1, (len(y_test)))
m = x_train.shape[1]
return x_train, x_test, y_train, y_test, m
class Neural_Network():
def __init__(self):
np.random.seed(1)
self.weights = np.random.rand(30, 1) * 0.01
self.bias = np.zeros(shape=(1, 1))
def sigmoid(self, x):
return 1 / (1 + np.exp(-x))
def train(self, x_train, y_train, iterations, m, learning_rate=0.5):
for i in range(iterations):
z = np.dot(self.weights.T, x_train) + self.bias
a = self.sigmoid(z)
cost = (-1 / m) * np.sum(y_train * np.log(a) + (1 - y_train) * np.log(1 - a))
if (i % 500 == 0):
print("Cost after iteration %i: %f" % (i, cost))
dw = (1 / m) * np.dot(x_train, (a - y_train).T)
db = (1 / m) * np.sum(a - y_train)
self.weights = self.weights - learning_rate * dw
self.bias = self.bias - learning_rate * db
def predict(self, inputs):
m = inputs.shape[1]
y_predicted = np.zeros((1, m))
z = np.dot(self.weights.T, inputs) + self.bias
a = self.sigmoid(z)
for i in range(a.shape[1]):
y_predicted[0, i] = 1 if a[0, i] > 0.5 else 0
return y_predicted
if __name__ == "__main__":
'''
step-1 : Loading data set
x_train is of shape: (30, 381)
y_train is of shape: (1, 381)
x_test is of shape: (30, 188)
y_test is of shape: (1, 188)
'''
x_train, x_test, y_train, y_test, m = load_dataset()
neuralNet = Neural_Network()
'''
step-2 : Train the network
'''
neuralNet.train(x_train, y_train,10000,m)
y_predicted = neuralNet.predict(x_test)
print("Accuracy on test data: ")
print(accuracy_score(y_test, y_predicted)*100)
该程序给出以下输出：
C:\Python36\python.exe C:/Users/LENOVO/PycharmProjects/MarkDmo001/Numpy.py
Cost after iteration 0: 5.263853
C:/Users/LENOVO/PycharmProjects/MarkDmo001/logisticReg.py:25: RuntimeWarning: overflow encountered in exp
return 1 / (1 + np.exp(-x))
C:/Users/LENOVO/PycharmProjects/MarkDmo001/logisticReg.py:33: RuntimeWarning: divide by zero encountered in log
cost = (-1 / m) * np.sum(y_train * np.log(a) + (1 - y_train) * np.log(1 - a))
C:/Users/LENOVO/PycharmProjects/MarkDmo001/logisticReg.py:33: RuntimeWarning: invalid value encountered in multiply
cost = (-1 / m) * np.sum(y_train * np.log(a) + (1 - y_train) * np.log(1 - a))
Cost after iteration 500: nan
Cost after iteration 1000: nan
Cost after iteration 1500: nan
Cost after iteration 2000: nan
Cost after iteration 2500: nan
Cost after iteration 3000: nan
Cost after iteration 3500: nan
Cost after iteration 4000: nan
Cost after iteration 4500: nan
Cost after iteration 5000: nan
Cost after iteration 5500: nan
Cost after iteration 6000: nan
Cost after iteration 6500: nan
Cost after iteration 7000: nan
Cost after iteration 7500: nan
Cost after iteration 8000: nan
Cost after iteration 8500: nan
Cost after iteration 9000: nan
Cost after iteration 9500: nan
Accuracy:
0.0
最佳答案
问题是梯度爆炸。您需要将输入标准化为[0, 1]。
如果您在训练数据中查看特征3和特征23，则将看到大于3000的值。将这些值与初始权重相乘后，它们仍位于范围[0, 30]中。因此，在第一次迭代中，z向量仅包含正数，其值最多约为50。结果，a向量（S型输出）看起来像这样：
[0.9994797 0.99853904 0.99358676 0.99999973 0.98392862 0.99983016 0.99818802 ...]
因此，第一步，您的模型始终以高置信度预测1。但这并不总是正确的，并且模型输出的高概率会导致较大的渐变，当您查看
dw的最大值时可以看到该渐变。就我而言dw[3]是388dw[23]是571其他值位于
[0, 55]中。因此，您可以清楚地看到这些功能中的大量输入是如何导致爆炸梯度的。由于梯度下降现在朝相反的方向迈进了一大步，因此下一步中的权重不在[0, 0.01]中，而是在[-285, 0.002]中，这只会使情况变得更糟。在下一次迭代中，z包含-1百万左右的值，这会导致S型函数溢出。解
将输入标准化为
[0, 1]在
[-0.01, 0.01]中使用权重，以使它们彼此大致抵消。否则，您在z中的值仍会根据您拥有的特征数量线性地缩放。至于标准化输入，您可以使用sklearn的
MinMaxScaler：x_train, x_test, y_train, y_test, m = load_dataset()
scaler = MinMaxScaler()
x_train_normalized = scaler.fit_transform(x_train.T).T
neuralNet = Neural_Network()
'''
step-2 : Train the network
'''
neuralNet.train(x_train_normalized, y_train,10000,m)
# Use the same transformation on the test inputs as on the training inputs
x_test_normalized = scaler.transform(x_test.T).T
y_predicted = neuralNet.predict(x_test_normalized)
之所以使用
.T是因为sklearn希望训练输入的形状为(num_samples, num_features)，而您的x_train和x_test的形状为(num_features, num_samples)。关于python-3.x - 如何获得逻辑回归的正确答案？，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/47807402/