我正在尝试使用Incanter data analysis库在Clojure中实现一个简单的逻辑回归示例。我已经成功编写了Sigmoid和Cost函数,但是Incanter的BFGS最小化函数似乎给我带来了很多麻烦。
(ns ml-clj.logistic
(:require [incanter.core :refer :all]
[incanter.optimize :refer :all]))
(defn sigmoid
"compute the inverse logit function, large positive numbers should be
close to 1, large negative numbers near 0,
z can be a scalar, vector or matrix.
sanity check: (sigmoid 0) should always evaluate to 0.5"
[z]
(div 1 (plus 1 (exp (minus z)))))
(defn cost-func
"computes the cost function (J) that will be minimized
inputs:params theta X matrix and Y vector"
[X y]
(let
[m (nrow X)
init-vals (matrix (take (ncol X) (repeat 0)))
z (mmult X init-vals)
h (sigmoid z)
f-half (mult (matrix (map - y)) (log (sigmoid (mmult X init-vals))))
s-half (mult (minus 1 y) (log (minus 1 (sigmoid (mmult X init-vals)))))
sub-tmp (minus f-half s-half)
J (mmult (/ 1 m) (reduce + sub-tmp))]
J))
当我尝试
(minimize (cost-func X y) (matrix [0 0]))
给minimize
一个函数并启动参数时,REPL会引发错误。ArityException Wrong number of args (2) passed to: optimize$minimize clojure.lang.AFn.throwArity (AFn.java:437)
我对最小化功能到底期望什么感到非常困惑。
作为参考,我用python重写了所有代码,并且所有代码都使用相同的最小化算法按预期运行。
import numpy as np
import scipy as sp
data = np.loadtxt('testSet.txt', delimiter='\t')
X = data[:,0:2]
y = data[:, 2]
def sigmoid(X):
return 1.0 / (1.0 + np.e**(-1.0 * X))
def compute_cost(theta, X, y):
m = y.shape[0]
h = sigmoid(X.dot(theta.T))
J = y.T.dot(np.log(h)) + (1.0 - y.T).dot(np.log(1.0 - h))
cost = (-1.0 / m) * J.sum()
return cost
def fit_logistic(X,y):
initial_thetas = np.zeros((len(X[0]), 1))
myargs = (X, y)
theta = sp.optimize.fmin_bfgs(compute_cost, x0=initial_thetas,
args=myargs)
return theta
输出
Current function value: 0.594902
Iterations: 6
Function evaluations: 36
Gradient evaluations: 9
array([ 0.08108673, -0.12334958])
我不明白为什么Python代码可以成功运行,但是我的Clojure实现失败。有什么建议么?
更新资料
重新读取
minimize
的文档字符串,我一直在尝试计算cost-func
的派生类,这将引发新的错误。(def grad (gradient cost-func (matrix [0 0])))
(minimize cost-func (matrix [0 0]) (grad (matrix [0 0]) X))
ExceptionInfo throw+: {:exception "Matrices of different sizes cannot be differenced.", :asize [2 1], :bsize [1 2]} clatrix.core/- (core.clj:950)
使用
trans
将1xn col矩阵转换为nx1行矩阵只会产生相同的错误,但具有相反的错误。:asize [1 2], :bsize [2 1]}
我在这里很迷路。
最佳答案
关于您的实现,我什么也没说,但是incanter.optimize/minimize期望(至少)三个参数,而您只给了两个参数:
Arguments:
f -- Objective function. Takes a collection of values and returns a scalar
of the value of the function.
start -- Collection of initial guesses for the minimum
f-prime -- partial derivative of the objective function. Takes
a collection of values and returns a collection of partial
derivatives with respect to each variable. If this is not
provided it will be estimated using gradient-fn.
不幸的是,我无法在这里直接告诉您要提供什么(
f-prime
?),但也许有人可以提供。顺便说一句,我认为ArityException Wrong number of args (2) passed to [...]
实际上在这里很有帮助。编辑:实际上,我认为上面的文档字符串不正确,因为源代码未使用
gradient-fn
来估计f-prime
。也许您可以使用incanter.optimize/gradient
生成自己的?