结合序列模型中Keras中的两个损失函数与ndarray输出

问题描述

我正在Keras中训练CNN模型(图像和LiDAR中的目标检测(笑嘻嘻的Lyft比赛)).作为输出，我有一个34通道围带.因此，输出尺寸为:LENGTH x WIDTH X 34.前10个通道用于不同类别的对象(最好是一个热矢量)，其余24个通道是3D边界框的坐标.

I am training a CNN model in Keras (object detection in image and LiDAR (Kaggle Lyft Competition)). As an output I have a 34 channel gird. So output dimension is: LENGTH x WIDTH X 34.First 10 channels are for different categories of objects (ideally as one hot vector) and rest of 24 channels are coordinates of bounding box in 3D.

对于我想使用的前10个频道:keras.losses.categorical_crossentropy，对于其余24个频道:keras.losses.mean_squared_error

For first 10 channels I want to use: keras.losses.categorical_crossentropy, and for rest of 24: keras.losses.mean_squared_error

此外，由于对象的数量大不相同，为避免偏差，我在对每个类进行权衡.由于输出是ndarray，因此我不得不编写自定义损失函数(用于类加权).代码主要来自:

Also since numbers of objects differ drastically, to avoid bias, I am weighing each class. Since output is ndarray, I had to write custom loss function (for class weighing). Code is mostly from:Custom loss function for U-net in keras using class weights: `class_weight` not supported for 3+ dimensional targets

def weightedLoss(weightsList):

def lossFunc(true, pred):

    axis = -1 #if channels last 
    #axis=  1 #if channels first


    #argmax returns the index of the element with the greatest value
    #done in the class axis, it returns the class index    
    classSelectors = K.argmax(true, axis=axis) 

    #considering weights are ordered by class, for each class
    #true(1) if the class index is equal to the weight index   
    one64 = np.ones(1, dtype=np.int64)  #Needed to avod int32 and int64 error
    classSelectors = [K.equal(one64[0]*i, classSelectors) for i in range(len(weightsList))]

    #casting boolean to float for calculations  
    #each tensor in the list contains 1 where ground true class is equal to its index 
    #if you sum all these, you will get a tensor full of ones. 
    classSelectors = [K.cast(x, K.floatx()) for x in classSelectors]

    #for each of the selections above, multiply their respective weight
    weights = [sel * w for sel,w in zip(classSelectors, weightsList)] 

    #sums all the selections
    #result is a tensor with the respective weight for each element in predictions
    weightMultiplier = weights[0]
    for i in range(1, len(weights)):
        weightMultiplier = weightMultiplier + weights[i]

    op_chan_loss = keras.losses.categorical_crossentropy
    op_box_loss = keras.losses.mean_squared_error
    #make sure your originalLossFunc only collapses the class axis
    #you need the other axes intact to multiply the weights tensor
    print(type(true), type(pred))
    loss = op_chan_loss(true, pred) 
    loss = loss * weightMultiplier

    return loss
return lossFunc

但是，我不确定如何在此自定义损失函数中将两个损失函数组合在一起.请帮忙.

However I am not sure how to combine two loss functions together in this custom loss function. Please help.

推荐答案

使用两个输出.您的模型必须是Functional API模型:

Use two outputs. Your model must be a Functional API model:

#basic example of the initial part of your model
inputs = Input(input_shape)
intermediate_output = Conv2D(...)(inputs)
intermediate_output = Conv2D(...)(intermediate_output)

在模型中的某个点，您将分离两个分支.也许最后一层或更早(由于两个输出的性质不同，也许(需要测试)，每个输出都需要一层或两层才能更好地适应.

At some point in your model, you will separate two branches. Maybe the last layer or a little earlier (since the nature of the two outputs is different, maybe (needs test, of course) each output needs one or two layers to adapt better.

output1 = SomeLayer(...)(intermediate_output)
....
output1 = Dense(10, activation='softmax', name='cat_out')(output1) 


output2 = SomeLayer(...)(intermediate_output)
....
output2 = SomeLayer(24, name='bound_out')(output2) #maybe choose an activation

创建具有两个输出的模型:

Create a model with two outputs:

model = Model(inputs, [output1, output2])

现在，这使您可以为每个输出具有不同的损失函数，并具有不同的度量标准:

Now this allows you to have a different loss function for each output, and different metrics:

model.compile(loss = [weightedLoss(weights), 'mse'], 
              metrics = {'cat_out':[metrics1], 'bound_out':[metrics2], 
              optimizer = ...)

训练有两个输出:

model.fit(x_train, [y_train[:,:,:,:10], y_train[:,:,:,-24:]], ...)

---

我没有检查您的损失函数，但我看到它有循环(通常对张量运算不利).

---

I didn't check your loss function, but I saw it has loops (usually a bad thing for tensor operations).

我建议您的体重清单为张量:

I suggest that your weight list be a tensor:

weightsTensor = K.constant(listOfWeights)

然后在选择权重时，尝试使用tf.gather或tf.gather_nd.我没有花时间思考什么是正确的函数以及如何使用它，但是一旦完成，您只需将结果乘以权重即可:

And when you're selecting your weights, try to use tf.gather or tf.gather_nd. I didn't spend time thinking about what is the correct function and how to use it, but once you do it, you simply multiply your results by the weights:

#this may need change and might be a little troublesome to understand
selectedWeights = tf.gather_nd(weightsTensor, classSelectors, ....) 

loss = keras.losses.categorical_crossentropy(true, pred)
loss = selectedWeights * loss

这篇关于结合序列模型中Keras中的两个损失函数与ndarray输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！