Tensorflow Keras 将权重从一个模型复制到另一个模型

问题描述

使用 Tensorflow 1.4.1 中的 Keras，如何将权重从一个模型复制到另一个模型?

Using Keras from Tensorflow 1.4.1, how does one copy weights from one model to another?

作为一些背景，我正在尝试在 DeepMind 发布 DQN 之后为 Atari 游戏实现一个 deep-q 网络 (DQN).我的理解是该实现使用了两个网络，Q 和 Q'.使用梯度下降训练 Q 的权重，然后定期将权重复制到 Q'.

As some background, I'm trying to implement a deep-q network (DQN) for Atari games following the DQN publication by DeepMind. My understanding is that the implementation uses two networks, Q and Q'. The weights of Q are trained using gradient descent, and then the weights are copied periodically to Q'.

这是我如何构建 Q 和 Q':

Here's how I build Q and Q':

ACT_SIZE   = 4
LEARN_RATE = 0.0025
OBS_SIZE   = 128

def buildModel():
  model = tf.keras.models.Sequential()

  model.add(tf.keras.layers.Lambda(lambda x: x / 255.0, input_shape=OBS_SIZE))
  model.add(tf.keras.layers.Dense(128, activation="relu"))
  model.add(tf.keras.layers.Dense(128, activation="relu"))
  model.add(tf.keras.layers.Dense(ACT_SIZE, activation="linear"))
  opt = tf.keras.optimizers.RMSprop(lr=LEARN_RATE)

  model.compile(loss="mean_squared_error", optimizer=opt)

  return model

我调用了两次以获得 Q 和 Q'.

I call that twice to get Q and Q'.

我在下面有一个 updateTargetModel 方法，它是我尝试复制权重的方法.代码运行良好，但我的整体 DQN 实现失败.我真的只是想验证这是否是将权重从一个网络复制到另一个网络的有效方法.

I have an updateTargetModel method below that is my attempt at copying weights. The code runs fine, but my overall DQN implementation is failing. I'm really just trying to verify if this is a valid way of copying weights from one network to another.

def updateTargetModel(model, targetModel):
  modelWeights       = model.trainable_weights
  targetModelWeights = targetModel.trainable_weights

  for i in range(len(targetModelWeights)):
    targetModelWeights[i].assign(modelWeights[i])

这里还有另一个问题讨论了在磁盘中保存和加载权重(Tensorflow Copy Weights Issuea>)，但没有公认的答案.还有一个关于从单个层加载权重的问题(Copying weights from oneConv2D 层到另一个)，但我想复制整个模型的权重.

There's another question here that discusses saving and loading weights to and from disk (Tensorflow Copy Weights Issue), but there's no accepted answer. There is also a question about loading weights from individual layers (Copying weights from one Conv2D layer to another), but I'm wanting to copy the entire model's weights.

推荐答案

实际上，您所做的不仅仅是复制权重.您让这两个模型一直完全相同.每次更新一个模型时 - 第二个模型也更新 - 因为两个模型具有相同的 weights 变量.

Actually what you've done is much more than simply copying weights. You made these two models identical all the time. Every time you update one model - the second one is also updated - as both models have the same weights variables.

如果您只想复制权重 - 最简单的方法是通过以下命令:

If you want to just copy weights - the simplest way is by this command:

target_model.set_weights(model.get_weights()) 

这篇关于Tensorflow Keras 将权重从一个模型复制到另一个模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！