Tensorflow - 获取队列中的样本数量?

问题描述

对于性能监控，我想关注当前排队的示例.我正在平衡用于填充队列的线程数量和队列的最佳最大大小.我如何获得这些信息?我正在使用 tf.train.batch()，但我猜信息可能在 FIFOQueue 的某个位置?我原以为这是一个局部变量，但我没有找到.

For performance monitoring I would like to keep an eye on the currently queued example. I am balancing the amount of threads I'm using for filling the queue and the optimal maximum size of the queue.How do I obtain this information? I am using a tf.train.batch(), but I guess the information might be somewhere down in the FIFOQueue?I would have expected this to be a local variable but I haven't found it.

推荐答案

tldr: 如果你的队列是由 tf.batch 创建的，你可以使用 sess.run("batch/fifo_queue_Size:0")

tldr: if your queue is created by tf.batch, you can get size with sess.run("batch/fifo_queue_Size:0")

一个 FIFOQueue 对象提供了一个 size() 方法，该方法创建一个操作，给出队列中元素的数量.但是，如果您使用 tf.batch，则在方法内部创建 FIFOQueue，并且此对象不会对外公开.

A FIFOQueue object provides a size() method which creates an op that gives number of elements on queue. However, if you are using tf.batch, FIFOQueue is created inside the method and this object is not exposed externally.

特别是您在 input 中看到了这一点.py

queue = _which_queue(dynamic_pad)(
    capacity=capacity, dtypes=types, shapes=shapes, shared_name=shared_name)
print("Enqueueing: ", enqueue_many, tensor_list, shapes)
_enqueue(queue, tensor_list, num_threads, enqueue_many)
summary.scalar("queue/%s/fraction_of_%d_full" % (queue.name, capacity),
               math_ops.cast(queue.size(), dtypes.float32) *
               (1. / capacity))

由于 queue 是本地的，您无法掌握它的 size() 方法.但是，由于 size() 已被调用以构建摘要，因此相应的 size 操作位于图中，您可以按名称调用它.您可以通过执行这样的操作来找到节点的名称

Since queue is local, you can't get a hold of its size() method. However since the size() has been called in order to construct the summary, the appropriate size op is in the graph and you can call it by name. You can find the name of the node by doing something like this

x = tf.constant(1)
q = tf.train.batch([x], 2)
tf.get_default_graph().as_graph_def()

你会看到

node {
  name: "batch/fifo_queue_Size"
  op: "QueueSize"
  input: "batch/fifo_queue"
  attr {
    key: "_class"
    value {
      list {

由此你可以看出 batch/fifo_queue_Size 是操作的名称，因此 batch/fifo_queue_Size:0 是第一个输出的名称，所以你可以通过执行以下操作来获取大小:

From this you can tell that batch/fifo_queue_Size is the name of the op, and hence batch/fifo_queue_Size:0 is the name of the first output, so you can get the size by doing something like this:

sess.run("batch/fifo_queue_Size:0")

如果您有多个 batch 操作，名称将自动删除为 batch_1/fifo_queue_Size、batch_2/fifo_queue_Size 等

If you have multiple batch ops, the names will be automatically deduped into batch_1/fifo_queue_Size, batch_2/fifo_queue_Size, etc

或者，您可以使用 tf.batch(...name="mybatch") 调用您的节点，然后张量的名称将为 mybatch/fifo_queue_Size:0

Alternatively you can call your node with tf.batch(...name="mybatch") and then the name of tensor will be mybatch/fifo_queue_Size:0

这篇关于Tensorflow - 获取队列中的样本数量?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！