为什么Google Colab TPU的速度和我的计算机一样慢?

问题描述

由于我的数据集很大且PC上没有很多功能，因此我认为在Google Colab上使用TPU是个好主意.

Since I have a large dataset and not much power in my PC, I thought it was a good idea to use TPU on Google Colab.

所以，这是我的TPU配置:

So, here is my TPU configuration :

try:
    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()
    print('Running on TPU ', tpu.master())
except ValueError:
    tpu = None

if tpu:
    tf.config.experimental_connect_to_cluster(tpu)
    tf.tpu.experimental.initialize_tpu_system(tpu)
    strategy = tf.distribute.experimental.TPUStrategy(tpu)
else:
    strategy = tf.distribute.get_strategy()

print("REPLICAS: ", strategy.num_replicas_in_sync)

这是我的训练:

hist = model.fit(train_dataset, epochs=10, verbose=1, steps_per_epoch=count_data_items(filenames)//64)

推荐答案

仅创建策略还不够.您应该正确使用此策略.

It is not enough to create a strategy. You should use this strategy correctly.

您可能必须调整管道，增加批次大小等.

You probably have to tune your pipeline, increase batch size, etc.

在这里查看: https://cloud.google.com/tpu/docs/performance-guide

另一个重要的一点是 TPU 有一个预热期–在第一次调用(每次调用都使用新的输入形状)期间，它会花费大量时间来构建计算图.

Another important point is that TPU has a warm-up period — it spends a lot of time building a computation graph during the first calls (every call with a new input shape).

这篇关于为什么Google Colab TPU的速度和我的计算机一样慢?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！