混合 TensorFlow 和 TensorFlow Federated 代码的推荐方法是什么?

问题描述

TensorFlow (TF) 和 TensorFlow Federated (TFF) 是不同的功能层，旨在很好地协同工作(顾名思义).

TensorFlow (TF) and TensorFlow Federated (TFF) are different layers of functionality that are designed to play well together (as the names implie).

不过，它们是不同的东西，旨在解决不同的问题.

Still, they are different things designed to solve different problems.

我想知道以一种既可以被 vanilla TF 也可以在 TFF 工作负载中使用的方式来描述计算的最佳方式，以及人们可能想要避免的那种陷阱.

I wonder what is the best way to describe computation in a way that can be used by both vanilla TF and in TFF workloads, as well as the kind of pitfalls that one might want to avoid.

推荐答案

很好的问题.事实上，至少有 3 种方法可以组合使用 TFF 的 TensorFlow 代码，每种方法都有自己的优点.

Great question. Indeed, there are at least 3 ways to approach composition of TensorFlow code for use with TFF, each with its own merits.

1. 使用 TensorFlow 的组合机制 (defuns) 是推荐的方式，前提是它适用于您的特定情况.TensorFlow 已经有编写代码的机制，我们不想重新发明轮子.我们在 TFF 中创建自己的组合机制(@tff.tf_computation)的原因是为了处理特定的限制(例如在 TF 中缺乏对接口级别的数据集的支持，以及需要 TF 组件与TFF 的其余部分)，理想情况下，我们会将这种机制的使用限制在真正需要它的情况下.

如果可能，使用@tf.function 装饰 TensorFlow 组件，并在将其嵌入 @tff.federated_computation 之前，仅在顶层将整个 TensorFlow 块包装为 @tff.tf_computation.这样做的众多好处之一是它允许您使用标准 TensorFlow 工具测试 TFF 之外的组件.

When possible, decorate TensorFlow components using @tf.function, and wrap the entire TensorFlow block as a @tff.tf_computation only at the top level, before embedding it in a @tff.federated_computation. One of the many benefits of this is that it allows you to test components outside of TFF, using standard TensorFlow tools.

因此，我们鼓励并首选以下内容:

So, the following is encouraged and preferred:

# here using TensorFlow's compositional mechanism (defuns)
# rather than TFF's to decorate "foo"
@tf.function(...)
def foo(...):
  ...

@tff.tf_computation(...)
def bar(...):
  # here relying on TensorFlow to embed "foo" as a component of "bar"
  ...foo(...)...

1. 使用 Python 的组合机制(未修饰的普通 Python 函数)也是一个不错的选择，尽管它不如 (1) 好，因为它只会导致一个代码体在定义时嵌入到另一个代码体中，因为 TFF 跟踪使用所有 TFF 修饰的 Python 函数来构造要执行的计算的序列化表示，而不会为您提供隔离或任何其他特殊好处.

您可能仍希望使用此模式来允许您的组件在 TFF 之外进行测试，或者在 (1) 或 (3) 都不起作用的情况下进行测试.

You may still want to use this pattern to allow your components to be tested outside of TFF, or in situations where neither (1) or (3) works.

因此，如果 (1) 不起作用，您应该首先考虑以下替代方案:

So, the following is an alternative you should consider first if (1) doesn't work:

# here composing things in Python, no special TF or TFF mechanism employed
def foo(...):
  # keep in mind that in this case, "foo" can access and tamper with
  # the internal state of "bar" - you get no isolation benefits
  ...

@tff.tf_computation(...)
def bar(...):
  # here effectively just executing "foo" within "bar" at the
  # time "bar" is traced
  ...foo(...)...

1. 不建议使用 TFF 的组合机制 (@tff.tf_computation)，除非 - 如上所述 - 在需要它的情况下，例如当 TensorFlow 组件需要接受数据集作为参数时，或者如果它将只能从 @tff.federated_computation 调用.请记住，TFF 对数据集作为参数的支持仍处于试验阶段，虽然在某些情况下它可能是唯一的解决方案，但您仍然可能会遇到问题.您可以期待实施会不断发展.

不鼓励(虽然目前有时需要):

Not encouraged (although currently sometimes necessary):

# here using TFF's compositional mechanism
@tff.tf_computation(...)
def foo(...):
  # here you do get isolation benefits - "foo" is traced and
  # serialized by TFF, but you can expect that e.g., some
  # tf.data.Dataset features won't work
  ...

@tff.tf_computation(...)
def bar(...):
  # here relying on TFF to embed "foo" within "bar"
  ...foo(...)...

这篇关于混合 TensorFlow 和 TensorFlow Federated 代码的推荐方法是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！