我尝试使用 twisted.protocols.tls 实现一个可以在 TLS 上运行 TLS 的协议(protocol)，这是一个使用内存 BIO 的 OpenSSL 接口(interface)。
我将它实现为一个协议(protocol)包装器，它主要看起来像一个常规的 TCP 传输，但它有 startTLS 和 stopTLS 方法分别用于添加和删除一层 TLS。这适用于第一层 TLS。如果我通过“ native ”Twisted TLS 传输运行它，它也能正常工作。但是，如果我尝试使用此包装器提供的 startTLS 方法添加第二个 TLS 层，则会立即出现握手错误，并且连接最终会处于某种未知的不可用状态。
包装器和两个让它工作的助手看起来像这样:
from twisted.python.components import proxyForInterface
from twisted.internet.error import ConnectionDone
from twisted.internet.interfaces import ITCPTransport, IProtocol
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory
class TransportWithoutDisconnection(proxyForInterface(ITCPTransport)):
"""
A proxy for a normal transport that disables actually closing the connection.
This is necessary so that when TLSMemoryBIOProtocol notices the SSL EOF it
doesn't actually close the underlying connection.
All methods except loseConnection are proxied directly to the real transport.
"""
def loseConnection(self):
pass
class ProtocolWithoutConnectionLost(proxyForInterface(IProtocol)):
"""
A proxy for a normal protocol which captures clean connection shutdown
notification and sends it to the TLS stacking code instead of the protocol.
When TLS is shutdown cleanly, this notification will arrive. Instead of telling
the protocol that the entire connection is gone, the notification is used to
unstack the TLS code in OnionProtocol and hidden from the wrapped protocol. Any
other kind of connection shutdown (SSL handshake error, network hiccups, etc) are
treated as real problems and propagated to the wrapped protocol.
"""
def connectionLost(self, reason):
if reason.check(ConnectionDone):
self.onion._stopped()
else:
super(ProtocolWithoutConnectionLost, self).connectionLost(reason)
class OnionProtocol(ProtocolWrapper):
"""
OnionProtocol is both a transport and a protocol. As a protocol, it can run over
any other ITransport. As a transport, it implements stackable TLS. That is,
whatever application traffic is generated by the protocol running on top of
OnionProtocol can be encapsulated in a TLS conversation. Or, that TLS conversation
can be encapsulated in another TLS conversation. Or **that** TLS conversation can
be encapsulated in yet *another* TLS conversation.
Each layer of TLS can use different connection parameters, such as keys, ciphers,
certificate requirements, etc. At the remote end of this connection, each has to
be decrypted separately, starting at the outermost and working in. OnionProtocol
can do this itself, of course, just as it can encrypt each layer starting with the
innermost.
"""
def makeConnection(self, transport):
self._tlsStack = []
ProtocolWrapper.makeConnection(self, transport)
def startTLS(self, contextFactory, client, bytes=None):
"""
Add a layer of TLS, with SSL parameters defined by the given contextFactory.
If *client* is True, this side of the connection will be an SSL client.
Otherwise it will be an SSL server.
If extra bytes which may be (or almost certainly are) part of the SSL handshake
were received by the protocol running on top of OnionProtocol, they must be
passed here as the **bytes** parameter.
"""
# First, create a wrapper around the application-level protocol
# (wrappedProtocol) which can catch connectionLost and tell this OnionProtocol
# about it. This is necessary to pop from _tlsStack when the outermost TLS
# layer stops.
connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
connLost.onion = self
# Construct a new TLS layer, delivering events and application data to the
# wrapper just created.
tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)
tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)
# Push the previous transport and protocol onto the stack so they can be
# retrieved when this new TLS layer stops.
self._tlsStack.append((self.transport, self.wrappedProtocol))
# Create a transport for the new TLS layer to talk to. This is a passthrough
# to the OnionProtocol's current transport, except for capturing loseConnection
# to avoid really closing the underlying connection.
transport = TransportWithoutDisconnection(self.transport)
# Make the new TLS layer the current protocol and transport.
self.wrappedProtocol = self.transport = tlsProtocol
# And connect the new TLS layer to the previous outermost transport.
self.transport.makeConnection(transport)
# If the application accidentally got some bytes from the TLS handshake, deliver
# them to the new TLS layer.
if bytes is not None:
self.wrappedProtocol.dataReceived(bytes)
def stopTLS(self):
"""
Remove a layer of TLS.
"""
# Just tell the current TLS layer to shut down. When it has done so, we'll get
# notification in *_stopped*.
self.transport.loseConnection()
def _stopped(self):
# A TLS layer has completely shut down. Throw it away and move back to the
# TLS layer it was wrapping (or possibly back to the original non-TLS
# transport).
self.transport, self.wrappedProtocol = self._tlsStack.pop()
我有简单的客户端和服务器程序来执行此操作，可从启动板 (
bzr branch lp:~exarkun/+junk/onion ) 获得。当我使用它两次调用上面的 startTLS 方法时，没有中间调用 stopTLS ，出现这个 OpenSSL 错误:OpenSSL.SSL.Error: [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')]
为什么事情会出错？
最佳答案
OnionProtocol 至少有两个问题:
TLSMemoryBIOProtocol 变成了 wrappedProtocol ，这时它应该是最外面的； ProtocolWithoutConnectionLost不会弹出任何TLSMemoryBIOProtocol小号断OnionProtocol的堆栈，因为经过connectionLost小号FileDescriptor或doRead方法返回断开的原因doWrite只调用。 如果不改变
OnionProtocol 管理其堆栈的方式，我们无法解决第一个问题，并且在我们弄清楚新的堆栈实现之前，我们无法解决第二个问题。不出所料，正确的设计是数据在 Twisted 中流动方式的直接结果，因此我们将从一些数据流分析开始。Twisted 表示与
twisted.internet.tcp.Server 或 twisted.internet.tcp.Client 实例建立的连接。由于我们程序中唯一的交互发生在 stoptls_client 中，我们将只考虑进出 Client 实例的数据流。让我们用一个最小的
LineReceiver 客户端来热身，它回显从端口 9999 上的本地服务器收到的线路:from twisted.protocols import basic
from twisted.internet import defer, endpoints, protocol, task
class LineReceiver(basic.LineReceiver):
def lineReceived(self, line):
self.sendLine(line)
def main(reactor):
clientEndpoint = endpoints.clientFromString(
reactor, "tcp:localhost:9999")
connected = clientEndpoint.connect(
protocol.ClientFactory.forProtocol(LineReceiver))
def waitForever(_):
return defer.Deferred()
return connected.addCallback(waitForever)
task.react(main)
一旦建立的连接建立，
Client 成为我们的 LineReceiver 协议(protocol)的传输和中介输入和输出:来自服务器的新数据导致 react 器调用
Client 的 doRead 方法，然后将收到的数据传递给 LineReceiver 的 0x251812143141 方法。最后，当至少有一条线路可用时，dataReceived 调用 LineReceiver.dataReceived。我们的应用程序通过调用
LineReceiver.lineReceived 将一行数据发送回服务器。这在绑定(bind)到协议(protocol)实例的传输上调用 LineReceiver.sendLine，它与处理传入数据的 write 实例相同。 Client 安排了 react 器发送的数据，而 Client.write 实际上是通过套接字发送数据。我们已经准备好查看从不调用
Client.doWrite 的 OnionClient 的行为:startTLS 被包裹在 OnionClient s 中，这是我们尝试嵌套 TLS 的关键。作为 OnionProtocol 的子类， twisted.internet.policies.ProtocolWrapper 的实例是一种协议(protocol)传输三明治；它表现为一个较低级别的传输协议(protocol)，并表现为一个协议(protocol)的传输，它通过在连接时由 OnionProtocol 建立的伪装包装。现在，
WrappingFactory 调用 Client.doRead ，它将数据代理到 OnionProtocol.dataReceived 。作为 OnionClient 的传输， OnionClient 接受从 OnionProtocol.write 发送的行并将它们代理到 OnionClient.sendLine ，它自己的传输。这是 Client 、它的包装协议(protocol)和它自己的传输之间的正常交互，因此自然地数据流进和流出每个都没有任何问题。ProtocolWrapper 做了一些不同的事情。它试图在已建立的协议(protocol)传输对之间插入一个新的 OnionProtocol.startTLS——恰好是一个 ProtocolWrapper 。这看起来很简单: TLSMemoryBIOProtocol 将上层协议(protocol)存储为其 ProtocolWrapper attribute 和 proxies wrappedProtocol and other attributes down to its own transport 。 write 应该能够注入(inject)一个新的 startTLS 将 TLSMemoryBIOProtocol 包装到连接中，方法是在它自己的 OnionClient 和 18431343 上修补该实例:def startTLS(self):
...
connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
connLost.onion = self
# Construct a new TLS layer, delivering events and application data to the
# wrapper just created.
tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)
# Push the previous transport and protocol onto the stack so they can be
# retrieved when this new TLS layer stops.
self._tlsStack.append((self.transport, self.wrappedProtocol))
...
# Make the new TLS layer the current protocol and transport.
self.wrappedProtocol = self.transport = tlsProtocol
这是第一次调用
wrappedProtocol 后的数据流:正如预期的那样，传送到
transport 的新数据被路由到存储在 startTLS 上的 OnionProtocol.dataReceived ，它将解密的明文传递给 TLSMemoryBIOProtocol141。 _tlsStack 还将其数据传递给 OnionClient.dataReceived ，后者对其进行加密并将生成的密文发送到 OnionClient.sendLine 和 TLSMemoryBIOProtocol.write 。不幸的是，这个方案在第二次调用
OnionProtocol.write 后失败了。根本原因是这一行: self.wrappedProtocol = self.transport = tlsProtocol
每次调用
Client.write 都会用最里面的 startTLS 替换 startTLS ，即使 wrappedProtocol 收到的数据是由最外面的:然而，
TLSMemoryBIOProtocol 是正确嵌套的。 Client.doRead只能调用其运输的transport - 那就是，OnionClient.sendLine - 所以write应与最里面OnionProtocol.write替换其OnionProtocol确保写入连续嵌套的加密附加层。那么，解决方案是确保数据依次通过
transport 上的第一个 TLSMemoryBIOProtocol 流到下一个，以便每一层加密都按照应用的相反顺序剥离:鉴于此新要求，将
TLSMemoryBIOProtocol 表示为列表似乎不太自然。幸运的是，线性地表示传入的数据流暗示了一种新的数据结构:错误和正确的传入数据流都类似于一个单向链表，其中
_tlsStack 用作 _tlsStack 的下一个链接，而 wrappedProtocol 用作 0x25181224313441该列表应从 ProtocolWrapper 向下增长，并始终以 protocol 结尾。该错误的发生是因为违反了该排序不变量。单向链表适合将协议(protocol)插入堆栈，但难以将它们弹出，因为它需要从其头部向下遍历到节点才能删除。当然，每次接收数据时都会发生这种遍历，因此关注的是额外遍历所隐含的复杂性，而不是最坏情况的时间复杂性。幸运的是，该列表实际上是双向链接的:
Client 属性将每个嵌套协议(protocol)与其前身链接起来，因此 OnionProtocol 可以在最终通过网络发送数据之前依次进行较低级别的加密。我们有两个哨兵来帮助管理列表:OnionClient 必须始终在顶部，transport 必须始终在底部。将两者放在一起，我们最终得到:
from twisted.python.components import proxyForInterface
from twisted.internet.interfaces import ITCPTransport
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory
class PopOnDisconnectTransport(proxyForInterface(ITCPTransport)):
"""
L{TLSMemoryBIOProtocol.loseConnection} shuts down the TLS session
and calls its own transport's C{loseConnection}. A zero-length
read also calls the transport's C{loseConnection}. This proxy
uses that behavior to invoke a C{pop} callback when a session has
ended. The callback is invoked exactly once because
C{loseConnection} must be idempotent.
"""
def __init__(self, pop, **kwargs):
super(PopOnDisconnectTransport, self).__init__(**kwargs)
self._pop = pop
def loseConnection(self):
self._pop()
self._pop = lambda: None
class OnionProtocol(ProtocolWrapper):
"""
OnionProtocol is both a transport and a protocol. As a protocol,
it can run over any other ITransport. As a transport, it
implements stackable TLS. That is, whatever application traffic
is generated by the protocol running on top of OnionProtocol can
be encapsulated in a TLS conversation. Or, that TLS conversation
can be encapsulated in another TLS conversation. Or **that** TLS
conversation can be encapsulated in yet *another* TLS
conversation.
Each layer of TLS can use different connection parameters, such as
keys, ciphers, certificate requirements, etc. At the remote end
of this connection, each has to be decrypted separately, starting
at the outermost and working in. OnionProtocol can do this
itself, of course, just as it can encrypt each layer starting with
the innermost.
"""
def __init__(self, *args, **kwargs):
ProtocolWrapper.__init__(self, *args, **kwargs)
# The application level protocol is the sentinel at the tail
# of the linked list stack of protocol wrappers. The stack
# begins at this sentinel.
self._tailProtocol = self._currentProtocol = self.wrappedProtocol
def startTLS(self, contextFactory, client, bytes=None):
"""
Add a layer of TLS, with SSL parameters defined by the given
contextFactory.
If *client* is True, this side of the connection will be an
SSL client. Otherwise it will be an SSL server.
If extra bytes which may be (or almost certainly are) part of
the SSL handshake were received by the protocol running on top
of OnionProtocol, they must be passed here as the **bytes**
parameter.
"""
# The newest TLS session is spliced in between the previous
# and the application protocol at the tail end of the list.
tlsProtocol = TLSMemoryBIOProtocol(None, self._tailProtocol, False)
tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)
if self._currentProtocol is self._tailProtocol:
# This is the first and thus outermost TLS session. The
# transport is the immutable sentinel that no startTLS or
# stopTLS call will move within the linked list stack.
# The wrappedProtocol will remain this outermost session
# until it's terminated.
self.wrappedProtocol = tlsProtocol
nextTransport = PopOnDisconnectTransport(
original=self.transport,
pop=self._pop
)
# Store the proxied transport as the list's head sentinel
# to enable an easy identity check in _pop.
self._headTransport = nextTransport
else:
# This a later TLS session within the stack. The previous
# TLS session becomes its transport.
nextTransport = PopOnDisconnectTransport(
original=self._currentProtocol,
pop=self._pop
)
# Splice the new TLS session into the linked list stack.
# wrappedProtocol serves as the link, so the protocol at the
# current position takes our new TLS session as its
# wrappedProtocol.
self._currentProtocol.wrappedProtocol = tlsProtocol
# Move down one position in the linked list.
self._currentProtocol = tlsProtocol
# Expose the new, innermost TLS session as the transport to
# the application protocol.
self.transport = self._currentProtocol
# Connect the new TLS session to the previous transport. The
# transport attribute also serves as the previous link.
tlsProtocol.makeConnection(nextTransport)
# Left over bytes are part of the latest handshake. Pass them
# on to the innermost TLS session.
if bytes is not None:
tlsProtocol.dataReceived(bytes)
def stopTLS(self):
self.transport.loseConnection()
def _pop(self):
pop = self._currentProtocol
previous = pop.transport
# If the previous link is the head sentinel, we've run out of
# linked list. Ensure that the application protocol, stored
# as the tail sentinel, becomes the wrappedProtocol, and the
# head sentinel, which is the underlying transport, becomes
# the transport.
if previous is self._headTransport:
self._currentProtocol = self.wrappedProtocol = self._tailProtocol
self.transport = previous
else:
# Splice out a protocol from the linked list stack. The
# previous transport is a PopOnDisconnectTransport proxy,
# so first retrieve proxied object off its original
# attribute.
previousProtocol = previous.original
# The previous protocol's next link becomes the popped
# protocol's next link
previousProtocol.wrappedProtocol = pop.wrappedProtocol
# Move up one position in the linked list.
self._currentProtocol = previousProtocol
# Expose the new, innermost TLS session as the transport
# to the application protocol.
self.transport = self._currentProtocol
class OnionFactory(WrappingFactory):
"""
A L{WrappingFactory} that overrides
L{WrappingFactory.registerProtocol} and
L{WrappingFactory.unregisterProtocol}. These methods store in and
remove from a dictionary L{ProtocolWrapper} instances. The
C{transport} patching done as part of the linked-list management
above causes the instances' hash to change, because the
C{__hash__} is proxied through to the wrapped transport. They're
not essential to this program, so the easiest solution is to make
them do nothing.
"""
protocol = OnionProtocol
def registerProtocol(self, protocol):
pass
def unregisterProtocol(self, protocol):
pass
(这也可以在 GitHub 上找到。)
第二个问题的解决方案在于
transport.write 。原始代码试图通过 Client 从堆栈中弹出一个 TLS session ，但是因为只有关闭的文件描述符会导致 OnionClient 被调用，它无法删除没有关闭底层套接字的已停止的 TLS session 。在撰写本文时，
PopOnDisconnectTransport 正好在两个地方调用其传输的 connectionLost : connectionLost 和 TLSMemoryBIOProtocol 。 loseConnection被称为主动关闭( _shutdownTLS ， _tlsShutdownFinished ， _shutdownTLS 和after loseConnection and all pending writes have been flushed)，而abortConnection被称为被动关闭(handshake failures，empty reads，read errors，和write errors)。这意味着关闭连接的双方都可以在 unregisterProducer 期间从堆栈中弹出已停止的 TLS session 。 loseConnection 是幂等的，因为 _tlsShutdownFinished 通常是幂等的，而 loseConnection 肯定期望它是幂等的。将堆栈管理逻辑放在
PopOnDisconnectTransport 的缺点是它取决于 loseConnection 的实现细节。通用的解决方案需要跨多个 Twisted 级别的新 API。在那之前，我们一直在使用 Hyrum's Law 的另一个例子。
关于openssl - 尝试使用此代码通过 TLS 运行 TLS 时，为什么会出现握手失败？，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/5130080/