python - python和扭曲的代理，如何动态压缩？

使用twistedmatrix ProxyClient时，我该如何压缩和处理响应部分？

我需要检查文本或javascript和ajax查询/答案。我应该使用handleResponseEnd吗？

我认为它在handleResponsePart内，但似乎我误解了一点或某些东西，这是我的基本代码：

from twisted.python import log
from twisted.web import http, proxy

class ProxyClient(proxy.ProxyClient):
    """Mange returned header, content here.

    Use `self.father` methods to modify request directly.
    """
    def handleHeader(self, key, value):
            # change response header here
            log.msg("Header: %s: %s" % (key, value))
            proxy.ProxyClient.handleHeader(self, key, value)

    def handleResponsePart(self, buffer):
    # this part below do not work,
    # looks like @ this moment i do not have 'Content-Encoding' or 'Content-Type'
    # what am i misunderstading?
            cEncoding = self.father.getAllHeaders().get('Content-Encoding', '')
            cType = self.father.getAllHeaders().get('Content-Type', '')
            print >> sys.stderr, 'Content-Encoding', cEncoding
            print >> sys.stderr, 'Content-Type', cType
            if ('text' in cType.lower() or 'javascript' in cType.lower()) and 'gzip' in cEncoding.lower():

                buf = StringIO(buffer)
                s = gzip.GzipFile(mode="rb", fileobj=buf)
                content = s.read(len(buffer))

                # here process content as it should be gunziped

    proxy.ProxyClient.handleResponsePart(self, buffer)

class ProxyClientFactory(proxy.ProxyClientFactory):
    protocol = ProxyClient

class ProxyRequest(proxy.ProxyRequest):
    protocols = dict(http=ProxyClientFactory)

class Proxy(proxy.Proxy):
    requestFactory = ProxyRequest

class ProxyFactory(http.HTTPFactory):
    protocol = Proxy

从我的日志中，我有：

2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Date: Tue, 11 Jun 2013 12:07:25 GMT
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Server: Apache
...
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Content-Type: text/html;charset=ISO-8859-1
...
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Content-Encoding: gzip
...
2013-06-11 14:07:33+0200 [ProxyClient,client] Header: Connection: close

因此我应该同时具备两个条件！我想念什么？

即使我对第二种方式不感兴趣，也就是删除这样的请求接受，是否可以这样做：
（顺便说一句，它似乎无法正常工作，或者经过测试的网络服务器不关心我们不想接收gzip内容的事实）

class ProxyRequest(proxy.ProxyRequest):
    protocols = dict(http=ProxyClientFactory)

    def process(self):
        # removing the accept so that we do not tell "i'm ok with gzip encoded content" and should receive only not gzip-ed
        self.requestHeaders.removeHeader('accept')
        self.requestHeaders.removeHeader('accept-encoding')

最佳答案

您必须将数据块收集到handleResponsePart中的StringIO缓冲区中，然后使用handleResponseEnd中的GzipFile进行解码。