请求库在HTTPS代理CONNECT上强制使用HTTP/1.1

是否有一种方法可以获取请求，或者底层的urllib3，或者甚至更低的http库，以始终使用http1.1，即使另一端似乎正在使用1.0?这里有一个示例程序(不幸的是，它要求您安装具有RTCP的IBM Ration Integration Tester安装程序才能真正复制)以重现该问题: 将http.client导入为http_clienthttp_client.HTTPConnection.debuglevel = 1导入日志汇入要求logging.basicConfig()logging.getLogger().setLevel(logging.DEBUG)requests_log = logging.getLogger("requests.packages.urllib3")requests_log.setLevel(logging.DEBUG)requests_log.propagate = Truerequests.post("https://host:8443/axl"，headers = {"soapAction":'"CUCM:DB ver = 9.1 updateSipTrunk"'}，data ='< soapenv:信封xmlns:soapenv ="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tns ="http://www.cisco.com/AXL/API/9.1">< soapenv:Header/>< soapenv:Body>< tns:updateSipTrunk><名称> PLACEHOLDER</名称>< newName> PLACEHOLDER</newName><目的地><目的地> p; 10.10.1.5</addressIpv4>< sortOrder> 1</sortOrder></destination>//目的地</tns:updateSipTrunk>//soapenv:Body></soapenv:Envelope，验证= False) (通过HTTPS_PROXY环境变量配置代理)在错误之前调试输出，请注意HTTP/1.0: INFO:requests.packages.urllib3.connectionpool:启动新的HTTPS连接(1):host.com发送:b'CONNECT host.com:8443 HTTP/1.0 \ r \ n'发送:b'\ r \ n'标头:主机:host.com:8443标头:Proxy-agent:Green Hat HTTPS Proxy/1.0 RHEL 6中出现的确切错误文本是: requests.exceptions.SSLError:[SSL:SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3警报握手失败(_ssl.c:646) 即使在此处显示了Host标头，它也不会显示在网络上.我通过tcpdump确认了这一点: 14:03:14.315049 IP sourcehost.53214>desthost.com:标志[P.]，seq 0:32，ack 1，win 115，选项[nop，nop，TS val 2743933964 ecr 4116114841]，长度320x0000:0000 0c07 ac00 0050 56b5 4044 0800 4500 ....... PV.@ D..E.0x0010:0054 3404 4000 4006 2ca0 0af8 3f15 0afb .T4.@.@ .....0x0020:84f8 cfde 0c7f a4f8 280a 4ebd b425 8018 ........(.N ..％..0x0030:0073 da46 0000 0101 080a a38d 1c0c f556 .s.F ..... V0x0040:XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX ..CONNECT.host0x0050:XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX xx:8443.HTTP/1.00x0060:0d0a 当我用冗长的方式卷曲它时，输出结果如下: *关于connect()到proxy proxy-host.com端口3199(#0)*正在尝试10.**.**.** ...已连接*连接到proxy-host.com(10.**.**.**)端口3199(#0)*建立到host.com的HTTP代理隧道:8443>CONNECT host.com:8443 HTTP/1.1>主持人:host.com:8443>用户代理:curl/7.19.7(x86_64-redhat-linux-gnu)libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2>代理连接:保持活动>soapAction:"CUCM:DB ver = 9.1 updateSipTrunk"><HTTP/1.0 200 OK<主持人:host.com:8443<代理:Green Hat HTTPS Proxy/1.0<*代理对连接请求的回答为确定"*使用证书路径初始化NSS:sql:/etc/pki/nssdb* CAfile:/path/to/store/ca-bundle.crtCApath:无*使用TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256的SSL连接 在此之后被截断.连接后，您可以从代理看到HTTP/1.0响应.curl的tcpdump也清楚地显示了主机头以及HTTP 1.1. *我无法完全确定这是根本问题，因为我无法对其进行测试.我确实看到了HTTP/1.0响应，并且可以说我的非工作Python代码发送CONNECT HTTP/1.0消息，而工作的Java发送HTTP/1.1消息，Curl也是如此.这个问题可能无关紧要(尽管我发现不太可能)，或者Python的行为异常，而不是Java/curl.我只是不知道要知道什么.那么，有没有一种方法可以强制urllib3/requests始终使用HTTP v1.1?解决方案 httplib ( requests 所依赖的HTTP(S)繁重)始终使用 HTTP/1.0 使用 CONNECT : Lib/httplib.py:788 : def _tunnel(自己):self.send("CONNECT％s:％d HTTP/1.0 \ r \ n"％(self._tunnel_host，self._tunnel_port))对于标头，self._tunnel_headers.iteritems()中的值:self.send(％s:％s \ r \ n"％(标题，值))self.send("\ r \ n")< ...> 因此，除了通过编辑子例程之外，您不能在此处强制"它使用"HTTP/1.1".如果代理不支持HTTP/1.0，则可能是问题所在-特别是1.0不需要 Host:标头，实际上，如您通过比较日志输出所看到的使用上面的代码， httplib 不会发送它.尽管实际上，代理可能会期望它.但是，在这种情况下，您应该从代理那里得到一个错误或响应CONNECT的东西-除非代理太乏味以至于它用 Host(主机)代替了一些默认(或垃圾):，无论如何都会返回 200 并尝试将God-knows-where连接到那里，这时您将超时.您可以使 httplib 将 Host:头添加到CONNECT中，方法是将其添加到 _tunnel_headers 中(间接): s = requests.Session()proxy_url = os.environ ['HTTPS_PROXY']s.proxies ["https"] = proxy_url#必须在此处指定代理，因为env变量仅由httplib代码检测到#while我们需要触发更早起作用的请求的代理逻辑#"https"表示任何https主机.由于会话保留cookie，#通过它向多个主机发出请求毫无意义.pm = s.get_adapter("https://").proxy_manager_for(proxy_url)pm.proxy_headers ['Host'] ="host.com"del pm，proxy_url< ...>s.get('https://host.com') I am having a problem with a misbehaving HTTP Proxy server. I have no control over the proxy server, unfortunately -- it's an 'enterprise' product from IBM. The proxy server is part of a service virtualization solution being leveraged for software testing.The fundamental issue (I think*) is that the proxy server sends back HTTP/1.0 responses. I can get it to work fine from SOAP UI ( A Java application) and curl from the command line, but Python refuses to connect. From what I can tell, Python is behaving correctly, and the other two are not, as the server expects HTTP/1.1 responses (it wants Host headers, at the very least, to route the service request to a given stub).Is there a way to get Requests, or the underlying urllib3, or the even farther down http lib to always use http1.1, even if the other end appears to be using 1.0?Here is a sample program (unfortunately, it requires you to have an IBM Ration Integration Tester installation with RTCP to really replicate) to reproduce the problem:import http.client as http_clienthttp_client.HTTPConnection.debuglevel = 1import loggingimport requestslogging.basicConfig()logging.getLogger().setLevel(logging.DEBUG)requests_log = logging.getLogger("requests.packages.urllib3")requests_log.setLevel(logging.DEBUG)requests_log.propagate = Truerequests.post("https://host:8443/axl", headers={"soapAction": '"CUCM:DB ver=9.1 updateSipTrunk"'}, data='<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tns="http://www.cisco.com/AXL/API/9.1"><soapenv:Header/><soapenv:Body><tns:updateSipTrunk><name>PLACEHOLDER</name><newName>PLACEHOLDER</newName><destinations><destination><addressIpv4>10.10.1.5</addressIpv4><sortOrder>1</sortOrder></destination></destinations></tns:updateSipTrunk></soapenv:Body></soapenv:Envelope>', verify=False)(Proxy is configured via HTTPS_PROXY environment variable)Debug output before the error, note the HTTP/1.0:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): host.comsend: b'CONNECT host.com:8443 HTTP/1.0\r\n'send: b'\r\n'header: Host: host.com:8443header: Proxy-agent: Green Hat HTTPS Proxy/1.0The exact error text that occurs in RHEL 6 is:requests.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:646)Even though the Host header is shown here, it does NOT show up on the wire. I confirmed this with a tcpdump:14:03:14.315049 IP sourcehost.53214 > desthost.com: Flags [P.], seq 0:32, ack 1, win 115, options [nop,nop,TS val 2743933964 ecr 4116114841], length 32 0x0000: 0000 0c07 ac00 0050 56b5 4044 0800 4500 [email protected]. 0x0010: 0054 3404 4000 4006 2ca0 0af8 3f15 0afb .T4.@.@.,...?... 0x0020: 84f8 cfde 0c7f a4f8 280a 4ebd b425 8018 ........(.N..%.. 0x0030: 0073 da46 0000 0101 080a a38d 1c0c f556 .s.F...........V 0x0040: XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX ..CONNECT.host 0x0050: XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX xx:8443.HTTP/1.0 0x0060: 0d0aWhen I curl it with verbose, this is what the output looks like:* About to connect() to proxy proxy-host.com port 3199 (#0)* Trying 10.**.**.** ... connected* Connected to proxy-host.com (10.**.**.**) port 3199 (#0)* Establish HTTP proxy tunnel to host.com:8443> CONNECT host.com:8443 HTTP/1.1> Host: host.com:8443> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2> Proxy-Connection: Keep-Alive> soapAction: "CUCM:DB ver=9.1 updateSipTrunk">< HTTP/1.0 200 OK< Host: host.com:8443< Proxy-agent: Green Hat HTTPS Proxy/1.0<* Proxy replied OK to CONNECT request* Initializing NSS with certpath: sql:/etc/pki/nssdb* CAfile: /path/to/store/ca-bundle.crt CApath: none* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256Truncated after this point. You can see the HTTP/1.0 response from the proxy after connecting. The curl's tcpdump also clearly shows the host header, as well as HTTP 1.1.*I can't be entirely sure this is the fundamental issue, as I can't test it. I do see HTTP/1.0 responses, and can tell that my non-working Python code sends CONNECT HTTP/1.0 messages, while the working Java sends HTTP/1.1 messages, as does Curl. It's possible the problem is unrelated (although I find that unlikely) or that Python is misbehaving, and not Java/curl. I simply don't know enough to know for sure.So, is there a way to force urllib3/requests to use HTTP v1.1 at all times? 解决方案 httplib (which requests relies upon for HTTP(S) heavy lifting) always uses HTTP/1.0 with CONNECT:Lib/httplib.py:788:def _tunnel(self): self.send("CONNECT %s:%d HTTP/1.0\r\n" % (self._tunnel_host, self._tunnel_port)) for header, value in self._tunnel_headers.iteritems(): self.send("%s: %s\r\n" % (header, value)) self.send("\r\n") <...>So you can't "force" it to use "HTTP/1.1" here other than by editing the subroutine.This MAY be the problem if the proxy doesn't support HTTP/1.0 - in particular, 1.0 does not require a Host: header, and indeed, as you can see by comparing your log output with the code above, httplib does not send it. While, in verity, a proxy may expect it regardless. But if this is the case, you should've gotten an error from the proxy or something in response to CONNECT -- unless the proxy is so borken that it substitutes some default (or garbage) for Host:, returns 200 anyway and tries to connect God-knows-where, at which point you're getting timeouts.You can make httplib add the Host: header to CONNECT by adding it to _tunnel_headers (indirectly):s=requests.Session()proxy_url=os.environ['HTTPS_PROXY']s.proxies["https"]=proxy_url# have to specify proxy here because env variable is only detected by httplib code#while we need to trigger requests' proxy logic that acts earlier# "https" means any https host. Since a Session persists cookies,#it's meaningless to make requests to multiple hosts through it anyway.pm=s.get_adapter("https://").proxy_manager_for(proxy_url)pm.proxy_headers['Host']="host.com"del pm,proxy_url<...>s.get('https://host.com') 这篇关于请求库在HTTPS代理CONNECT上强制使用HTTP/1.1的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！ 上岸，阿里云！