服务器上

服务器上

关注
发信
关注(28)粉丝(399)

Google App Engine - 在开发服务器上拒绝代理/隧道的权限

本文介绍了Google App Engine - 在开发服务器上拒绝代理/隧道的权限的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述



我使用的请求库，在GAE标准环境下修补了猴子，一切都完美无缺，除了向请求调用添加额外的，因此期望的套接字对象与期望套接字对象不同， code>（通过从解释器的内存中删除select将导致更多问题或根本不工作），所以我想替换 select 模块（它是内置）与标准pythonic之一，我该怎么做？



Nevermind ...

这种噩梦有什么方法可以结束并仍然使用标准环境？如果它在部署服务器上没有任何修补程序的情况下工作，为什么不能使用dev_appserver工具？



我只是不想再做任何修改补丁，我只是想通过 Permission denied 错误，而不是在远程部署服务器下抛出错误，使用相同的代码逻辑，不管我们是否使用请求，urlfetch ，httplib，套接字等（只是代理所需的隧道）。

解决方案

深入挖掘问题并且似乎是到这个文件：

platform / google_appengine / google / appengine / api / remote_socket / _remote_socket_stub.py 它有一些模拟套接字对，选项）在常量 _MOCK_SOCKET_OPTIONS 中找到，它缺少（1,3）对，我的意思是下面的级别和选项的产物之一（我猜星号是所需的值）：
$ b

级别（值：1）：


  • SOCKET_TCP_NODELAY

  • SOCKET_IP_TOS

  • SOCKET_SOL_SOCKET *

  • SOCKET_SO_DEBUG


选项（值：3）：


  • SOCKET_IP_HDRINCL

  • SOCKET_SO_TYPE *

  • SOCKET_TCP_CORK



所以通过设置代理，我想我要求 SOL_SOCKET：SO_TYPE 组，但模拟值如何，它是一个与十六进制相关的字符串为00000000（用于keepalive）和01000000（reuseaddr）：

我在哪里可以找到像（1,3）缺失对这样的值？





稍后编辑：



我添加了SOL_SOCKET：SO_TYPE = 80000000在SSL模块上（需要启用以支持https;无需修补，只需在app.yaml中启用即可）：
通过识别 128 的套接字类型，我猜可能是以下常量之一：


  • socket.MSG_EOR

  • socket.SOMAXCONN

  • socket.TIPC_SRC_DROPPABLE





稍后...



所以我意识到128值是实际值我的小端模拟值，因此我已将组更改为：SOL_SOCKET：SO_TYPE = 01000000，并且它可以识别 socket.SOCK_STREAM 套接字类型，该类型实际上起作用因为 ssl 不理解GAE的自定义套接字对象：（< google.appengine.api.remote_socket._remote_socket.socket.socket object在0x7f0cd037f690> ）。尝试了这一点，并且没有请求猴子补丁。



我的结论



GAE自定义 socket 不完全使用SSL（在隧道中）和Python标准 socket 没有与GAE（无补丁的猜测）选择模块，关于dev_appserver沙箱。



解决方案：将远程部署的相同行为引入本地开发环境。


Disclaimer: this is not a duplicate!

I'm using requests library, monkey patched under GAE standard environment and everything works flawlessly, except the times when I'm adding the extra proxy argument to the request call. This proxy thing implies tunneling and socket operations which are disabled by default by the standard configuration of both development and deploy environments:

NotImplementedError: HTTP CONNECT Tunnelling is not supported https://pastebin.com/YDsG9we7

Nice, so we need to enable real sockets support under our GAE platform, therefore adding:

env_variables: GAE_USE_SOCKETS_HTTPLIB : "True"

And even without doing the requests toolbelt monkey patch (not using urlfetch at all), we get a permission denied: https://pastebin.com/ifBFKi3K

This usually happens when the sandbox thinks that you're accessing a forbidden server location or port, but the proxy is fully functional and guess what, I've tested the very same code (without any added/removed functionality due to development or deploy server as current environment) and it works just as expected on the deploy, without any kind of Permission denied errors, by making the request completely successfull (if we try without httplib sockets enabled, we still get that Tunnelling is not supported error under both scenarios and this is the expected behavior).

Now, I've got a working solution on the deploy (which is what it really counts), but this is not enough for me, I just want to test these requests through proxies under my development server and environment too, so what I've tried to do is the following:

  • let's import our own standard sockets library by applying these: "ImportError: No module named _ssl" with dev_appserver.py from Google App Engine
    • can't monkey patch the sandbox.py file for white-listing _sockets and _ssl etc., no problem, I've hard-patched that directly on disk into the original file
    • replacing into memory the default sockets, httplib imports etc. with the standard pythonic ones
    • ISSUE: ValueError: select only supported on socket objects. https://pastebin.com/gHFM6QiF ; and this is somehow expected to happen, as the expected socket object is different from the expected one due to the standard sockets import but GAE's own select (and by removing select from the interpreter's memory will just cause more issues or not work at all), so I want to replace the select module (which is a built-in) too with the standard pythonic one, how can I do that?

Nevermind...

Is there any method through which this nightmare will end and still using the standard environment? If it works on the deploy server without any patches at all, why shouldn't also work with the dev_appserver tool too?

I just don't want to do any more patches, I just want to get past that Permission denied error which is not thrown under the remote deploy server, using the very same code logic, no matter if we use requests, urlfetch, httplib, sockets etc. (just the proxy's required tunneling thing).

解决方案

Digged-in the issue further and seems to be into this file:

platform/google_appengine/google/appengine/api/remote_socket/_remote_socket_stub.py which has some mocked socket pairs of (level, option) found in constant _MOCK_SOCKET_OPTIONS which is missing the (1, 3) pair and by this I mean one of the product of levels and options below (I guess the asterisk is the desired value):

Levels (value: 1):

  • SOCKET_TCP_NODELAY
  • SOCKET_IP_TOS
  • SOCKET_SOL_SOCKET *
  • SOCKET_SO_DEBUG

Options (value: 3):

  • SOCKET_IP_HDRINCL
  • SOCKET_SO_TYPE *
  • SOCKET_TCP_CORK

So by setting the proxy, I guess I'm asking for the SOL_SOCKET:SO_TYPE group, but what about the mocked value, which is a hex binary related string as "00000000" (for keepalive) and "01000000" (reuseaddr): https://github.com/GoogleCloudPlatform/python-compat-runtime/blob/master/appengine-compat/exported_appengine_sdk/google/appengine/api/remote_socket/_remote_socket_stub.py#L97

Where can I find values like these for the (1, 3) missing pairs?

Later edit:

I've added the "SOL_SOCKET:SO_TYPE=80000000" group and now it breaks on the ssl module (which is required to be enabled in order to support https; no patching needed, just enable it in app.yaml): https://pastebin.com/9KQjdEgLby recognizing the socket type of being 128, this may be one of the following constants I guess:

  • socket.MSG_EOR
  • socket.SOMAXCONN
  • socket.TIPC_SRC_DROPPABLE

Later...

So I realized the 128 value is actual my little-endian mocked value above, therefore I've changed the group into: "SOL_SOCKET:SO_TYPE=01000000" and this recognizes the socket.SOCK_STREAM socket type, which actually works just for this check, but then again, it crashes, because ssl doesn't understand GAE's custom socket object: https://pastebin.com/t2pUuW2V (<google.appengine.api.remote_socket._remote_socket.socket object at 0x7f0cd037f690>). Tried this with and without the requests monkey-patching.

My conclusion

The GAE custom socket is not working entirely with SSL (on tunneling) and Python standard socket is not working with GAE's (non-patch-able I guess) select module, regarding the dev_appserver sandbox.

Solution: bring the same behavior of the remote deploy into the local development environment.

这篇关于Google App Engine - 在开发服务器上拒绝代理/隧道的权限的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！

08-28 19:14
Powered by LMLPHP ©2024 服务器上 0.043885