将模型放入 ctypes 库的回调函数中

问题描述

我正在尝试使用托管 VM 功能设置基于 Google App Engine 的应用程序.

I am trying to setup an application based on the Google App Engine using the Managed VM feature.

我正在使用使用 ctypes 用 C++ 编写的共享库

I am using a shared library written in C++ using ctypes

cdll.LoadLibrary('./mylib.so')

注册一个回调函数

CB_FUNC_TYPE = CFUNCTYPE(None, eSubscriptionType)
cbFuncType = CB_FUNC_TYPE(scrptCallbackHandler)

我想将数据保存到 ndb 数据存储区

in which i want to save data to the ndb datastore

def scrptCallbackHandler(arg):
    model = Model(name=str(arg.data))
    model.put()

我正在注册一个回调函数，我想在其中从 C++ 程序中获取数据并将其放入 ndb 数据存储区.这会导致错误.在开发服务器上，它的行为略有不同，因此在生产服务器上:

I am registering a callback function in which i want to take the Data from the C++ program and put it in the ndb datastore. This results in an error. On the devserver it behaves slightly different, so from a production server:

暂停生成器 _put_tasklet(context.py:343) 引发 BadRequestError(Application Id (app) format is invalid: '_')LOG 2 1429698464071045 暂停生成器 put(context.py:810) 引发 BadRequestError(Application Id)(app) 格式无效:'_')回溯(最近一次调用最后一次):文件_ctypes/callbacks.c"，第 314 行，在调用回调函数"中 文件/home/vmagent/app/isw_cloud_client.py"，第 343 行，在 scrptCallbackHandler node.put() 中文件/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/model.py"，第 3380 行，在 _put 中返回 self._put_async(**ctx_options).get_result()文件/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/tasklets.py"，第 325 行，在 get_result self.check_success() 中文件/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/tasklets.py"，第 368 行，在 _help_tasklet_along value = gen.throw(exc.__class__, exc, tb)文件/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/context.py"，第 810 行，输入键 = yield self._put_batcher.add(entity, options)文件/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/tasklets.py"，第 368 行，在 _help_tasklet_along value = gen.throw(exc.__class__, exc, tb)文件/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/context.py"，第 343 行，在 _put_tasklet 中，keys = yield self._conn.async_put(options, datastore_entities)文件/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/tasklets.py"，第 454 行，_on_rpc_completion result = rpc.get_result()文件/home/vmagent/python_vm_runtime/google/appengine/api/apiproxy_stub_map.py"，第 613 行，在 get_result 中返回 self.__get_result_hook(self)文件/home/vmagent/python_vm_runtime/google/appengine/datastore/datastore_rpc.py"，第 1827 行，在 __put_hook self.check_rpc_success(rpc)文件/home/vmagent/python_vm_runtime/google/appengine/datastore/datastore_rpc.py"，第 1342 行，check_rpc_success 引发 _ToDatastoreError(err)google.appengine.api.datastore_errors.BadRequestError:应用程序 ID(应用程序)格式无效:'_'

C++ 程序的启动由对请求处理程序的调用触发，但在后台运行并接受应在回调中处理的传入数据.

The start of the C++ program is triggered by a call to a Request handler but runs in the background and accepts incoming data which should be processed in the callback.

更新: 正如 Tim 已经指出的那样，wsgi 处理程序的上下文似乎丢失了.最有可能的解决方案是以某种方式创建应用程序上下文.

Update: As Tim pointed out already it seems that the context of the wsgi handler is lost. Most likely the solution here would be to create the application context somehow.

推荐答案

我只是在猜测我的问题是什么，我想告诉我我做了什么来解决它.

I am only guessing what is my problem and i want to tell what i did to solve it.

回调函数的执行上下文与 Python 应用程序的其余部分有些不同.回调中的任何异步操作都会失败.我尝试进行 http 调用或将其保存到数据存储区.操作永远不会完成，并且在 60 秒后应用程序显示他们崩溃的错误.我猜这是因为python如何管理执行和相应的内存分配.

The execution context of the callback functions is somewhat different than the rest of the python application. Any asynchronous operation in the callback fails. I tried doing an http call or saving it to the datastore. The operations never finish and after 60s the application shows an error that they crashed. I guess this is because how the python manages the execution and the corresponding memory allocation.

通过将回调包装在类中的闭包中，我能够在对象的上下文中执行回调.这不是真正的问题，但可以在以下答案中找到解决方案:如何使用 python ctypes 使方法作为回调工作?

I was able to execute the callback in an object's context by wrapping it in a closure within a class. This wasnt really the problem but the solution can be found in this answer: How can I get methods to work as callbacks with python ctypes?

对于我的解决方案，我现在使用另一个模块上的云端点和 ctypes 模块上的后台线程的组合.

For my solution i am now using a combination of cloud-endpoints on another module and background threads on the ctypes-module.

在 C-Callback 中，我启动了一个后台线程，它能够进行异步工作

Within the C-Callback i start a background thread, which is able to do asynchronous work

# Start a background thread using the background thread service from GAE
background_thread.start_new_background_thread(putData, [name, value])

这里是它执行的简单任务:

And here the simple task it executes:

# Here i call my cloud-endpoints
def putData(name, value):
    body = {
        'name' : 'name',
        'value' : int(value)
    }
    res = service.objects().create(body=body).execute()

当然我需要做错误处理和其他事情，但对我来说这是一个很好的解决方案.

Of course i need to do error handling and additional stuff, but for me this is a good solution.

注意:在bg线程中将models添加到datastore失败，因为bg线程中的环境与应用不同，应用id没有设置.

这篇关于将模型放入 ctypes 库的回调函数中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！