如何在Python C-API中动态创建派生类型

问题描述

假设我们具有有关为Python编写C扩展模块的教程.现在我们要创建一个派生类型，只覆盖Noddy的__new__()方法.

Assume we have the type Noddy as defined in the tutorial on writing C extension modules for Python. Now we want to create a derived type, overwriting only the __new__() method of Noddy.

当前，我使用以下方法(为了便于阅读而对错误进行检查):

Currently I use the following approach (error checking stripped for readability):

PyTypeObject *BrownNoddyType =
    (PyTypeObject *)PyType_Type.tp_alloc(&PyType_Type, 0);
BrownNoddyType->tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE;
BrownNoddyType->tp_name = "noddy.BrownNoddy";
BrownNoddyType->tp_doc = "BrownNoddy objects";
BrownNoddyType->tp_base = &NoddyType;
BrownNoddyType->tp_new = BrownNoddy_new;
PyType_Ready(BrownNoddyType);

这可行，但是我不确定这是否是正确的方法.我本来希望我必须设置 Py_TPFLAGS_HEAPTYPE 标志，同样，因为我在堆上动态分配了类型对象，但这样做会导致解释器出现段错误.

This works, but I'm not sure if it is The Right Way To Do It. I would have expected that I have to set the Py_TPFLAGS_HEAPTYPE flag, too, because I dynamically allocate the type object on the heap, but doing so leads to a segfault in the interpreter.

我还考虑过使用PyObject_Call()或类似方法显式调用type()，但是我放弃了这个想法.我需要将函数BrownNoddy_new()包装在Python函数对象中，并创建一个映射__new__到此函数对象的字典，这似乎很愚蠢.

I also thought about explicitly calling type() using PyObject_Call() or similar, but I discarded the idea. I would need to wrap the function BrownNoddy_new() in a Python function object and create a dictionary mapping __new__ to this function object, which seems silly.

解决此问题的最佳方法是什么?我的方法正确吗?我错过了接口功能吗?

What is the best way to go about this? Is my approach correct? Is there an interface function I missed?

有关python-dev邮件列表上的相关主题，有两个主题(1) (2).从这些线程和一些实验中，我得出结论，除非通过调用type()分配类型，否则不应该设置Py_TPFLAGS_HEAPTYPE.无论是手动分配类型还是调用type()更好，这些线程中都有不同的建议.如果只有我知道包装应该放在tp_new插槽中的C函数的推荐方法是什么，我会对后者感到满意.对于常规方法，此步骤很容易-我可以使用 PyDescr_NewMethod() 获取合适的包装对象.但是，我不知道如何为__new__()方法创建这样的包装对象.也许我需要未记录的函数PyCFunction_New()来创建这样的包装对象.

There are two threads on a related topic on the python-dev mailing list (1) (2). From these threads and a few experiments I deduce that I shouldn't set Py_TPFLAGS_HEAPTYPE unless the type is allocated by a call to type(). There are different recommendations in these threads whether it is better to allocate the type manually or to call type(). I'd be happy with the latter if only I knew what the recommended way to wrap the C function that is supposed to go in the tp_new slot is. For regular methods this step would be easy -- I could just use PyDescr_NewMethod() to get a suitable wrapper object. I don't know how to create such a wrapper object for my __new__() method, though -- maybe I need the undocumented function PyCFunction_New() to create such a wrapper object.

推荐答案

我在修改扩展以使其与Python 3兼容时遇到了相同的问题，并在尝试解决该页面时找到了此页面.

I encountered the same problem when I was modifying an extension to be compatible with Python 3, and found this page when I was trying to solve it.

我最终确实通过阅读Python解释器的源代码来解决它， PEP 0384 和 C-API .

I did eventually solve it by reading the source code for the Python interpreter, PEP 0384 and the documentation for the C-API.

设置Py_TPFLAGS_HEAPTYPE标志告诉解释器将您的PyTypeObject重铸为PyHeapTypeObject，其中包含还必须分配的其他成员.在某些时候，解释器会尝试引用这些额外的成员，如果您不分配它们，则会导致段错误.

Setting the Py_TPFLAGS_HEAPTYPE flag tells the interpreter to recast your PyTypeObject as PyHeapTypeObject, which contains additional members that must also be allocated. At some point the interpreter attempts to refer to these extra members and, if you leave them unallocated, it will cause a segfault.

Python 3.2引入了C结构PyType_Slot和PyType_Spec以及C函数PyType_FromSpec，它们简化了动态类型的创建.简而言之，您可以使用PyType_Slot和PyType_Spec指定PyTypeObject的tp_*成员，然后调用PyType_FromSpec进行分配和初始化内存的工作.

Python 3.2 introduced the C structures PyType_Slot and PyType_Spec and the C function PyType_FromSpec that simplify the creation of dynamic types. In a nutshell, you use PyType_Slot and PyType_Spec to specify the tp_* members of the PyTypeObject and then call PyType_FromSpec to do the dirty work of allocating and initialising the memory.

从PEP 0384开始，我们有:

From PEP 0384, we have:

typedef struct{
  int slot;    /* slot id, see below */
  void *pfunc; /* function pointer */
} PyType_Slot;

typedef struct{
  const char* name;
  int basicsize;
  int itemsize;
  int flags;
  PyType_Slot *slots; /* terminated by slot==0. */
} PyType_Spec;

PyObject* PyType_FromSpec(PyType_Spec*);

(以上不是PEP 0384的文字副本，它也包含const char *doc作为PyType_Spec的成员.但是该成员未出现在源代码中.)

(The above isn't a literal copy from PEP 0384, which also includes const char *doc as a member of PyType_Spec. But that member doesn't appear in the source code.)

要在原始示例中使用它们，请假定我们有一个C结构BrownNoddy，该结构扩展了基类Noddy的C结构.然后我们将:

To use these in the original example, assume we have a C structure, BrownNoddy, that extends the C structure for the base class Noddy. Then we would have:

PyType_Slot slots[] = {
    { Py_tp_doc, "BrownNoddy objects" },
    { Py_tp_base, &NoddyType },
    { Py_tp_new, BrownNoddy_new },
    { 0 },
};
PyType_Spec spec = { "noddy.BrownNoddy", sizeof(BrownNoddy), 0,
                      Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, slots };
PyTypeObject *BrownNoddyType = (PyTypeObject *)PyType_FromSpec(&spec);

这应该完成原始代码中的所有操作，包括调用PyType_Ready，以及创建动态类型所需的操作，包括设置Py_TPFLAGS_HEAPTYPE以及为PyHeapTypeObject分配和初始化额外的内存.

This should do everything in the original code, including calling PyType_Ready, plus what is necessary for creating a dynamic type, including setting Py_TPFLAGS_HEAPTYPE, and allocating and initialising the extra memory for a PyHeapTypeObject.

我希望这会有所帮助.

这篇关于如何在Python C-API中动态创建派生类型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！