问题描述
我无法正确初始化NumPy C API。我想我已经隔离了问题,从不同的翻译单元调用 import_array
,但我不知道为什么这应该是重要。
最小工作示例:
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#include< Python.h>
#include< numpy / npy_3kcompat.h>
#include< numpy / arrayobject.h>
void initialize();
#endif
file1.cpp / p>
#includeheader1.hpp
void * wrap_import_array()
{
import_array();
return(void *)1;
}
void initialize()
{
wrap_import_array();
}
file2.cpp
$ b
#includeheader1.hpp
#include< iostream>
void * loc_wrap_import_array()
{
import_array();
return(void *)1;
}
void loc_initialize()
{
loc_wrap_import_array();
}
int main()
{
Py_Initialize();
#ifdef USE_LOC_INIT
loc_initialize();
#else
initialize();
#endif
npy_intp dms [] = {5};
std :: cout<< creation descr<< std :: endl;
PyArray_Descr * dtype = PyArray_DescrFromType(NPY_FLOAT64);
std :: cout<< zeros<< std :: endl;
PyArray_Zeros(1,dims,dtype,0);
std :: cout<< 清理< std :: endl;
return 0;
}
编译器命令:
g ++ file1.cpp file2.cpp -o segissue -lpython3.4m -I / usr / include / python3.4m -DUSE_LOC_INIT
./segissue
#runs罚款
g ++ file1.cpp file2.cpp -o segissue -lpython3.4m -I / usr / include / python3.4m
./segissue
#segfaults
我用Clang 3.6.0,GCC 4.9.2,Python 2.7和Python 3.4测试了这个适当修改 wrap_import_array
,因为这在Python 2.x和3.x之间是不同的)。各种组合都给出相同的结果:如果我不调用 loc_initialize
,程序将在 PyArray_DescrFromType
呼叫。我有NumPy版本1.8.2。作为参考,我在Ubuntu 15.04运行这个。
最令人困惑的是似乎在不同的翻译单元中调用 import_array
b
我缺少什么?为什么必须从同一个翻译单元调用 import_array
才能真正生效?更重要的是,当我从Boost.NumPy包装器之类的不同翻译单元调用 import_array
时,如何让它工作?
通过NumPy头部挖掘,我想我已经找到一个解决方案:
$ c> numpy / __ multiarray_api.h ,有一个部分处理内部API缓冲区应该在哪里。为了简明起见,以下是相关的代码段:
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
#define PyArray_API PY_ARRAY_UNIQUE_SYMBOL
#endif
#if defined(NO_IMPORT)|| defined(NO_IMPORT_ARRAY)
extern void ** PyArray_API;
#else
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
void ** PyArray_API;
#else
static void ** PyArray_API = NULL;
#endif
#endif
看起来这是为了允许多个模块定义自己的内部API缓冲区,其中每个模块必须调用自己的 import_array
define。
要得到几个翻译单元使用相同的内部API缓冲区在每个模块,定义 PY_ARRAY_UNIQUE_SYMBOL
到一些库的唯一名称,然后每个翻译单位定义了import_array包装器定义 NO_IMPORT
或 NO_IMPORT_ARRAY
。顺便提一下,对于ufunc特性也有类似的宏: PY_UFUNC_UNIQUE_SYMBOL
和 NO_IMPORT
/ NO_IMPORT_UFUNC
。
修改后的工作示例:
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#ifndef MYLIBRARY_USE_IMPORT
#define NO_IMPORT
#endif
#define PY_ARRAY_UNIQUE_SYMBOL MYLIBRARY_ARRAY_API
#define PY_UFUNC_UNIQUE_SYMBOL MYLIBRARY_UFUNC_API
#include< Python.h>
#include< numpy / npy_3kcompat.h>
#include< numpy / arrayobject.h>
void initialize();
#endif
file1.cpp / p>
#define MYLIBRARY_USE_IMPORT
#includeheader1.hpp
void * wrap_import_array
{
import_array();
return(void *)1;
}
void initialize()
{
wrap_import_array();
}
file2.cpp
$ b
#includeheader1.hpp
#include< iostream>
int main()
{
Py_Initialize();
initialize();
npy_intp dims [] = {5};
std :: cout<< creation descr<< std :: endl;
PyArray_Descr * dtype = PyArray_DescrFromType(NPY_FLOAT64);
std :: cout<< zeros<< std :: endl;
PyArray_Zeros(1,dims,dtype,0);
std :: cout<< 清理< std :: endl;
return 0;
}
我不知道这个黑客有什么陷阱,任何更好的替代品,但这似乎至少编译和运行没有任何segfaults。
I'm having problems getting the NumPy C API to properly initialize. I think I've isolated the problem to calling import_array
from a different translation unit, but I don't know why this should matter.
Minimal working example:
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#include <Python.h>
#include <numpy/npy_3kcompat.h>
#include <numpy/arrayobject.h>
void initialize();
#endif
file1.cpp
#include "header1.hpp"
void* wrap_import_array()
{
import_array();
return (void*) 1;
}
void initialize()
{
wrap_import_array();
}
file2.cpp
#include "header1.hpp"
#include <iostream>
void* loc_wrap_import_array()
{
import_array();
return (void*) 1;
}
void loc_initialize()
{
loc_wrap_import_array();
}
int main()
{
Py_Initialize();
#ifdef USE_LOC_INIT
loc_initialize();
#else
initialize();
#endif
npy_intp dims[] = {5};
std::cout << "creating descr" << std::endl;
PyArray_Descr* dtype = PyArray_DescrFromType(NPY_FLOAT64);
std::cout << "zeros" << std::endl;
PyArray_Zeros(1, dims, dtype, 0);
std::cout << "cleanup" << std::endl;
return 0;
}
Compiler commands:
g++ file1.cpp file2.cpp -o segissue -lpython3.4m -I/usr/include/python3.4m -DUSE_LOC_INIT
./segissue
# runs fine
g++ file1.cpp file2.cpp -o segissue -lpython3.4m -I/usr/include/python3.4m
./segissue
# segfaults
I've tested this with Clang 3.6.0, GCC 4.9.2, Python 2.7, and Python 3.4 (with a suitably modified wrap_import_array
because this is different between Python 2.x and 3.x). The various combinations all give the same result: if I don't call loc_initialize
, the program will segfault in the PyArray_DescrFromType
call. I have NumPy version 1.8.2. For reference, I'm running this in Ubuntu 15.04.
What baffles me most of all is this C++ NumPy wrapper appears to get away with calling import_array
in a different translation unit.
What am I missing? Why must I call import_array
from the same translation unit in order for it to actually take effect? More importantly, how do I get it to work when I call import_array
from a different translation unit like the Boost.NumPy wrapper does?
After digging through the NumPy headers, I think I've found a solution:
in numpy/__multiarray_api.h
, there's a section dealing with where an internal API buffer should be. For conciseness, here's the relevant snippet:
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
#define PyArray_API PY_ARRAY_UNIQUE_SYMBOL
#endif
#if defined(NO_IMPORT) || defined(NO_IMPORT_ARRAY)
extern void **PyArray_API;
#else
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
void **PyArray_API;
#else
static void **PyArray_API=NULL;
#endif
#endif
It looks like this is intended to allow multiple modules define their own internal API buffer, in which each module must call their own import_array
define.
A consistent way to get several translation units to use the same internal API buffer is in every module, define PY_ARRAY_UNIQUE_SYMBOL
to some library unique name, then every translation unit other than the one where the import_array wrapper is defined defines NO_IMPORT
or NO_IMPORT_ARRAY
. Incidentally, there are similar macros for the ufunc features: PY_UFUNC_UNIQUE_SYMBOL
, and NO_IMPORT
/NO_IMPORT_UFUNC
.
The modified working example:
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#ifndef MYLIBRARY_USE_IMPORT
#define NO_IMPORT
#endif
#define PY_ARRAY_UNIQUE_SYMBOL MYLIBRARY_ARRAY_API
#define PY_UFUNC_UNIQUE_SYMBOL MYLIBRARY_UFUNC_API
#include <Python.h>
#include <numpy/npy_3kcompat.h>
#include <numpy/arrayobject.h>
void initialize();
#endif
file1.cpp
#define MYLIBRARY_USE_IMPORT
#include "header1.hpp"
void* wrap_import_array()
{
import_array();
return (void*) 1;
}
void initialize()
{
wrap_import_array();
}
file2.cpp
#include "header1.hpp"
#include <iostream>
int main()
{
Py_Initialize();
initialize();
npy_intp dims[] = {5};
std::cout << "creating descr" << std::endl;
PyArray_Descr* dtype = PyArray_DescrFromType(NPY_FLOAT64);
std::cout << "zeros" << std::endl;
PyArray_Zeros(1, dims, dtype, 0);
std::cout << "cleanup" << std::endl;
return 0;
}
I don't know what pitfalls there are with this hack or if there are any better alternatives, but this appears to at least compile and run without any segfaults.
这篇关于Segfault时import_array不在同一个翻译单元的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!