Pybind11:使用for循环通过OpenMP访问python对象

问题描述

我正在尝试在python字典的所有元素上运行c ++函数.为此，我在字典中的所有元素上使用c ++中的for循环.在这种情况下，据我所知，可以使用#pragma omp parallel for simd子句来加快速度.但是，当我运行它时，出现错误:

I am trying to operate a c++ function on all elements of a python dictionary. For this I use a for loop in c++ over all elements of the dictionary. In this case and as far as I understand, this can be sped up using the #pragma omp parallel for simd clause. However, when I run it, I get the error:

修改

我在此帖子上已读到源于在c ++中访问Python对象的方式，但是这里不再赘述.我猜问题出在访问具有多个线程的python对象.我目前仍在搜索.

I have read on this post that the problem comes from the way of accessing a Python object in c++, however it does not go into further discussion. I guess the problem comes from accessing a python object with multiple threads. I am still currently searching.

任何提示将不胜感激.

Any hint would be highly appreciated.

非常感谢.

C ++代码

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include <pybind11/stl.h>
#include <omp.h>
#include <chrono>
#include <thread>

namespace py = pybind11;
py::module nn = py::module::import("neat.nn");

py::object create_seq(
  py::object self
  ){

  unsigned int z;
  #pragma omp parallel for simd
  for(z = 0; z < 50000; z++){
      double value = self.attr("dict").attr("__getitem__")(z).cast<double>();
      value *=2;
      self.attr("dict").attr("__setitem__")(z, value);
  }
  return self;
}

PYBIND11_MODULE(error, m){

    m.doc() = "pybind11 module for iterating over generations";

    m.def("create_seq", &create_seq,
      "the function which creates a sequence");

}

Python代码

import os
import error

class func():
    def __init__(self):

        dictionary = {}
        for i in range(50000):
            dictionary[i] = i

        self.dict = dictionary
        self.array = None

    def modify_dict(self):
        return error.create_seq(self)

if __name__ == '__main__':
    # Determine path to configuration file. This path manipulation is
    # here so that the script will run successfully regardless of the
    # current working directory.
    local_dir = os.path.dirname(__file__)
    a = func()
    a.modify_dict()
    print(a.dict)

编译为:

g++ -O3 -Wall -shared -std=c++14 -fopenmp -fPIC `python3 -m pybind11 --includes` openmp.cpp -o error.so

推荐答案

问题确实是通过多个线程访问python对象.我不知道为什么的确切原因，但这根本是不可能的.诀窍是将您的python对象转换为python对象列表，必须将其转换为向量.只有这样，多个线程才能访问向量的各种python对象.

The problem is indeed accessing the python object with multiple threads. I don't know the exact reasons why, but this is simply impossible. The trick is to transform your python object into a list of python objects, which must be converted into a vector. Only then, the multiple threads access the various python objects of the vector.

在我的情况下，我没有传递整个字典，而是传递了列表中字典的值，然后将该列表转换为python对象的向量.

In my case, I passed not the entire dictionary rather than the values of the dictionary in a list, and then converted this list into a vector of python objects.

在简历中，它类似于:

error.create_seq(list(dict.values()))

以及在c ++中:

std::vector<py::object> = dict.cast<std::vector<py::object>>();

，然后可以应用#pragma omp parallel子句

这篇关于Pybind11:使用for循环通过OpenMP访问python对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！