具有DBI数据库处理程序的Perl Parallel :: ForkManager

问题描述

我有兴趣并行执行多个数据库操作.我玩过Perl Parallel :: ForkManager，但尚未将其用于任何数据库.我已经阅读到对此的支持不是很好.有人有经验吗?

im interested in performing multiple database actions in parallel. I have played with Perl Parallel::ForkManager but not used it with any databases yet. I have read that database connectivity is not supported very well with this. Does anyone have experience with this?

作为一个例子，我可能会生成一个系统调用(DBI可以工作)而不是原始代码，即

As an example i would probably be spawning a system call(which does the DBI work) NOT raw code, i.e.

#!/opt/local/bin/perl -w

use strict;
use Parallel::ForkManager;

$| = 1;

my $max_procs = 10;

my $pm =  new Parallel::ForkManager($max_procs);

for (my $var = 0; $var <= 10; $var++) {
  my $pid = $pm->start('proc'.$var) and next;
  sleep ( 2 );
  system( "./DBworker.pl $var" );
  $pm->finish(0);
}

print "Waiting for child procs\n";
$pm->wait_all_children;
print "complete!\n";

推荐答案

如果工作是由其他程序完成的，则fork操作没有危险.当您打开与数据库的连接然后进行分叉时，就会出现危险.孩子不能重用父母的联系.但是，请查看 DBIx::Connector ，为您处理fork之后需要处理的事情，并且运行多个程序通常不是正确的答案.

If the work is being done by other programs, there is no danger to forking. The danger comes when you open a connection to the database and then fork. The child can't reuse the parents connection; however, take a look at DBIx::Connector, it handles the things you need to do after forking for you and running multiple programs is generally not the right answer.

#!/usr/bin/perl

use strict;
use warnings;

use DBIx::Connector;
use Parallel::ForkManager;

my $dsn  = "dbi:SQLite:dbname=foo.db";
my $user = "";
my $pass = "";
my $conn = DBIx::Connector->new($dsn, $user, $pass,
    {
        AutoCommit       => 0,
        PrintError       => 0,
        RaiseError       => 1,
        ChopBlanks       => 1,
        FetchHashKeyName => 'NAME_lc',
    }
);
END { unlink "foo.db" }

#setup table
$conn->run(fixup => sub {
    my $dbh = $_;
    $dbh->do("create table foo ( id integer, name char(35) )");
    my $sth = $dbh->prepare("insert into foo (id, name) values (?, ?)");
    while (<DATA>) {
        chomp;
        $sth->execute(split /,/);
    }
});

my $pm = Parallel::ForkManager->new(3);

my $sth = $conn->dbh->prepare("select * from foo where id = ?");
for my $id (1 .. 3) {
    next if $pm->start;
    $sth->execute($id);
    while (my $row = $sth->fetchrow_hashref) {
        print "$id saw $row->{id} => $row->{name}\n";
    }
    $pm->finish;
}

$pm->wait_all_children;

print "done\n";

__DATA__
1,foo
2,bar
3,baz

这篇关于具有DBI数据库处理程序的Perl Parallel :: ForkManager的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！