R - 优化调用的 foreach %dopar% 函数内部的问题

问题描述

限时删除！！

从 optim 调用包含 foreach %dopar% 构造的函数会导致错误:

Calling a function that includes foreach %dopar% construct from optim causes an error:

> workers <- startWorkers(6) # 6 cores
>
> registerDoSMP(workers)
>
> t0 <- Sys.time()
>
> optim(w,maxProb2,control=list(fnscale=-1))
>
> Error in { : task 1 failed - "unused argument(s) (isPrebuilt = TRUE)"
>
> Sys.time()-t0
>
> Time difference of 2.032 secs
>
> stopWorkers(workers)

被调用的函数如下所示:

The called function looks like that:

> maxProb2 <- function(wp) {
>
>   r <- foreach (i=s0:s1, .combine=c) %dopar% { pf(i,x[i,5],wp,isPrebuilt=TRUE) }
>
>   cat("w=",wp,"max=",sum(r),"
")
>
>   sum(r)
>
> }

pf 是其他一些函数，x 是预先计算的元素的静态表.

pf is some other function, x is a static table of pre-computed elements.

同样只调用一次要优化的函数会导致同样的错误:

Also calling the function to be optimized just once causes the same error:

> workers <- startWorkers(6) # 6 cores
>
> Warning message:
> In startWorkers(6) : there is an existing doSMP session using doSMP1
>
> registerDoSMP(workers)
>
> maxProb2(w)
> Error in { : task 1 failed - "unused argument(s) (isPrebuilt = TRUE)"
>
> stopWorkers(workers)

奇怪的是，相同的代码在直接调用一次时可以正常工作(优化多次调用相同的函数):

What's strange, the identical code works fine when called directly a single time (optim calles the same function many times):

> workers <- startWorkers(6) # 6 - ilosc rdzeni
>
> Warning message:
> In startWorkers(6) : there is an existing doSMP session using doSMP1
>
> registerDoSMP(workers)
>
> r <- foreach (i=s0:s1, .combine=c) %dopar% { pf(i,x[i,5],w,isPrebuilt=TRUE) }
>
> sum(r)
> [1] 187.1781
>
> stopWorkers(workers)

当使用 %do% 而不是 %dopar% 时，被调用的函数 (maxProb2) 工作正常.

The called function (maxProb2) works fine, when %do% is used instead of %dopar%.

如何正确调用包含 foreach %dopar% 构造的函数?

更新 2011-07-17:

UPDATE 2011-07-17:

我已将 pf 函数重命名为 probf 但问题仍然存在.

I have renamed the pf function into probf but the problem remains.

probf 函数是在脚本中定义的，而不是在某些外部包中.

probf functions is defined in the script, not in some external package.

两个注意事项:操作系统:Windows 7，IDE:Revolution Analytics Enterprise 4.3

Two notes: OS: Windows 7, IDE: Revolution Analytics Enterprise 4.3

> workers <- startWorkers(workerCount = 3)
>
> registerDoSMP(workers)
>
> maxProb2(w)
>
Error in { : task 1 failed - "could not find function "probf""

推荐答案

我遇到了同样的问题，问题是环境未包含在子线程中.你的错误

I ran into the same problem an the issue is with the environment not being included in the sub-threads. Your error

{ 中的错误:任务 1 失败 - 找不到函数simple_fn""

可以通过这个非常简单的例子重现:

can be reproduced by this very simple example:

simple_fn <- function(x)
    x+1

test_par <- function(){
    library("parallel")
    no_cores <- detectCores()
    library("foreach")
    cl<-makeCluster(no_cores)
    library("doSNOW")
    registerDoSNOW(cl)
    out <- foreach(i=1:10) %dopar% {
        simple_fn(i)
    }

    stopCluster(cl)
    return(out)
}

test_par()

现在您需要做的就是将 foreach(i=1:10) 更改为 foreach(i=1:10, .export=c("simple_fn")).如果您想导出完整的全局环境，那么只需编写 .export=ls(envir=globalenv()) 即可，无论好坏.

Now all you need to to is to change the foreach(i=1:10) into foreach(i=1:10, .export=c("simple_fn")). If you want to export your complete global environment then just write .export=ls(envir=globalenv()) and you will have it for better or worse.

这篇关于R - 优化调用的 foreach %dopar% 函数内部的问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！

1403页，肝出来的..