我有一个相当长的测试任务,由SCons自动完成,可以并行化。然而,它目前依赖于使用chdir=1
,这在目前是不容易删除的。现在,当我分别使用-j2
时,作业就会失败,下面的最小(非)工作示例说明了原因:chdir不是基于每个线程应用的,而是同时影响所有作业。
这种行为是否有污点?有什么方法可以防止这种情况发生吗?
# File SConstruct
import os, time
def my_build_fun(target, source, env):
for i in range(1, 5):
out = "my_build_fun: %d %s %s" % (i, str(source[0]), os.getcwd())
print out
time.sleep(0.5)
return None
bld = Builder(action = my_build_fun,
suffix = '.output',
src_suffix = '.input',
chdir=1)
env = Environment(BUILDERS = {'Foo' : bld})
Alias('do_a', env.Foo('folder_a/do_a'))
Alias('do_b', env.Foo('folder_b/do_b'))
Default(['do_a', 'do_b'])
操作
+--- SConstruct
+-+- folder_a
| \--- do_a.input
\-+- folder_b
\--- do_b.input
SetOption('num_jobs', 2)
的结果:如预期scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
os.chdir('folder_a')
my_build_fun(["folder_a\do_a.output"], ["folder_a\do_a.input"])
my_build_fun: 1 folder_a\do_a.input H:\Playground\folder_a
my_build_fun: 2 folder_a\do_a.input H:\Playground\folder_a
my_build_fun: 3 folder_a\do_a.input H:\Playground\folder_a
my_build_fun: 4 folder_a\do_a.input H:\Playground\folder_a
os.chdir('H:\\Playground')
os.chdir('folder_b')
my_build_fun(["folder_b\do_b.output"], ["folder_b\do_b.input"])
my_build_fun: 1 folder_b\do_b.input H:\Playground\folder_b
my_build_fun: 2 folder_b\do_b.input H:\Playground\folder_b
my_build_fun: 3 folder_b\do_b.input H:\Playground\folder_b
my_build_fun: 4 folder_b\do_b.input H:\Playground\folder_b
os.chdir('H:\\Playground')
scons: done building targets.
对于
scons -j1
的结果:试图进入scons -j2
这确实不存在。scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
os.chdir('folder_a')
my_build_fun(["folder_a\do_a.output"], ["folder_a\do_a.input"])
my_build_fun: 1 folder_a\do_a.input H:\Playground\folder_a
os.chdir('folder_b')
my_build_fun(["folder_b\do_b.output"], ["folder_b\do_b.input"])
scons: *** [folder_b\do_b.output] folder_b: The system cannot find the file specified
my_build_fun: 2 folder_a\do_a.input H:\Playground\folder_a
my_build_fun: 3 folder_a\do_a.input H:\Playground\folder_a
my_build_fun: 4 folder_a\do_a.input H:\Playground\folder_a
my_build_fun
os.chdir('H:\\Playground')
scons: building terminated because of errors.
最佳答案
从SCons手册页
http://www.scons.org/doc/2.3.2/HTML/scons-man.html
警告:Python只为所有
线。这意味着使用chdir参数将不起作用
使用SCons-j选项,因为由
scon在开始更改目录时相互干扰。
因此,如果无法从生成的基本目录运行脚本。解决这个问题的(当前)最简单的方法是创建一个脚本,该脚本将cd,然后运行您的命令。或者把它放到执行的shell命令中
env.Command(b,a,"cd ${SOURCE.dir}; do whatever -o ${TARGET.file} -i ${SOURCE.file}").
关于python - SCons中chdir = 1和num_jobs> 1的组合,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/30099640/