本文介绍了并行,跟踪结果和计数运行的bash命令的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我不知道如何,如果可能的话,我可以在bash并行处理多个命令创建一个简单的作业管理。也就是说,我有命令的大名单跑了,我想有运行在任何给定的时间他们两个。

I was wondering how, if possible, I can create a simple job management in BASH to process several commands in parallel. That is, I have a big list of commands to run, and I'd like to have two of them running at any given time.

我知道不少关于bash的,所以这里是使需求棘手:

I know quite a bit about bash, so here are the requirements that make it tricky:


  • 命令具有可变的运行时间,所以我不能只产卵2,等待,然后继续接下来的两个。只要一个命令完成下一个命令必须执行。

  • 控制进程需要知道每个命令的出口code,以便它可以保持共多少失败

我在想不知何故,我可以使用陷阱,但我没有看到一个简单的方法来获得处理程序中一个孩子的退出值。

I'm thinking somehow I can use trap but I don't see an easy way to get the exit value of a child inside the handler.

那么,如何可以做到这一点任何想法?

So, any ideas on how this can be done?

嗯,这里是概念code,它也许应该工作的一些证据,但它打破庆典:无效的命令行生成,挂,有时一个核心转储

Well, here is some proof of concept code that should probably work, but it breaks bash: invalid command lines generated, hanging, and sometimes a core dump.

# need monitor mode for trap CHLD to work
set -m
# store the PIDs of the children being watched
declare -a child_pids

function child_done
{
    echo "Child $1 result = $2"
}

function check_pid
{
    # check if running
    kill -s 0 $1
    if [ $? == 0 ]; then
        child_pids=("${child_pids[@]}" "$1")
    else
        wait $1
        ret=$?
        child_done $1 $ret
    fi
}

# check by copying pids, clearing list and then checking each, check_pid
# will add back to the list if it is still running
function check_done
{
    to_check=("${child_pids[@]}")
    child_pids=()

    for ((i=0;$i<${#to_check};i++)); do
        check_pid ${to_check[$i]}
    done
}

function run_command
{
    "$@" &
    pid=$!
    # check this pid now (this will add to the child_pids list if still running)
    check_pid $pid
}

# run check on all pids anytime some child exits
trap 'check_done' CHLD

# test
for ((tl=0;tl<10;tl++)); do
    run_command bash -c "echo FAIL; sleep 1; exit 1;"
    run_command bash -c "echo OKAY;"
done

# wait for all children to be done
wait

请注意,这不是我最终希望,反而会铺垫得到我想要的东西。

Note that this isn't what I ultimately want, but would be groundwork to getting what I want.

跟进:我已经实现了一个系统,在Python做到这一点。因此,使用Python的脚本任何人都可以具备上述功能。请参阅

Followup: I've implemented a system to do this in Python. So anybody using Python for scripting can have the above functionality. Refer to shelljob

推荐答案

是awesomesauce:

GNU Parallel is awesomesauce:

$ parallel -j2 < commands.txt
$ echo $?

有将设置退出状态到失败的命令数。如果你有超过253的命令,检查 - joblog 。如果你不知道所有的命令前面,看看 - BG

It will set the exit status to the number of commands that failed. If you have more than 253 commands, check out --joblog. If you don't know all the commands up front, check out --bg.

这篇关于并行,跟踪结果和计数运行的bash命令的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

11-02 08:09