如何在多进程系统中实现锁定?

问题描述

我们正在并行运行许多jenkins项目.我们正在使用python，我们选择了使用pyenv管理虚拟环境.不幸的是，pyenv具有众所周知的比赛条件.要变通解决此问题，我想在流程级别实现锁定.我想做的是:

We are running lots of jenkins projects in parallel. We are using python, and we have chosen to manage the virtual environments with pyenv. Unfortunately, pyenv has a well-known race condition. To work around the problem, I would like to implement locking at the process level. What I want to do is:

lock some resource (a file?)
do my pyenv stuff
unlock the resource

我的脚本是用bash编写的.如何在bash中实现资源锁定/解锁?

My scripts are written in bash. How can I implement resource locking / unlocking in bash?

推荐答案

因此，想要跨进程锁定时，您在Unix世界中的朋友是一个名为flock的命令.它在OS级别上作为原子操作实现，对于这种事情非常有用.您可以在此处阅读更多相关信息.这是使用方法:

So your friend in the unix world when wanting a cross-process lock is a command called flock. It is implemented as an atomic operation at the OS level and is extremely useful for this sort of thing. You can read more about it here. Here is how you can use it:

  # Wait for lock on  (fd 222) for 10 seconds
  (flock -w 10 222 || exit 1

  {
      # Do the operations you want to here
  }) 222>/path/to/lockfile

---

这里有几个技巧.首先，通常在使用输出重定向时，bash会先打开文件，然后再尝试尝试flock.但是，在这里，因为我们有()，bash首先会启动第一个命令为flock的子shell. flock将尝试获得对文件句柄222的锁定.然后，flock将锁定文件描述符.锁定文件描述符后，将运行{}中的代码.在运行之后，将文件描述符222的内容写入锁定文件(即什么都没有)，关闭文件并释放锁定.就像C一样，关闭文件会释放一个锁.当然，没有什么比杰出的@CharlesDuffy(帽子技巧@codeforester)更能说明问题的了，后者解释了.

---

There are several tricks here. First, normally when using output redirection, bash will open a file first before even attempting the flock. Here, though, because we have the () bash will first kick off a subshell whose first command is flock. flock will attempt to obtain a lock on file handle 222. Flock will then lock the file descriptor. After locking the file descriptor, the code in the {} is run. After that is run, the contents of file descriptor 222 are written to the lock file (i.e., nothing), the file is closed and the lock is released. This is just like C where closing a file releases a lock. Of course, no on explains it better than the illustrious @CharlesDuffy (hat tip @codeforester) who explains what is going on here.

这篇关于如何在多进程系统中实现锁定?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！