fortran：等待打开一个文件，直到被另一个应用程序关闭

问题描述

我有一个fortran代码，需要读取一系列ascii数据文件（全部大约25 Gb）。基本上，代码打开给定的ascii文件，读取信息并使用它来执行一些操作，然后关闭它。然后打开另一个文件，读取信息，执行一些操作并再次关闭它。其余的ascii文件等。

总的来说，每次完整的运行大约需要10小时。我通常需要使用不同的参数运行多个独立的计算，我所做的是按顺序运行每个独立的计算，以便最后如果我有10个独立的计算，总CPU时间为100小时。

更快的方法是使用群集机器上的不同处理器同时运行10个独立计算，但问题是如果给定的计算需要打开并读取给出ascii文件已被打开，并被另一个计算使用，然后代码显然是一个错误。

我想知道是否有方法来验证是否ascii文件已被其他计算使用，如果是这样，要求代码等待ascii文件最终关闭。

任何帮助都会有很大的帮助。
非常感谢。

Obamakoak。

解决方案

我知道这是一个旧线程，但我一直在为我自己的代码挣扎。

我的第一个尝试是在某个进程上创建一个变量（例如主设备），并使用单向被动MPI专门访问该变量。这很花哨，并且效果很好，但只适用于更新版本的MPI。

另外，我的代码看起来很开心（使用 READWRITE status）文件，这些文件在其他进程中也是打开的。 b
$ b

em>外部锁定文件，如所述。在你的情况下，代码可能如下所示：






• 进程使用<$ c检查锁定文件是否存在$ c> NEW 语句，如果文件已经存在，则失败。它看起来像这样：

    file_exists = .true。 
   while（file_exists）
   open（STATUS ='NEW'，unit = 11，file = lock_file_name，iostat = open_stat）
   if（open_stat.eq.0）then 
   file_exists = .false。 
   open（STATUS ='OLD'，ACTION = READWRITE'，unit = 12，file = data_file_name，iostat = ierr）
   if（ierr.ne.0）stop 
   else 
   call sleep（1）
   end if 
   end do 
    




• 该文件现在仅由当前进程打开。做你需要做的操作，例如阅读，写作。

完成后，关闭数据文件，最后关闭锁定文件

$ b $

  close（12，iostat = ierr）
 if（ierr.ne.0）stop 
 close（11，status ='DELETE'，iostat = ierr）
 if（ierr.ne.0）stop 
  

我希望这对于其他进程可能有用。其他人有同样的问题。

I have a fortran code which needs to read a series of ascii data files (which all together are about 25 Gb). Basically the code opens a given ascii file, reads the information and use it to do some operations, and then close it. Then opens another file, reads the information, do some operations, and close it again. And so on with the rest of ascii files.

Overall each complete run takes about 10h. I usually need to run several independent calculations with different parameters, and the way I do is to run each independent calculation sequentially, so that at the end if I have 10 independent calculations, the total CPU time is 100h.

A more rapid way would be to run the 10 independent calculations at the same time using different processors on a cluster machine, but the problem is that if a given calculation needs to open and read data from a given ascii file which has been already opened and it's being used by another calculation, then the code gives obviously an error.

I wonder whether there is a way to verify if a given ascii file is already being used by another calculation, and if so to ask the code to wait until the ascii file is finally closed.

Any help would be of great help.Many thanks in advance.

Obamakoak.

解决方案

I know this is an old thread but I've been struggling with the same issue for my own code.

My first attempt was creating a variable on a certain process (e.g. the master) and accessing this variable exclusively using one-sided passive MPI. This is fancy and works well, but only with newer versions of MPI.

Also, my code seemed happy to open (with READWRITE status) files that were also open in other processes.

Therefore, the easiest workaround, if your program has file access, is to make use of an external lock file, as described here. In your case, the code might look something like this:

• A process checks whether the lock file exists using the NEW statement, which fails if a file already exists. It will look something like:

  file_exists = .true.
  do while (file_exists)
      open(STATUS='NEW',unit=11,file=lock_file_name,iostat=open_stat)
      if (open_stat.eq.0) then
          file_exists = .false.
          open(STATUS='OLD',ACTION=READWRITE',unit=12,file=data_file_name,iostat=ierr)
          if (ierr.ne.0) stop
      else
          call sleep(1)
      end if
  end do
  

• The file is now opened exclusively by the current process. Do the operations you need to do, such as reading, writing.

• When you are done, close the data file and finally the lock file

  close(12,iostat=ierr)
  if (ierr.ne.0) stop
  close(11,status='DELETE',iostat=ierr)
  if (ierr.ne.0) stop
  

• The data file is now again unlocked for the other processes.

I hope this may be useful for other people who have the same problem.

这篇关于fortran：等待打开一个文件，直到被另一个应用程序关闭的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！