IOError:[Errno 24]打开的文件太多:

问题描述

我有一个巨大的文件，大约要写入450个文件中.我收到错误，显示为too many files open.我在网上搜索并找到了一些解决方案，但没有帮助.

I have a huge file that I am writing into approximately 450 files. I am getting error as too many files open. I searched the web and found some solution but it is not helping.

import resource
resource.setrlimit(resource.RLIMIT_NOFILE, (1000,-1))
>>> len(pureResponseNames) #Filenames
434
>>> resource.getrlimit(resource.RLIMIT_NOFILE)
(1000, 9223372036854775807)
>>> output_files = [open(os.path.join(outpathDirTest, fname) + ".txt", "w") for fname in pureResponseNames]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 24] Too many open files: 'icd9_737.txt'
>>>

我还从命令行中更改了ulimit，如下所示:

I also changed ulimit from the command line as below:

$ ulimit -n 1200
$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1200
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited
$

我仍然遇到相同的错误.PS:我也重新启动了系统并运行程序，但没有成功.

I am still getting the same error.PS: I also restarted my system and run the program but with no success.

推荐答案

打开文件过多"的错误总是很棘手-您不仅要纠结于ulimit，而且还必须检查系统范围的限制和OSX特有的. 此SO帖子提供了有关OSX中打开文件的更多信息.(Spoiler警告:默认值为256).

"Too many open files" errors are always tricky – you not only have to twiddle with ulimit, but you also have to check system-wide limits and OSX-specifics. This SO post gives more information on open files in OSX. (Spoiler alert: the default is 256).

但是，通常很容易限制必须同时打开的文件数.如果我们看一下Stefan Bollman的示例，我们可以轻松地将其更改为:

However, it is often easy to limit the number of files that have to be open at the same time. If we look at Stefan Bollman's example, we can easily change that to:

pureResponseNames = ['f'+str(i) for i in range(434)]
outpathDirTest="testCase/"
output_files = [os.path.join(outpathDirTest, fname) + ".txt" for fname in pureResponseNames]

for filename in range(output_files):
    with open(filename, 'w') as f:
        f.write('This is a test of file nr.'+str(i))

这篇关于IOError:[Errno 24]打开的文件太多:的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！