在Python中，当不再引用memmap对象时，是否可以使Numpy的memmap重载以删除自身?

问题描述

当某些数据不适合内存使用时，我正在尝试使用memmap，并利用memmap的能力欺骗代码，以为它只是一个ndarray.为了进一步扩展使用memmap的方式，我想知道是否有可能使memmap的取消引用运算符重载以删除memmap文件.

I am trying to use memmap when certain data doesn't fit in memory and employ memmap's ability to trick code into thinking it's just an ndarray. To further expand on this way of using memmap I was wondering if it would be possible to overload memmap's dereference operator to delete the memmap file.

例如:

from tempfile import mkdtemp
import os.path as path
filename = path.join(mkdtemp(), 'tmpfile.dat')
{
    out = np.memmap(filename, dtype=a.dtype, mode='w+', shape=a.shape)
}
# At this point out is out of scope, so the overloaded 
# dereference function would delete tmpfile.dat

这听起来可行吗?有什么我没想到的吗?

Does this sound feasible/has this been done? Is there something I am not thinking of?

谢谢！

推荐答案

只需在np.memmap打开文件后删除该文件关闭对文件描述符的最后一个引用后，系统将删除该文件.

just delete the file after it has been opened by np.memmapthe file will then be deleted by the system after the last reference to the file descriptor is closed.

python临时文件的工作方式如下，可以非常方便地与with上下文管理器构造一起使用:

python temporary files work like this and can very conveniently be used with the with context manger construct:

with tempfile.NamedTemporaryFile() as f:
    # file gone now from the filesystem 
    # but f still holds a reference so it still exists and uses space (see /prof<pid>/fd)
    # open it again (will not work on windows)
    x = np.memmap(f.name, dtype=np.float64, mode='w+', shape=(3,4))
# file path is unlinked but exists on disk with the open file reference in x
del x
# now all references are gone and the file is properly deleted

这篇关于在Python中，当不再引用memmap对象时，是否可以使Numpy的memmap重载以删除自身?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！