问题描述
我试图清理与未刷新文件时发生问题的I / O在几个方案，在不同的语言，在Linux上运行的缓冲区。冲洗缓冲区的解决方案是很容易的，但是未刷新缓冲区的这个问题发生相当随意。而不是追求什么可能会导致它的帮助下，我感兴趣的是如何创建（复制）和诊断这种情况。
I am trying to clear up an issue that occurs with unflushed file I/O buffers in a couple of programs, in different languages, running on Linux. The solution of flushing buffers is easy enough, but this issue of unflushed buffers happens quite randomly. Rather than seek help on what may cause it, I am interested in how to create (reproduce) and diagnose this kind of situation.
这导致了两个部分的问题：
This leads to a two-part question:
-
它是人为地构建情况下是可行的地方，对于给定的时间内，人们可以有输出缓冲器的被称为的是未刷新？我搜索都转向了空。一个平凡的基准是锤硬盘驱动器（例如交换）在一个进程尝试从另一个进程写入大量数据。虽然这种作品，它使系统无法使用实际。我不能闲逛，看看发生了什么事情
Is it feasible to artificially and easily construct instances where, for a given period of time, one can have output buffers that are known to be unflushed? My searches are turning up empty. A trivial baseline is to hammer the hard drive (e.g. swapping) in one process while trying to write a large amount of data from another process. While this "works", it makes the system practically unusable: I can't poke around and see what's going on.
从在Linux命令有能够识别一个给定的进程未刷新文件输出缓冲？这是不是可以在命令行中运行，或者是否有必要直接查询内核？我一直在寻找 FSYNC ，同步，的ioctl ， 刷新，的bdflush ，及其他。然而，缺乏创建缓冲区未刷新的方法，目前还不清楚这些东西可能揭示。
Are there commands from within Linux that can identify that a given process has unflushed file output buffers? Is this something that can be run at the command line, or is it necessary to query the kernel directly? I have been looking at fsync, sync, ioctl, flush, bdflush, and others. However, lacking a method for creating unflushed buffers, it's not clear what these may reveal.
为了重现他人，在C＃1的例子是优秀的，但问题是真正的语言无关 - 只知道一种方法来创建这个情况，我在工作的其他语言将帮助
In order to reproduce for others, an example for #1 in C would be excellent, but the question is truly language agnostic - just knowing an approach to create this situation would help in the other languages I'm working in.
更新1：我对任何混乱表示歉意。正如一些人指出，缓冲区可以在内核空间和用户空间。这有助于发现问题：我们正在创造大脏内核缓冲区。这种区分和答案彻底解决＃1：现在看来很清楚如何重新创建在任何用户空间或内核空间未刷新缓冲区。确定哪些进程ID有脏内核缓冲区尚不清楚，但。
Update 1: My apologies for any confusion. As several people have pointed out, buffers can be in the kernel space or the user space. This helped pinpoint the problems: we're creating big dirty kernel buffers. This distinction and the answers completely resolve #1: it now seems clear how to re-create unflushed buffers in either user space or kernel space. Identifying which process ID has dirty kernel buffers is not yet clear, though.
推荐答案
如果您有兴趣在内核缓冲数据，那么你可以调整通过的/ proc / sys目录中的sysctl虚拟机回写/ VM /脏_ * 。尤其是， dirty_expire_centisecs 是年龄，在第二，在此脏数据变得适合写回的百分之一。增加该值会给你的时间在做你的调查，一个更大的窗口。您也可以增加 dirty_ratio 和 dirty_background_ratio （这是系统内存的百分比，确定在该同步和异步回写点分别启动）。
If you are interested in the kernel-buffered data, then you can tune the VM writeback through the sysctls in /proc/sys/vm/dirty_*. In particular, dirty_expire_centisecs is the age, in hundredths of a second, at which dirty data becomes eligible for writeback. Increasing this value will give you a larger window of time in which to do your investigation. You can also increase dirty_ratio and dirty_background_ratio (which are percentages of system memory, defining the point at which synchronous and asynchronous writeback start respectively).
实际创建脏页很容易 - 只需写（2）来在 MAP_SHARED 文件的映射。
Actually creating dirty pages is easy - just write(2) to a file and exit without syncing, or dirty some pages in a MAP_SHARED mapping of a file.
这篇关于创建未刷新文件输出缓冲器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！