Summary
File I/O on Microsoft Windows can be synchronous or asynchronous. The default behavior for I/O is synchronous, where an I/O function is called and returns when the I/O is complete. Asynchronous I/O allows an I/O function to return execution back to the caller immediately, but the I/O is not assumed to be complete until some future time. The operating system notifies the caller when the I/O is complete. Alternatively, the caller can determine the status of the outstanding I/O operation by using services of the operating system.
The advantage ofasynchronous I/O is that the caller has time to do other work or issuemore requests while the I/O operation is being completed. The termOverlapped I/O is frequently used for Asynchronous I/O andNon-overlapped I/O for Synchronous I/O. This article uses the termsAsynchronous and Synchronous for I/O operations. This article assumesthe reader has familiarity with the File I/O functions such asCreateFile, ReadFile, WriteFile.
Frequently, asynchronous I/Ooperations behave just as synchronous I/O. Certain conditions that thisarticle discusses in the later sections make the I/O operations complete synchronously. The caller has no time for background work because theI/O functions do not return until the I/O is complete.
Severalfunctions are related to synchronous and asynchronous I/O. This articleuses ReadFile and WriteFile as examples; good alternatives would beReadFileEx and WriteFileEx. Although this article discusses only diskI/O specifically, many of the principles can be applied to other typesof I/O, such as serial I/O or network I/O.
The advantage ofasynchronous I/O is that the caller has time to do other work or issuemore requests while the I/O operation is being completed. The termOverlapped I/O is frequently used for Asynchronous I/O andNon-overlapped I/O for Synchronous I/O. This article uses the termsAsynchronous and Synchronous for I/O operations. This article assumesthe reader has familiarity with the File I/O functions such asCreateFile, ReadFile, WriteFile.
Frequently, asynchronous I/Ooperations behave just as synchronous I/O. Certain conditions that thisarticle discusses in the later sections make the I/O operations complete synchronously. The caller has no time for background work because theI/O functions do not return until the I/O is complete.
Severalfunctions are related to synchronous and asynchronous I/O. This articleuses ReadFile and WriteFile as examples; good alternatives would beReadFileEx and WriteFileEx. Although this article discusses only diskI/O specifically, many of the principles can be applied to other typesof I/O, such as serial I/O or network I/O.
More Information
Set Up Asynchronous I/O
The FILE_FLAG_OVERLAPPED flag must be specified in CreateFile when the file is opened. This flag allows I/O operations on the file to be performed asynchronously. Here is an example: HANDLE hFile;
hFile = CreateFile(szFileName,
GENERIC_READ,
0,
NULL,
OPEN_EXISTING,
FILE_FLAG_NORMAL | FILE_FLAG_OVERLAPPED,
NULL);
if (hFile == INVALID_HANDLE_VALUE)
ErrorOpeningFile();
There are many things a program can do while waiting for asynchronous operations to complete,such as queuing additional operations, or doing background work. Forexample, the following code properly handles overlapped andnon-overlapped completion of a read operation. It does nothing more than wait for the outstanding I/O to complete:
if (!ReadFile(hFile,
pDataBuf,
dwSizeOfBuffer,
&NumberOfBytesRead,
&osReadOperation )
{
if (GetLastError() != ERROR_IO_PENDING)
{
// Some other error occurred while reading the file.
ErrorReadingFile();
ExitProcess(0);
}
else
// Operation has been queued and
// will complete in the future.
fOverlapped = TRUE;
}
else
// Operation has completed immediately.
fOverlapped = FALSE;
if (fOverlapped)
{
// Wait for the operation to complete before continuing.
// You could do some background work if you wanted to.
if (GetOverlappedResult( hFile,
&osReadOperation,
&NumberOfBytesTransferred,
TRUE))
ReadHasCompleted(NumberOfBytesTransferred);
else
// Operation has completed, but it failed.
ErrorReadingFile();
}
else
ReadHasCompleted(NumberOfBytesRead);
If, on the other hand, an operation is completed immediately, then &NumberOfBytesRead passed into ReadFile is validfor the number of bytes read. In this case, ignore the OVERLAPPEDstructure passed into ReadFile; do not use it with GetOverlappedResultor WaitForSingleObject.
Another caveat with asynchronousoperation is that you must not use an OVERLAPPED structure until itspending operation has completed. In other words, if you have threeoutstanding I/O operations, you must use three OVERLAPPED structures. If you reuse an OVERLAPPED structure, you will receive unpredictableresults in the I/O operations and you may experience data corruption.Additionally, before you can use an OVERLAPPED structure for the firsttime, or before you reuse it after an earlier operation has completed,you must correctly initialize it so no left-over data affects the newoperation.
The same type of restriction applies to the databuffer used in an operation. A data buffer must not be read or writtenuntil its corresponding I/O operation has completed; reading or writingthe buffer may cause errors and corrupted data.
Asynchronous I/O Still Appears to be Synchronous
If you followed the instructions earlier in this article, however, allyour I/O operations still typically complete synchronously in the orderissued, and none of the ReadFile operations returns FALSE withGetLastError() returning ERROR_IO_PENDING, this means you have no timefor any background work. Why does this occur?There are a number of reasons why I/O operations complete synchronously even if you have coded for asynchronous operation:
Compression
One obstruction to asynchronous operation is NTFS compression. The filesystem driver will not access compressed files asynchronously; insteadall operations are just made synchronous. This does not apply to filesthat are compressed with utilities similar to COMPRESS or PKZIP.NTFS Encryption
Similar to Compression, file encryption causes the system driver to convertasynchronous I/O to synchronous. If the files are decrypted, the I/Orequests will be asynchronous.Extending a File
Another reason that I/O operations are completed synchronously is theoperations themselves. On Windows NT, any write operation to a file that extends its length will be synchronous.
NOTE: Applications can make the previously mentioned write operationasynchronous by changing the Valid Data Length of the file by using the SetFileValidData function, and then issuing a WriteFile.
UsingSetFileValidData (which is available on Windows XP and later versions),applications can efficiently extend files without incurring aperformance penalty for zero-filling them.
Because the NTFS filesystem does not zero-fill the data up to the valid data length (VDL)that is defined by SetFileValidData, this function has securityimplications where the file may be assigned clusters that werepreviously occupied by other files. Therefore, SetFileValidDatarequires that the caller have the new SeManageVolumePrivilege enabled(by default, this is assigned only to administrators). Microsoftrecommends that ISVs carefully consider the implications of using thisfunction.
Cache
Most I/O drivers (disk,communications, and others) have special case code where, if an I/Orequest can be completed "immediately," the operation will be completedand the ReadFile or WriteFile function will return TRUE. In all ways,these types of operations appear to be synchronous. For a disk device,typically, an I/O request can be completed "immediately" when the datais cached in memory.Data Is not in Cache
The cache scheme can work against you, however, if the data is not in thecache. The Windows NT cache is implemented internally using filemappings. The memory manager in Windows NT does not provide anasynchronous page fault mechanism to manage the file mappings used bythe cache manager. The cache manager can, however, verify whether therequested page is in memory, so if you issue an asynchronous cachedread, and the pages are not in memory, the file system driver assumesthat you do not want your thread blocked and the request will be handled by a limited pool of worker threads. Control is returned to yourprogram after your ReadFile call with the read still pending.This works fine for a small number of requests, but because the pool ofworker threads is limited (currently three on a 16MB system), there will still be only a few requests queued to the disk driver at a particulartime. If you issue a lot of I/O operations for data that is not in thecache, the cache manager and memory manager become saturated and yourrequests are made synchronous.
The behavior of the cachemanager can also be influenced based on whether you access a filesequentially or randomly. Benefits of the cache are seen most whenaccessing files sequentially. The FILE_FLAG_SEQUENTIAL_SCAN flag in theCreateFile call will optimize the cache for this type of access.However, if you access files in a random fashion, use theFILE_FLAG_RANDOM_ACCESS flag in CreateFile to instruct the cache manager to optimize its behavior for random access.
Do not Use the Cache
The FILE_FLAG_NO_BUFFERING flag has the most effect on the behavior of thefile system for asynchronous operation. This is the best way toguarantee that I/O requests are actually asynchronous. It instructs thefile system to not use any cache mechanism at all.WARNING: There are some restrictions to using this flag that have to do with the data buffer alignment and the device's sector size. See the functionreference in the documentation for the CreateFile function for moreinformation about using this flag properly.
Real World Test Results
Following are some test results from the sample code. The magnitude of thenumbers is not important here and varies from computer to computer, butthe relationship of the numbers compared to each other illuminates thegeneral affect of the flags on performance.You can expect to see results similar to the following:
- Test 1
This test demonstrates that the previously mentioned program issued 500 I/O requests quickly and had a lot of time to do other work or issue more requests.
Asynchronous, unbuffered I/O: asynchio /f*.dat /n
Operations completed out of the order in which they were requested.
500 requests queued in 0.224264 seconds.
500 requests completed in 4.982481 seconds.
- Test 2
This test demonstrates that this program spent 4.495880 seconds calling ReadFile to complete its operations, whereas the test 1 spent only 0.224264 seconds to issue the same requests. In test 2, there was no "extra" time for the program to do any background work.
Synchronous, unbuffered I/O: asynchio /f*.dat /s /n
Operations completed in the order issued.
500 requests queued and completed in 4.495806 seconds.
- Test 3
This test demonstrates the synchronous nature of the cache. All reads were issued and completed in 0.251670 seconds. In other words, asynchronous requests were completed synchronously. This test also demonstrates the high performance of the cache manager when data is in the cache.
Asynchronous, buffered I/O: asynchio /f*.dat
Operations completed in the order issued.
500 requests issued and completed in 0.251670 seconds.
- Test 4
This test demonstrates the same results as in test 3. Note that synchronous reads from the cache complete a little faster than asynchronous reads from the cache. This test also demonstrates the high performance of the cache manager when data is in the cache.
Synchronous, buffered I/O: asynchio /f*.dat /s
Operations completed in the order issued.
500 requests and completed in 0.217011 seconds.
CONCLUSION
You can decide which method is best because it all depends on the type, size, and number of operations that your program performs.The default file access without specifying any special flags to CreateFile is a synchronous and cached operation.
NOTE: You do get some automatic asynchronous behavior in this mode becausethe file system driver does predictive asynchronous read-ahead andasynchronous lazy writing of modified data. Although this does not makethe application's I/O asynchronous, it is the ideal case for the vastmajority of simple applications.
If, on the other hand, yourapplication is not simple, you may have to do some profiling andperformance monitoring to determine the best method, similar to thetests illustrated earlier in this article. Profiling the time spent inthe ReadFile or WriteFile function and then comparing this time to howlong it takes for actual I/O operations to complete is extremely useful. If the majority of the time is spent in actually issuing the I/O, thenyour I/O is being completed synchronously. However, if the time spentissuing I/O requests is relatively small compared to the time it takesfor the I/O operations to complete, then your operations are beingtreated asynchronously. The sample code mentioned earlier in thisarticle uses the QueryPerformanceCounter function to do its own internal profiling.
Performance monitoring can help determine howefficiently your program is using the disk and the cache. Tracking anyof the performance counters for the Cache object will indicate theperformance of the cache manager. Tracking the performance counters forthe Physical Disk or Logical Disk objects will indicate the performanceof the disk systems.
There are several utilities that arehelpful in performance monitoring; PerfMon and DiskPerf are especiallyuseful. For the system to collect data on the performance of the disksystems, you must first issue the diskperf -y command. After you issuethe command, you must restart the system to start the data collection.
References
https://support.microsoft.com/zh-cn/help/156932/asynchronous-disk-i-o-appears-as-synchronous-on-windows
上次更新时间：2015年6月25日