使用async-await的并行多线程下载

问题描述

我的Windows服务-C#中有数百个要从网络下载的大文件.要求一次维护一次-最多可下载4个并行Web文件.

I have 100s of multiple big files to download from web in my windows service - c#. The requirement is to maintain at one time - max 4 parallel web file downloads.

我可以使用async等待来实现并发/并行下载，还是必须使用BackgroundWorker进程或线程?是异步等待多线程吗?请参阅下面的使用async-await的示例程序:

Can I achieve concurrent/parallel downloads using async await or do I have to use BackgroundWorker process or threads ? Is async-await multithreaded ?See my sample Program using async-await below:

    static int i = 0;

    Timer_tick()
    {
      while(i < 4)
      {
        i++;
        model = GetNextModel();
        await Download(model);
      }
    }

    private async Download(XYZ model)
    {
    Task<FilesetResult> t = DoWork(model);
    result = await t;
    //Use Result
    }

    private async Task<FilesetResult> Work(XYZ model)
    {
    fileresult = await api.Download(model.path)
    i--;
    return filesetresult;
    }

推荐答案

您可以使用SemaphoreSlim类限制并行运行的异步任务的数量.像这样:

You can limit number of async tasks running in parallel using SemaphoreSlim class. Something like:

List<DownloadRequest> requests = Enumerable.Range(0, 100).Select(x => new DownloadRequest()).ToList();
using (var throttler = new SemaphoreSlim(4))
{
    Task<DownloadResult>[] downloadTasks = requests.Select(request => Task.Run(async () =>
    {
        await throttler.WaitAsync();
        try
        {
            return await DownloadTaskAsync(request);
        }
        finally
        {
            throttler.Release();
        }
    })).ToArray();
    await Task.WhenAll(downloadTasks);
}

更新:感谢您的评论和已解决的问题.

Update: thank you for comments, fixed issues.

Update2 :动态请求列表示例解决方案

Update2: Sample solution for dynamic list of requests

public class DownloadManager : IDisposable
{
    private readonly SemaphoreSlim _throttler = new SemaphoreSlim(4);

    public async Task<DownloadResult> DownloadAsync(DownloadRequest request)
    {
        await _throttler.WaitAsync();
        try
        {
            return await api.Download(request);
        }
        finally
        {
            _throttler.Release();
        }
    }

    public void Dispose()
    {
        _throttler?.Dispose();
    }
}

这篇关于使用async-await的并行多线程下载的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！