问题描述
我正在研究一个asp.net MVC-5 Web应用程序，并且根据一些文章，我读到我尤其不应在Web服务器和.net Web应用程序内部使用并行方法.现在以我为例，我需要在一个foreach中发出大约1,500个 WebClient()调用，然后从 WebClient()调用中反序列化返回的json对象.我在使用 Parallel.Foreach 之前的原始代码如下，大约需要 15 分钟才能完成:-
I am working on an asp.net MVC-5 web application, and based on some articles i read that i should not use Parallel methods inside web servers and inside .net web applications espically.now in my case i have around 1,500 WebClient() calls that i need to issue inside a foreach, and then deserialize the return json object from the WebClient() calls. my original code before using Parallel.Foreach was as follow, which took around 15 minutes to complete:-
public async Task <List<Details2>> Get()
{
try
{
using (WebClient wc = new WebClient())
{
string url = currentURL + "resources?AUTHTOKEN=" + pmtoken;
var json = await wc.DownloadStringTaskAsync(url);
resourcesinfo = JsonConvert.DeserializeObject<ResourcesInfo>(json);
}
ForEach( var c in resourcesinfo.operation.Details)
{
ResourceAccountListInfo resourceAccountListInfo = new ResourceAccountListInfo();
using (WebClient wc = new WebClient())
{
string url = currentURL + "resources/" + c.RESOURCEID + "/accounts?AUTHTOKEN=" + pmtoken;
string tempurl = url.Trim();
var json = await wc.DownloadStringTaskAsync(tempurl);
resourceAccountListInfo = JsonConvert.DeserializeObject<ResourceAccountListInfo>(json);
}
if (resourceAccountListInfo.operation.Details.CUSTOMFIELD.Count > 0)
{
List<CUSTOMFIELD> customfield = resourceAccountListInfo.operation.Details.CUSTOMFIELD.Where(a =>
a.CUSTOMFIELDLABEL.ToLower() == "name"
).ToList();
if (customfield.Count == 1)
{
PMresourcesOnly.Add(resourceAccountListInfo.operation.Details);
}
}
}//end of foreach
return PMresourcesOnly.ToList();
}
catch (Exception e)
{
}
return new List<Details2>();
}
现在我做了以下修改:-
now i did the following modifications :-
- 我将
foreach替换为Parallel.ForEach -
因为我不应该在
Parallel.ForEach内部使用异步方法，所以我将DownloadStringTaskAsync更改为内部的:-DownloadStringParallel.Foreach
- i replace
foreachwithParallel.ForEach since i should not use async methods inside
Parallel.ForEachso i chnage theDownloadStringTaskAsynctoDownloadStringinside theParallel.Foreach:-
public async Task <List<Details2>> Get()
{
try
{
using (WebClient wc = new WebClient())
{
string url = currentURL + "resources?AUTHTOKEN=" + pmtoken;
var json = await wc.DownloadStringTaskAsync(url);
resourcesinfo = JsonConvert.DeserializeObject<ResourcesInfo>(json);
}
Parallel.ForEach(resourcesinfo.operation.Details, new ParallelOptions { MaxDegreeOfParallelism = 7 }, (c) =>
{
ResourceAccountListInfo resourceAccountListInfo = new ResourceAccountListInfo();
using (WebClient wc = new WebClient())
{
string url = currentURL + "resources/" + c.RESOURCEID + "/accounts?AUTHTOKEN=" + pmtoken;
string tempurl = url.Trim();
var json = wc.DownloadString(tempurl);
resourceAccountListInfo = JsonConvert.DeserializeObject<ResourceAccountListInfo>(json);
}
if (resourceAccountListInfo.operation.Details.CUSTOMFIELD.Count > 0)
{
List<CUSTOMFIELD> customfield = resourceAccountListInfo.operation.Details.CUSTOMFIELD.Where(a =>
a.CUSTOMFIELDLABEL.ToLower() == "name"
).ToList();
if (customfield.Count == 1)
{
PMresourcesOnly.Add(resourceAccountListInfo.operation.Details);
}
}
});//end of foreach
return PMresourcesOnly.ToList();
}
catch (Exception e)
{
}
return new List<Details2>();
}
现在，当我使用 Parallel.Foreach 时，执行时间从 15 分钟减少到大约 7 分钟.但是如果我的第二种方法有效，我有点困惑，那么任何人都可以在这些问题(或任何问题)上提出意见:-
now when i use the Parallel.Foreach the execution time was reduced from 15 minutes to around 7 minutes. but i am a bit confused if my second method is valid , so can anyone adivce on these questions (or any question):-
-
是否将
Parallel.Foreach与Webclient()一起使用是遵循的有效方法?还是我应该避免在.net和Web应用程序内部使用并行方法?
is using
Parallel.ForeachwithWebclient()a valid approach to follow ? or i should avoid using Parallel methods inside .net and web applications?
在使用 Parallel.Foreach 时，我可能会遇到任何问题，例如 return PMresourcesOnly.ToList(); 返回到客户端时，仍然有一些未完成的 wc.DownloadString(tempurl); ?
when using Parallel.Foreach could i face any problem such as that the return PMresourcesOnly.ToList(); is return to the client while there are still some wc.DownloadString(tempurl); that did not complete?
如果我想比较两种方法(Parallel.Foreach和Foreach)，结果将是相同的吗?
if i want to compare the 2 methods (Parallel.Foreach & Foreach) will the result be the same ?
在一些在线文章上，他们使用 Task.Factory.StartNew(()而不是使用 Parallel.foreach ，所以它们之间的主要区别是什么?
on some online articles they use Task.Factory.StartNew(() instead of using Parallel.foreach so what are the main differences between them ?
编辑我尝试如下定义 SemaphoreSlim :-
public async Task <List<Details2>> Get()
{
SemaphoreSlim throttler = new SemaphoreSlim(initialCount: 15);
try
{
//code goes here
var tasks = resourcesinfo.operation.Details.Select(c => TryDownloadResourceAsync(c.RESOURCEID,throttler)).ToList();
}
///---
private async Task<Details2> TryDownloadResourceAsync(string resourceId, SemaphoreSlim throttler)
{
await throttler.WaitAsync();
try
{
using (WebClient wc = new WebClient()) //get the tag , to check if there is a server with the same name & tag..
{}
}
finally
{
throttler.Release();
}
推荐答案
不，您绝对应该避免在ASP.NET应用程序内部使用并行方法.
No, you absolutely should avoid using parallel methods inside ASP.NET apps.
并行用于数据并行性(在一组数据项上运行相同的CPU绑定代码). StartNew 适用于动态任务并行性(正在运行项上的相同或不同的CPU绑定代码会随着您的处理而发生变化).
Parallel is for data parallism (running the same CPU-bound code over a collection of data items). StartNew is for dynamic task parallelism (running the same or different CPU-bound code over a collection of items that changes as you process it).
这两种方法都不适用，因为您要做的工作是I/O绑定的，而不是CPU绑定的.
Neither approach is appropriate here, since the work you have to do is I/O-bound, not CPU-bound.
您真正想要的是并发性(一次执行多项操作)，而不是 parallelism .与其使用并行并发(通过使用多个线程一次执行多项操作)，您想要的是异步并发(不使用线程一次执行多项操作).
What you actually want is concurrency (doing multiple things at a time), not parallelism. Instead of using parallel concurrency (doing multiple things at a time by using multiple threads), what you want is asynchronous concurrency (doing multiple things at a time using no threads).
可以通过 await Task.WhenAll 在代码中实现异步并发，例如:
Asynchronous concurrency is possible in code via await Task.WhenAll, as such:
private async Task<string> TryDownloadResourceAsync(string resourceId)
{
ResourceAccountListInfo resourceAccountListInfo = new ResourceAccountListInfo();
using (WebClient wc = new WebClient())
{
string url = currentURL + "resources/" + resourceId + "/accounts?AUTHTOKEN=" + pmtoken;
string tempurl = url.Trim();
var json = await wc.DownloadStringTaskAsync(tempurl);
resourceAccountListInfo = JsonConvert.DeserializeObject<ResourceAccountListInfo>(json);
}
if (resourceAccountListInfo.operation.Details.CUSTOMFIELD.Count > 0)
{
List<CUSTOMFIELD> customfield = resourceAccountListInfo.operation.Details.CUSTOMFIELD.Where(a =>
a.CUSTOMFIELDLABEL.ToLower() == "name"
).ToList();
if (customfield.Count == 1)
{
return resourceAccountListInfo.operation.Details;
}
}
return null;
}
public async Task <List<Details2>> Get()
{
try
{
using (WebClient wc = new WebClient())
{
string url = currentURL + "resources?AUTHTOKEN=" + pmtoken;
var json = await wc.DownloadStringTaskAsync(url);
resourcesinfo = JsonConvert.DeserializeObject<ResourcesInfo>(json);
}
var tasks = resourcesinfo.operation.Details.Select(c => TryDownloadResourceAsync(c.RESOURCEID)).ToList();
var results = await Task.WhenAll(tasks).Select(x => x != null);
return results.ToList();
}
catch (Exception e)
{
}
return new List<Details2>(); // Please, please don't do this in production.
}
最后一点，您可能想研究 HttpClient ，它是为异步操作而设计的，具有不错的属性，对于任何操作，您都只需要一个 同时通话的数量.
As a final note, you may want to look into HttpClient, which was designed for asynchronous operations and has the nice property that you only need one of them for any number of simultaneous calls.
这篇关于在asp.net mvc Web应用程序中将Parallel.Foreach与WebClient()一起使用是否有任何缺点或风险的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！