问题描述
HtmlHelper.GetTagsAndValues(htmlContent);
我得到这个错误:
at System.String.Split(String[] separator, Int32 count, StringSplitOptions options)
at System.String.Split(String[] separator, StringSplitOptions options)
at WebCrawler.Logic.CrawlerManager.UseRulesOnHtmlPage(Agencies agency, String pageUrl, List`1 listTagValuePair, RulesGroups ruleGroup) in D:\PROJEKTI\crawler\WebCrawlerSuite\WebCrawler.Logic\CrawlerManager.cs:line 263
at WebCrawler.Logic.CrawlerManager.GetAdvertismentFromHtmlContent(List`1 listTagValuePair, Agencies agency, String pageUrl) in D:\PROJEKTI\crawler\WebCrawlerSuite\WebCrawler.Logic\CrawlerManager.cs:line 191
at WebCrawler.Logic.CrawlerManager.ImportAdvertisment2Database.Work(Crawler crawler, PropertyBag propertyBag) in D:\PROJEKTI\crawler\WebCrawlerSuite\WebCrawler.Logic\CrawlerManager.cs:line 668
at WebCrawler.Logic.CrawlerManager.ImportAdvertisment2Database.Process(Crawler crawler, PropertyBag propertyBag) in D:\PROJEKTI\crawler\WebCrawlerSuite\WebCrawler.Logic\CrawlerManager.cs:line 584
我读了这篇文章:
如何防止此错误?
整个方法:
public static List<TagValuePair> GetTagsAndValues(string htmlContent)
{
List<TagValuePair> tagsValues = new List<TagValuePair>();
Dictionary<string, int> tagAppearance = new Dictionary<string, int>();
HtmlDocument doc = new HtmlDocument();
if (htmlContent != null)
{
doc.LoadHtml(htmlContent);
if (doc.DocumentNode.SelectNodes("//*") == null)
{
List<TagValuePair> tempList = new List<TagValuePair>();
tempList.Add(new TagValuePair("Error!", htmlContent, -1));
return tempList;
}
foreach (HtmlNode tag in doc.DocumentNode.SelectNodes("//*"))
{
try
{
if (!string.IsNullOrEmpty(tag.InnerHtml.Trim()))
{
if (!tagAppearance.Keys.Contains(tag.Name))
{
tagAppearance.Add(tag.Name, 1);
}
else
tagAppearance[tag.Name] = tagAppearance[tag.Name] + 1;
tagsValues.Add(new TagValuePair(tag.Name, tag.InnerHtml.Trim(), tagAppearance[tag.Name]));
}
else
{
// Help link: http://refactoringaspnet.blogspot.com/2010/04/using-htmlagilitypack-to-get-and-post_19.html
if (!string.IsNullOrEmpty(tag.GetAttributeValue("value", "").Trim()))
{
if (!tagAppearance.Keys.Contains("option value"))
{
tagAppearance.Add("option value", 1);
}
else
tagAppearance["option value"] = tagAppearance["option value"] + 1;
tagsValues.Add(new TagValuePair("option value", tag.GetAttributeValue("value", "").Trim(), tagAppearance["option value"]));
}
if (tag.NextSibling != null && !string.IsNullOrEmpty(tag.NextSibling.InnerHtml.Trim()))
{
if (!tagAppearance.Keys.Contains(tag.Name))
{
tagAppearance.Add(tag.Name, 1);
}
else
tagAppearance[tag.Name] = tagAppearance[tag.Name] + 1;
tagsValues.Add(new TagValuePair(tag.Name, tag.NextSibling.InnerHtml.Trim(), tagAppearance[tag.Name]));
}
}
}
catch (Exception)
{
return null;
}
}
}
确切的错误在这里:
doc.LoadHtml(htmlContent);
推荐答案
我建议您查看一个内存分析器，以确保您的应用程序中没有任何泄漏.假设您说它是在应用程序工作12个小时后发生的，这似乎表明它可能是一个缓慢的泄漏，最终导致OutOfMemory异常.
I would suggest looking at a memory profiler to ensure you haven't got any leaks in your application. Given you say it occurs after 12 hours of app working, it seems to indicate that it may be a slow leak that eventually causes the OutOfMemory exception.
您可以通过多种方式单调地保留引用，这将导致缓慢的泄漏.运行探查器将帮助您识别这些问题.可能不是导致问题的一行代码.可能只是一行代码经常向您显示将骆驼折断的稻草.
There are a number of ways that you can unitentionally hold onto references that will cause a slow leak. Running a profiler will help you identify these issues. It may not be the one line of code that is causing the problem. It may just be that the one line of code is often showing you the straw that breaks the camels back.
我之前曾使用 Redgates Ants Profiler (它提供14天的免费试用)，它帮助我减少了内存使用量并提高了性能.我最近似乎经常使用此工具，但这完全是因为我发现它是一个非常有价值的工具.
I have used Redgates Ants Profiler before (it comes with a 14 day free trial), and it helped me heaps to get memory usage down and increase performance. I seem to be plugging this a lot recently, but it is purely due to the fact I find it to be a very valuable tool.
Take a look through some of their walkthroughs and/or vidoes to see how to track down a leak.
这篇关于在C#中引发了类型为'System.OutOfMemoryException'的异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！