问题描述
我从特定网站获取数据的多种尝试仍然失败.一个简单的WebClient DownloadString一直可以使用到最近，但是似乎该站点现在拒绝了任何不具有浏览器功能的东西.
My multitude of attempts to obtain data from a specific website continue to fail. A simple WebClient DownloadString used to work until just recently, but it appears the site is now rejecting anything that does not behave as a browser.
这是我正在尝试的:
string news = "" ;
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.Accept] = "text/html, application/xhtml+xml, image/jxr, */*" ;
client.Headers[HttpRequestHeader.AcceptLanguage] = "en-AU,en-US;q=0.8,en-GB;q=0.5,en;q=0.3" ;
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0" ;
client.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate" ;
news = client.DownloadString("https://www.forexfactory.com/calendar.php?week=this") ;
}
结果为System.Net.WebException:基础连接已关闭:发送中发生意外错误. ---> System.IO.IOException:从传输流接收到意外的EOF或0个字节.
And the result is System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Received an unexpected EOF or 0 bytes from the transport stream.
正如我所说，简单的DownloadString以前就返回了预期的结果.但是看来，为了减少网页抓取，该网站正在关闭非浏览器的请求.我的代码每天只执行几次，所以我想看起来像是 为此目的的浏览器.
As I said, the simple DownloadString alone previously returned the expected results. But it appears that in an effort to reduce web scraping, the site is closing off non-browser requests. My code executes only a few times a day so I want to appear to be a browser for that purpose.
我必须在.NET 3.5中执行此操作.但是，我也应该说我不需要专门使用WebClient.如果有一个使用HttpWebRequest/HttpWebResponse的更健壮/高效的解决方案，我很乐意使用它.
I am constrained to do this in .NET 3.5. However, I should also say that I do not specifically need to use WebClient. If there is a more robust/efficient solution using, say, HttpWebRequest/HttpWebResponse, I'm happy to use it.
工作代码 以实现此目标非常表示赞赏！谢谢！
Working code to achieve this would begreatly appreciated! Thanks!
推荐答案
这是我正在尝试的:
string news = "" ;
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.Accept] = "text/html, application/xhtml+xml, image/jxr, */*" ;
client.Headers[HttpRequestHeader.AcceptLanguage] = "en-AU,en-US;q=0.8,en-GB;q=0.5,en;q=0.3" ;
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0" ;
client.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate" ;
news = client.DownloadString("https://www.forexfactory.com/calendar.php?week=this") ;
}结果为System.Net.WebException:基础连接已关闭:发送中发生意外错误. ---> System.IO.IOException:从传输流接收到意外的EOF或0个字节.
And the result is System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Received an unexpected EOF or 0 bytes from the transport stream.
正如我所说，简单的DownloadString以前就返回了预期的结果.但是看来，为了减少网页抓取，该网站正在关闭非浏览器的请求.我的代码每天只执行几次，所以我想看起来像是 为此目的的浏览器.
As I said, the simple DownloadString alone previously returned the expected results. But it appears that in an effort to reduce web scraping, the site is closing off non-browser requests. My code executes only a few times a day so I want to appear to be a browser for that purpose.
我必须在.NET 3.5中执行此操作.但是，我也应该说我不需要专门使用WebClient.如果有一个使用HttpWebRequest/HttpWebResponse的更健壮/高效的解决方案，我很乐意使用它.
I am constrained to do this in .NET 3.5. However, I should also say that I do not specifically need to use WebClient. If there is a more robust/efficient solution using, say, HttpWebRequest/HttpWebResponse, I'm happy to use it.
工作代码 以实现此目标非常表示赞赏！谢谢！
Working code to achieve this would begreatly appreciated! Thanks!
我看不到您的代码有任何问题.您必须对服务器进行过多的错误调试，以至于它关闭了连接.否则，您的应用程序和服务器之间的任何地方都必须存在网络问题.
I don't see any issue with your code. Either you must be bugging the server too much that it closing your connection. OR there must be a network issue in anywhere between your application and the server.
一件事.如果从服务器获取gzip压缩响应，您将如何解压缩?您是否出于任何特定原因需要它?
One thing. how are you going to decompress if you get gzip compressed response from the server? do you need it for any specific reason?
client.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate" ;
这篇关于UserAgent设置?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！