问题描述
我曾经写过一个简单的抓取工具",用JAVA为我下载http页面.现在,我正在尝试使用LWP模块将相同的内容重写为Perl.
I once wrote a simple 'crawler' to download http pages for me in JAVA.Now I'm trying to rewrite to same thing to Perl, using LWP module.
这是我的Java代码(效果很好):
This is my Java code (which works fine):
String referer = "http://example.com";
String url = "http://example.com/something/cgi-bin/something.cgi";
String params= "a=0&b=1";
HttpState initialState = new HttpState();HttpClient httpclient = new HttpClient();httpclient.setState(initialState);httpclient.getParams().setCookiePolicy(CookiePolicy.NETSCAPE);
HttpState initialState = new HttpState();HttpClient httpclient = new HttpClient();httpclient.setState(initialState);httpclient.getParams().setCookiePolicy(CookiePolicy.NETSCAPE);
PostMethod postMethod = new PostMethod(url);postMethod.addRequestHeader("Referer", referer);postMethod.addRequestHeader("User-Agent", " Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13");postMethod.addRequestHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8");postMethod.addRequestHeader("Content-Type", "application/x-www-form-urlencoded");
PostMethod postMethod = new PostMethod(url);postMethod.addRequestHeader("Referer", referer);postMethod.addRequestHeader("User-Agent", " Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13");postMethod.addRequestHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8");postMethod.addRequestHeader("Content-Type", "application/x-www-form-urlencoded");
String length = String.valueOf(params.length());postMethod.addRequestHeader("Content-Length", length);postMethod.setRequestBody(params);
String length = String.valueOf(params.length());postMethod.addRequestHeader("Content-Length", length);postMethod.setRequestBody(params);
httpclient.executeMethod(postMethod);
httpclient.executeMethod(postMethod);
这是Perl版本:
my $referer = "http://example.com/something/cgi-bin/something.cgi?module=A";
my $url = "http://example.com/something/cgi-bin/something.cgi";
my @headers = (
'User-Agent' => 'Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13',
'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Referer' => $referer,
'Content-Type' => 'application/x-www-form-urlencoded',
);
my @params = (
'a' => '0',
'b' => '1',
);
my $browser = LWP::UserAgent->new( );
$browser->cookie_jar({});
$response = $browser->post($url, @params, @headers);
print $response->content;
发布请求正确执行,但是我得到了另一个(主)网页.好像cookie不能正常工作...
The post request executes correctly, but I get another (main) webpage. As if cookies were not working properly...
有人猜错了吗?为什么我从JAVA和perl程序中得到不同的结果?
Any guesses what is wrong?Why I'm getting different result from JAVA and perl programs?
推荐答案
您要创建哈希,而不是数组-例如,代替:
You want to be creating hashes, not arrays - e.g. instead of:
my @params = ( 'a' => '0', 'b' => '1',);
my @params = ( 'a' => '0', 'b' => '1',);
您应该使用:
my %params = ( a => 0, b => 1,);
my %params = ( a => 0, b => 1,);
将参数传递给LWP :: UserAgent post方法时,您需要传递对哈希的引用,例如:
When passing the params to the LWP::UserAgent post method, you need to pass a reference to the hash, e.g.:
$response = $browser->post($url, \%params, %headers);
$response = $browser->post($url, \%params, %headers);
您还可以使用以下方法查看要发送到服务器的请求:
You could also look at the request you're sending to the server with:
print $response->request->as_string;
print $response->request->as_string;
您还可以使用处理程序自动转储请求和响应以进行调试:
You can also use a handler to automatically dump requests and responses for debugging purposes:
$ua->add_handler("request_send", sub { shift->dump; return });$ua->add_handler("response_done", sub { shift->dump; return });
$ua->add_handler("request_send", sub { shift->dump; return });$ua->add_handler("response_done", sub { shift->dump; return });
这篇关于Perl LWP中的Cookie的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!