问题描述
相关问题:
作为一个个人项目,我试图刮擦我大学的课程目录以提供它作为RESTful API。
As a personal project, I'm trying to scrape my University's course catalog to provide it as a RESTful API.
URL1: https://duapp3.drexel.edu/webtms_du/
URL2: https:// duapp3.drexel.edu/webtms_du/Colleges.asp?Term=201125&univ=DREX
转到URL1设置了需要的Cookie URL2。我正在成功接收和发送Cookie,如下所示:
Going to URL1 sets a cookie which is needed for URL2. I'm successfully receiving and sending the cookie like this:
curl -c recd url1
curl -v -b recd url2
但是,我得到的响应与我在浏览器中看到的响应不同。基本上,我的浏览器显示我的专业列表, curl
获取一个页面,而不是主菜单,它有
However, the response I get differs from the one I see in my browser. Basically, my browser shows me a list of majors, while curl
gets a page where instead of the list of majors, it has
< FONT COLOR = red size = 5> < / FONT>< FONT COLOR = red size = 5>请稍后再回来。< / FONT>
我想也许他们正在检查用户代理,
I figured maybe they were checking user-agent, so I tried this as well:
curl -b recdcookies --user-agentMozilla / 4.0(compatible; MSIE 5.01; Windows NT 5.0)https:// duapp3 .drexel.edu / webtms_du / Colleges.asp?Term = 201125& univ = DREX
但同样意外的反应。
浏览器和curl之间的不同输出是什么?只要 curl -v
显示它只设置一个cookie。是因为他们是 https
页面,我需要添加另一个参数到 curl
?
What is causing the different outputs between browser and curl? As far as curl -v
shows it only sets one cookie. Is it because they are https
pages, and I need to be adding another argument to the curl
?
推荐答案
在命令行中使用curl时,如果url包含&符号(&
),因为它在unix系统上有特殊的意义。
When using curl on the command line, you must enclose the url in quotes if it contains an ampersand (&
) because it has special meaning on unix systems.
这篇关于使用curl接收和发送cookie,但没有得到预期的响应的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!