用AJAX从python脚本的网页浏览器没有仿真或无头braws

用AJAX从python脚本的网页浏览器没有仿真或无头braws

本文介绍了如何调用AJAX从python脚本的网页浏览器没有仿真或无头brawser?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我是新来的Ajax和JavaScript。
我爬了一个网站中,我能够获取的相关细节与片的XPath 的帮助使用Python下载网页后(的urllib2 / 申请 /的)。

I am new to ajax and javascript.
I am crawling a website in which I am able to fetch relevant piece of details with the help of XPath after downloading webpage using Python(urllib2/request/mechanize).

在网页这些都是一些资料,只有点击一个链接后是可见的。而该链接调用 XHR 使用Firefox的Web开发者工具来获取,我发现了细节。 (++ or Tools >> Web Developer >> Network) I am showing that link and its javascript attributes that I can see using Firefox web developer tool (++ or Tools >> Web Developer >> Inspector) in attached image under thick black rectanguls.

我也能看到Ajax请求的URL,标题,响应和参数虽然同Firefox网页开发工具。相同的图像会显示在 http://i.stack.imgur.com/9jhfr.png

I am also able to see ajax request url, headers, response and parameters though same firefox web developer tools. same image is visible at http://i.stack.imgur.com/9jhfr.png

我在想,我所有的有效载荷帖子请求。我怎样才能让HTTP POST调用这个使用Python和请求/ urllib2的等方面的帮助?所以在回应我可以看到我点击该链接后显示在网页的详细信息。像
requests.get(URL,数据=< paramter_to_post,我可以在Firefox中看到>中标题=<请求头,我可以在Firefox&GT见;)

I am thinking that I have all the payload for POST request. How can I make http post call this using Python with the help of request/urllib2 etc? so in response I can see details which i show in webpage after clicking that link. Like
requests.get(url, data=<paramter_to_post which i can see in firefox>, headers=<request headers that I can see in firefox>)

我短
如何使用python模拟Ajax调用?或如何获取infromation我点击该链接后看到了什么?

I short
How to simulate ajax call using python? or how to fetch infromation which I see after clicking that link?

我可以使用自动执行此任务 /的或其他无头浏览器。但我想用HTTP发布解决这一点,并获取其正是发生在Firefox中,当我点击链接。

I can automate this task using Selenium/PhantomJS or other headless browser. but I want to solve this using HTTP Post and Get which is exactly happening in Firefox when I click the link.

推荐答案

那么首先安装萤火虫的( https://开头getfirebug。 COM /

Well first of all install firebug (https://getfirebug.com/)

然后转到您的网页,启动萤火虫,去萤火虫面板Net标签。现在,在这个选项卡中可以看到所有的get /职位要求你的Firefox发送到该网站。

Then go to your page, launch the firebug and go to the Net tab in firebug panel.Now in this tab you can see all of the get/posts calls your firefox sends to the website.

现在,你可以按一下周围,刷新页面,看看正在拨打的电话。你的情况按一下按钮,你会看到正在作出新的呼叫,你可能会发现它的HTML标签。

Now you can click around, refresh the page and see what calls are being made. In your case click the button and you'll see new calls are being made, you will probably find it at html tab.

在那里,你可以发现,一旦你点击它,你会看到请求和其他细节的电话。

There you can find a call once you click on it you'll see the request and other details.

进行参数的字典,将其安装到数据=在您的文章。您也可以在您的文章做它的字典并将其连接到标题=做标题。

make a dictionary of the parameters and attach it to "data=" in your post. you can also make headers by making a dictionary of it and attaching it to the "headers=" in your post.

请注意:很多网站使用Cookies来识别,如果呼叫由一个合法的浏览器发出的,因此可能需要相当多的与饼干和URL摆弄。 !

Take note: a lot of websites use cookies to identify if the calls are made by a legit browser so it might require quite a bit of fiddling with cookies and urls. !

很难举出例子,如果你不给我们的网站。

it's hard to give examples if you don't give us the website.

这篇关于如何调用AJAX从python脚本的网页浏览器没有仿真或无头brawser?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

08-01 04:19