pyspider

xiaoxiao2021-02-28  4

pyspider setup and use setupuse pyqery responsedoc


pyspider setup and use

setup

run command:

pip install pyspider

download and add PhantomJS.exe to system environment path

use

run command:

pyspider

visit http://localhost:5000

add code in project:

self.crawl('__START_URL__', fetch_type=’js’, callback=self.index_page)

add javascript code:

self.crawl('__START_URL__', fetch_type='js', js_script= """ function() { setTimeout("$('.more').click()", 1000); }""", callback=self.phantomjs_parser)

you can try at http://demo.pyspider.org/

pyqery

response.doc()

A PyQuery object of the response’s content. Links have made as absolute by default.

Refer to the documentation of PyQuery: https://pythonhosted.org/pyquery/

CSS 选择器参考手册

http://www.w3school.com.cn/cssref/css_selectors.asp

转载请注明原文地址: https://www.6miu.com/read-1750231.html

最新回复(0)