run command:
pip install pyspiderdownload and add PhantomJS.exe to system environment path
run command:
pyspidervisit http://localhost:5000
add code in project:
self.crawl('__START_URL__', fetch_type=’js’, callback=self.index_page)add javascript code:
self.crawl('__START_URL__', fetch_type='js', js_script= """ function() { setTimeout("$('.more').click()", 1000); }""", callback=self.phantomjs_parser)you can try at http://demo.pyspider.org/
A PyQuery object of the response’s content. Links have made as absolute by default.
Refer to the documentation of PyQuery: https://pythonhosted.org/pyquery/
CSS 选择器参考手册
http://www.w3school.com.cn/cssref/css_selectors.asp