問題:出現(xiàn)SyntaxError: invalid syntax
報錯代碼:
shiyanlou:louspider/ $ scrapy crawl mylouspider -o courses.json [11:00:46]
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 11, in <module>
sys.exit(execute())
File "/usr/local/lib/python3.5/dist-packages/scrapy/cmdline.py", line 148, in execute
cmd.crawler_process = CrawlerProcess(settings)
File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 243, in __init__
super(CrawlerProcess, self).__init__(settings)
File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 134, in __init__
self.spider_loader = _get_spider_loader(settings)
File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 330, in _get_spider_loader
return loader_cls.from_settings(settings.frozencopy())
File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 61, in from_settings
return cls(settings)
File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 25, in __init__
self._load_all_spiders()
File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 47, in _load_all_spiders
for module in walk_modules(name):
File "/usr/local/lib/python3.5/dist-packages/scrapy/utils/misc.py", line 71, in walk_modules
submod = import_module(fullpath)
File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 669, in exec_module
File "<frozen importlib._bootstrap_external>", line 775, in get_code
File "<frozen importlib._bootstrap_external>", line 735, in source_to_code
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
File "/home/shiyanlou/louspider/louspider/spiders/mylouspider.py", line 16
item['image'] = course.xpath('.//div[@class="course-img"]/img/@src').extract()[0].strip()
^
SyntaxError: invalid syntax
shiyanlou:louspider/ $ ls [11:01:00]
items.py louspider scrapy.cfg
shiyanlou:louspider/ $ cd louspider [11:03:38]
shiyanlou:louspider/ $ ls [11:08:16]
__init__.py middlewares.py __pycache__ spiders
items.py pipelines.py settings.py
shiyanlou:louspider/ $ vi items.py [11:08:17]
shiyanlou:louspider/ $ scrapy crawl mylouspider -o courses.json [11:08:40]
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 11, in <module>
sys.exit(execute())
File "/usr/local/lib/python3.5/dist-packages/scrapy/cmdline.py", line 148, in execute
cmd.crawler_process = CrawlerProcess(settings)
File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 243, in __init__
super(CrawlerProcess, self).__init__(settings)
File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 134, in __init__
self.spider_loader = _get_spider_loader(settings)
File "/usr/local/lib/python3.5/dist-packages/scrapy/crawler.py", line 330, in _get_spider_loader
return loader_cls.from_settings(settings.frozencopy())
File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 61, in from_settings
return cls(settings)
File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 25, in __init__
self._load_all_spiders()
File "/usr/local/lib/python3.5/dist-packages/scrapy/spiderloader.py", line 47, in _load_all_spiders
for module in walk_modules(name):
File "/usr/local/lib/python3.5/dist-packages/scrapy/utils/misc.py", line 71, in walk_modules
submod = import_module(fullpath)
File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 669, in exec_module
File "<frozen importlib._bootstrap_external>", line 775, in get_code
File "<frozen importlib._bootstrap_external>", line 735, in source_to_code
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
File "/home/shiyanlou/louspider/louspider/spiders/mylouspider.py", line 16
item['image'] = course.xpath('.//div[@class="course-img"]/img/@src').extract()[0].strip()
^
SyntaxError: invalid syntax
mylouspider.py
1 import scrapy
2 from louspider.items import LouSpiderItem
3 from scrapy.selector import Selector
4
5 class LouSpider(scrapy.Spider):
6 name = "mylouspider"
7 allowed_domains = ["shiyanlou.com"]
8 start_urls = ['https://www.shiyanlou.com/courses/?category=all&course_ty
pe=all&tag=all&fee=free']
9 def parse(self, response):
10 hxs = Selector(response)
11 courses = hxs.xpath('//div[@class="col-md-3 col-sm-6 course"]')
12 for course in courses:
13 item = CourseItem()
14 item['name'] = course.xpath('.//div[@class="course-name"]/text( )').extract()[0].strip()
15 item['learned'] = course.xpath('.//span[@class="course-per-num p ull-left"]/text()').extract([1].strip()
16 item['image'] = course.xpath('.//div[@class="course-img"]/img/@s rc').extract()[0].strip()
17 yield item
items.py
import scrapy
class LouSpiderItem(scrapy.Item):
# define the fields for your item here like:
name = scrapy.Field()
image = scrapy.Field()
learned = scrapy.Field()
利用ipython通過scrap shell測試xpath可以抓取
但是整體運行就報錯
新手求大神解答 感謝!?。?!
北大青鳥APTECH成立于1999年。依托北京大學優(yōu)質雄厚的教育資源和背景,秉承“教育改變生活”的發(fā)展理念,致力于培養(yǎng)中國IT技能型緊缺人才,是大數(shù)據(jù)專業(yè)的國家
北大青鳥中博軟件學院創(chuàng)立于2003年,作為華東區(qū)著名互聯(lián)網(wǎng)學院和江蘇省首批服務外包人才培訓基地,中博成功培育了近30000名軟件工程師走向高薪崗位,合作企業(yè)超4
中公教育集團創(chuàng)建于1999年,經(jīng)過二十年潛心發(fā)展,已由一家北大畢業(yè)生自主創(chuàng)業(yè)的信息技術與教育服務機構,發(fā)展為教育服務業(yè)的綜合性企業(yè)集團,成為集合面授教學培訓、網(wǎng)
達內教育集團成立于2002年,是一家由留學海歸創(chuàng)辦的高端職業(yè)教育培訓機構,是中國一站式人才培養(yǎng)平臺、一站式人才輸送平臺。2014年4月3日在美國成功上市,融資1
曾工作于聯(lián)想擔任系統(tǒng)開發(fā)工程師,曾在博彥科技股份有限公司擔任項目經(jīng)理從事移動互聯(lián)網(wǎng)管理及研發(fā)工作,曾創(chuàng)辦藍懿科技有限責任公司從事總經(jīng)理職務負責iOS教學及管理工作。
浪潮集團項目經(jīng)理。精通Java與.NET 技術, 熟練的跨平臺面向對象開發(fā)經(jīng)驗,技術功底深厚。 授課風格 授課風格清新自然、條理清晰、主次分明、重點難點突出、引人入勝。
精通HTML5和CSS3;Javascript及主流js庫,具有快速界面開發(fā)的能力,對瀏覽器兼容性、前端性能優(yōu)化等有深入理解。精通網(wǎng)頁制作和網(wǎng)頁游戲開發(fā)。
具有10 年的Java 企業(yè)應用開發(fā)經(jīng)驗。曾經(jīng)歷任德國Software AG 技術顧問,美國Dachieve 系統(tǒng)架構師,美國AngelEngineers Inc. 系統(tǒng)架構師。