We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
不知道为什么我的爬不出东西来,json文件是0kb的。。其中spider里面我改了一点:from scrapy.spiders import Spider(因为报错说要用spiders)。还有log改logging了,然后运行的结果看不大懂,望大佬指正
D:\LZZZZB\w3school>scrapy crawl w3school 2017-06-21 22:33:03 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: w3school ) 2017-06-21 22:33:03 [scrapy.utils.log] INFO: Overridden settings: {‘BOT_NAME’: ‘ w3school’, ‘NEWSPIDER_MODULE’: ‘w3school.spiders’, ‘ROBOTSTXT_OBEY’: True, ‘SPID ER_MODULES’: [‘w3school.spiders’]} 2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled extensions: [‘scrapy.extensions.corestats.CoreStats’, ‘scrapy.extensions.telnet.TelnetConsole’, ‘scrapy.extensions.logstats.LogStats’] 2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled downloader middlewares: [‘scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware’, ‘scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware’, ‘scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware’, ‘scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware’, ‘scrapy.downloadermiddlewares.useragent.UserAgentMiddleware’, ‘scrapy.downloadermiddlewares.retry.RetryMiddleware’, ‘scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware’, ‘scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware’, ‘scrapy.downloadermiddlewares.redirect.RedirectMiddleware’, ‘scrapy.downloadermiddlewares.cookies.CookiesMiddleware’, ‘scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware’, ‘scrapy.downloadermiddlewares.stats.DownloaderStats’] 2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled spider middlewares: [‘scrapy.spidermiddlewares.httperror.HttpErrorMiddleware’, ‘scrapy.spidermiddlewares.offsite.OffsiteMiddleware’, ‘scrapy.spidermiddlewares.referer.RefererMiddleware’, ‘scrapy.spidermiddlewares.urllength.UrlLengthMiddleware’, ‘scrapy.spidermiddlewares.depth.DepthMiddleware’] 2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled item pipelines: [‘w3school.pipelines.W3SchoolPipeline’] 2017-06-21 22:33:03 [scrapy.core.engine] INFO: Spider opened 2017-06-21 22:33:03 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pag es/min), scraped 0 items (at 0 items/min) 2017-06-21 22:33:03 [scrapy.extensions.telnet] DEBUG: Telnet console listening o n 127.0.0.1:6023 2017-06-21 22:33:03 [scrapy.core.engine] INFO: Closing spider (finished) 2017-06-21 22:33:03 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {‘finish_reason’: ‘finished’, ‘finish_time’: datetime.datetime(2017, 6, 21, 14, 33, 3, 262577), ‘log_count/DEBUG’: 1, ‘log_count/INFO’: 7, ‘start_time’: datetime.datetime(2017, 6, 21, 14, 33, 3, 252576)} 2017-06-21 22:33:03 [scrapy.core.engine] INFO: Spider closed (finished)
The text was updated successfully, but these errors were encountered:
No branches or pull requests
不知道为什么我的爬不出东西来,json文件是0kb的。。其中spider里面我改了一点:from scrapy.spiders import Spider(因为报错说要用spiders)。还有log改logging了,然后运行的结果看不大懂,望大佬指正
D:\LZZZZB\w3school>scrapy crawl w3school
2017-06-21 22:33:03 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: w3school
)
2017-06-21 22:33:03 [scrapy.utils.log] INFO: Overridden settings: {‘BOT_NAME’: ‘
w3school’, ‘NEWSPIDER_MODULE’: ‘w3school.spiders’, ‘ROBOTSTXT_OBEY’: True, ‘SPID
ER_MODULES’: [‘w3school.spiders’]}
2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled extensions:
[‘scrapy.extensions.corestats.CoreStats’,
‘scrapy.extensions.telnet.TelnetConsole’,
‘scrapy.extensions.logstats.LogStats’]
2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled downloader middlewares:
[‘scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware’,
‘scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware’,
‘scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware’,
‘scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware’,
‘scrapy.downloadermiddlewares.useragent.UserAgentMiddleware’,
‘scrapy.downloadermiddlewares.retry.RetryMiddleware’,
‘scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware’,
‘scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware’,
‘scrapy.downloadermiddlewares.redirect.RedirectMiddleware’,
‘scrapy.downloadermiddlewares.cookies.CookiesMiddleware’,
‘scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware’,
‘scrapy.downloadermiddlewares.stats.DownloaderStats’]
2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled spider middlewares:
[‘scrapy.spidermiddlewares.httperror.HttpErrorMiddleware’,
‘scrapy.spidermiddlewares.offsite.OffsiteMiddleware’,
‘scrapy.spidermiddlewares.referer.RefererMiddleware’,
‘scrapy.spidermiddlewares.urllength.UrlLengthMiddleware’,
‘scrapy.spidermiddlewares.depth.DepthMiddleware’]
2017-06-21 22:33:03 [scrapy.middleware] INFO: Enabled item pipelines:
[‘w3school.pipelines.W3SchoolPipeline’]
2017-06-21 22:33:03 [scrapy.core.engine] INFO: Spider opened
2017-06-21 22:33:03 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pag
es/min), scraped 0 items (at 0 items/min)
2017-06-21 22:33:03 [scrapy.extensions.telnet] DEBUG: Telnet console listening o
n 127.0.0.1:6023
2017-06-21 22:33:03 [scrapy.core.engine] INFO: Closing spider (finished)
2017-06-21 22:33:03 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{‘finish_reason’: ‘finished’,
‘finish_time’: datetime.datetime(2017, 6, 21, 14, 33, 3, 262577),
‘log_count/DEBUG’: 1,
‘log_count/INFO’: 7,
‘start_time’: datetime.datetime(2017, 6, 21, 14, 33, 3, 252576)}
2017-06-21 22:33:03 [scrapy.core.engine] INFO: Spider closed (finished)
The text was updated successfully, but these errors were encountered: