Dependencies
- distribute
- m01.mongo
- m01.remote
- p01.recipe.setup[paste]
- p01.recipe.setup
- s01.core
- transaction
- z3c.jsonrpc
- ZConfig
- zope.app.appsetup
- zope.app.publication
- zope.authentication
- zope.browserpage
- zope.component
- zope.event
- zope.interface
- zope.location
- zope.principalregistry
- zope.publisher
- zope.security
- zope.securitypolicy
- zope.site
- [test] m01.mongo[test]
- [test] s01.client
- [test] zope.testing
This package provides a daemon which is able to run buildout based scrapy spider packages. (see s01.demo). The daemon processes pending crawl jobs stored in mongodb. The daemon also provides a WSGI server providing a JSON-RPC 2.0 API. The JSON-RPC proxy from the s01.client package can be used for install packages and add new crawl jobs to the mongodb which is the queue.
Changes
0.5.0 (2012-11-18)
- initial implementation