Welcome, guest | Sign In | My Account | Store | Cart

Notice! PyPM is being replaced with the ActiveState Platform, which enhances PyPM’s build and deploy capabilities. Create your free Platform account to download ActivePython or customize Python with the packages you require and get automatic updates.

Download
ActivePython
INSTALL>
pypm install robotexclusionrulesparser

How to install robotexclusionrulesparser

  1. Download and install ActivePython
  2. Open Command Prompt
  3. Type pypm install robotexclusionrulesparser
 Python 2.7Python 3.2Python 3.3
Windows (32-bit)
1.4.0
1.6.1Never BuiltWhy not?
1.4.0 Available View build log
1.3 Available View build log
1.4.0
1.6.1Never BuiltWhy not?
1.4.0 Available View build log
1.3 Available View build log
Windows (64-bit)
1.4.0
1.6.1Never BuiltWhy not?
1.4.0 Available View build log
1.3 Available View build log
1.4.0
1.6.1Never BuiltWhy not?
1.4.0 Available View build log
1.3 Available View build log
Mac OS X (10.5+)
1.4.0
1.6.1Never BuiltWhy not?
1.4.0 Available View build log
1.3 Available View build log
1.4.0
1.6.1Never BuiltWhy not?
1.4.0 Available View build log
1.3 Available View build log
Linux (32-bit)
1.4.0
1.6.1Never BuiltWhy not?
1.4.0 Available View build log
1.3 Available View build log
1.4.0
1.6.1Never BuiltWhy not?
1.4.0 Available View build log
1.3 Available View build log
Linux (64-bit)
1.6.1 Available View build log
1.4.0 Available View build log
1.3 Available View build log
1.4.0
1.6.1Never BuiltWhy not?
1.4.0 Available View build log
1.3 Available View build log
1.6.1 Available View build log
 
License
http://creativecommons.org/licenses/BSD/
Depended by
Lastest release
version 1.6.1 on Sep 20th, 2013

Robotexclusionrulesparser is an alternative to the Python standard library module robotparser. It fetches and parses robots.txt files and can answer questions as to whether or not a given user agent is permitted to visit a certain URL.

This module has some features that the standard library module robotparser does not, including the ability to decode non-ASCII robots.txt files, respect for Expires headers and understanding of Crawl-delay and Sitemap directives and wildcard syntax in path names.

Complete documentation (including a comparison with the standard library module robotparser) is available in ReadMe.html.

Robotexclusionrulesparser is released under a BSD license.

Subscribe to package updates

Last updated Sep 20th, 2013

Download Stats

Last month:1

What does the lock icon mean?

Builds marked with a lock icon are only available via PyPM to users with a current ActivePython Business Edition subscription.

Need custom builds or support?

ActivePython Enterprise Edition guarantees priority access to technical support, indemnification, expert consulting and quality-assured language builds.

Plan on re-distributing ActivePython?

Get re-distribution rights and eliminate legal risks with ActivePython OEM Edition.