Welcome, guest | Sign In | My Account | Store | Cart

Notice! PyPM is being replaced with the ActiveState Platform, which enhances PyPM’s build and deploy capabilities. Create your free Platform account to download ActivePython or customize Python with the packages you require and get automatic updates.

Download
ActivePython
INSTALL>
pypm install buildout.eggtractor

How to install buildout.eggtractor

  1. Download and install ActivePython
  2. Open Command Prompt
  3. Type pypm install buildout.eggtractor
 Python 2.7Python 3.2Python 3.3
Windows (32-bit)
0.6 Available View build log
Windows (64-bit)
0.6 Available View build log
Mac OS X (10.5+)
0.6 Available View build log
Linux (32-bit)
0.6 Available View build log
Linux (64-bit)
0.6 Available View build log
 
License
GPL
Dependencies
Lastest release
version 0.6 on Jan 5th, 2011

buildout.eggtractor

Q: What is a buildout extension ?

A: http://pypi.python.org/pypi/zc.buildout#extensions

The problem

When developing zope/plone eggs with buildout I have to edit the buildout configuration file ( in 3 places ) each time I create/delete/rename a development egg in the src directory or in other development directories (sometime I have more than one).

I have to add/delete/rename the egg in the eggs option of the [buildout] and then add/delete/rename the egg path in the develop option of the [buildout] and in the end add/delete/rename the zcml option of the zope [instance] or in the configure.zcml file of my policy package. This is too much specially when the speed is set to development mode. I need a less boring way to develop.

Solution

buildout.eggtractor is a buildout extension that scans the src directory or a list of directories I give for eggs and picks them up automatically. So no more editing of the buildout's configuration file.

When buildout.eggtractor finds an egg in the scanned directory it:

System Message: WARNING/2 (<string>, line 32)

Literal block expected; none found.

1. adds the egg to the eggs option of all zope instance parts or to a set of given parts

  1. adds the egg's path in the develop option of the [buildout]

3. If tractor-autoload-zcml is not given or set to other thing than false, scans the egg folder for configure.zcml, meta.zcml and overrides.zcml and adds the appropriate zcml entries to the zcml option of the zope instance parts or to a set of given parts.

This steps are done on the fly when running buildout. So I can add/delete/rename an egg and it will be picked up.

NOTE: The extension does not write to the buildout's configuration file.

buildout.eggtractor options

tractor-src-directory: A set of directories to scan for development eggs. Defaults to the src directory of the buildout.

tractor-target-parts: A set of parts to update their eggs option with eggs found in the tractor-src-directory. Defaults to zope instance parts if any.

tractor-autoload-zcml(boolean): Update the zcml option of tractor-target-parts with the eggs found in tractor-src-directory. Defaults to true

tractor-zcml-top: A set of eggs to load their zcml files first. Defaults to an empty set.

How to use it

Using buildout.eggtractor is very simple. As said, it is a buildout extension. All I have to do is to declare it in the extensions option:

[buildout]

System Message: ERROR/3 (<string>, line 73)

Inconsistent literal block quoting.

parts =

extensions = buildout.eggtractor

That's all. buildout.eggtractor will scan the src directory and do its job every time I run the buildout command.

When I have other directories I want to scan I just add an tractor-src-directory option in the [buildout] and add my directories there:

[buildout]

System Message: ERROR/3 (<string>, line 85)

Inconsistent literal block quoting.

parts =

extensions = buildout.eggtractor

tractor-src-directory = dev-src1 dev-src2 src

In a few cases when the priority of loading zcml files matters. I add the egg to be loaded first in the tractor-zcml-top option in the [buildout]:

[buildout]

System Message: ERROR/3 (<string>, line 98)

Inconsistent literal block quoting.

parts =

extensions = buildout.eggtractor

tractor-src-directory = dev-src1 dev-src2 src

tractor-zcml-top = plone.app.mypackage1

If I want to add the eggs found in the development directories to the eggs option of a given set of parts, I add a tractor-target-parts option and add the parts there:

[buildout]

System Message: ERROR/3 (<string>, line 116)

Inconsistent literal block quoting.

parts = instance1 instance2 instance3

extensions = buildout.eggtractor

tractor-target-parts = instance1 instance3

This way only instance1 and instance3 will be updated.

If I have already other way to include the zcml files (ie: z3c.autoinclude) and don't want eggtractor to generate the zcml slugs, I add an tractor-autoload-zcml option and set it to false

In most cases you will only need to add buildout.eggtractor to the extensions option of the [buildout] without any extra configuration options.

LIMITATION:

The extension assumes that the egg name reflects its file system structure

example: if the egg name is com.mustap.www the extension assumes that the file system structure is one of the following:

System Message: WARNING/2 (<string>, line 142)

Literal block expected; none found.
  1. com.mustap.www/src/com/mustap/www
  2. com.mustap.www/com/mustap/www

This is where the extension looks for configure.zcml, meta.zcml and overrides.zcml files.

If the egg name has nothing to do with how it is structured on the system, the extension will ignore it.

XXX: I guess walking through the directory is better than this assumption.

In my case this is not a limitation as I choose my egg names that way.

Mustapha

email: mustap_at_gmail_com

web: http://www.mustap.com

Change history

0.6 (2008-10-29)
  • fixed install problem on windows:

System Message: WARNING/2 (<string>, line 171)

Bullet list ends without a blank line; unexpected unindent.

http://plone.org/support/forums#nabble-f293351

  • split buildout configuration values before we test if an egg was

System Message: WARNING/2 (<string>, line 174)

Bullet list ends without a blank line; unexpected unindent.

added already. using 'in' with a string doen't work if an egg's name is a substring of an already added egg (e.g. plone.app.content after plone.app.contentmenu). [csenger]

  • sort output of os.listdir (it's order is undefined) to make

System Message: WARNING/2 (<string>, line 180)

Bullet list ends without a blank line; unexpected unindent.

debugging easier. [csenger]

0.5 (2008-04-30)
  • Refactoring

System Message: WARNING/2 (<string>, line 188)

Bullet list ends without a blank line; unexpected unindent.

Added tractor-target-parts option Added tractor-autoload-zcml option Updated documentation Updated tests: need more tests [mustapha]

0.4 (2008-04-27)
  • Made sure the configure.zcml is added to package includes if a meta.zcml or

System Message: WARNING/2 (<string>, line 199)

Bullet list ends without a blank line; unexpected unindent.

overrides.zcml in the same packages have already been found. [hannosch]

0.3 (2008-04-27)
  • Use a new line as a separator for added entries. A space makes buildout

System Message: WARNING/2 (<string>, line 207)

Bullet list ends without a blank line; unexpected unindent.

think it has to deal with a version specifier. [hannosch]

0.2 (2008-04-27)
  • Added support for automatically finding multiple instances.

System Message: WARNING/2 (<string>, line 215)

Bullet list ends without a blank line; unexpected unindent.

[hannosch]

  • Better use the recipe name for finding the instance, as this is less likely

System Message: WARNING/2 (<string>, line 218)

Bullet list ends without a blank line; unexpected unindent.

to conflict. [hannosch]

0.1 (2008-04-27)
  • Whitespace fixes.

System Message: WARNING/2 (<string>, line 226)

Bullet list ends without a blank line; unexpected unindent.

[hannosch]

  • Created recipe with ZopeSkel.

System Message: WARNING/2 (<string>, line 229)

Bullet list ends without a blank line; unexpected unindent.

[mustapha]

Detailed Documentation

Tests for buildout.eggtractor buildout extension

Let's create a buildout configuration file:

>>> data = """

System Message: ERROR/3 (<string>, line 240)

Inconsistent literal block quoting.

... [buildout] ... parts = zope2 instance1 instance2 ... extensions = buildout.eggtractor ... eggs = ... develop = ... [instance1] ... recipe = plone.recipe.zope2instance ... zope2-location = ${zope2:location} ... user = admin:admin ... [instance2] ... recipe = plone.recipe.zope2instance ... zope2-location = ${zope2:location} ... user = admin:admin ... [zope2] ... recipe = plone.recipe.zope2install ... url = http://www.zope.org/Products/Zope/2.9.8/Zope-2.9.8-final.tgz ... """ >>> rmdir(tempdir, 'buildout.test') >>> cd(tempdir) >>> sh('mkdir buildout.test') mkdir buildout.test <BLANKLINE> >>> cd('buildout.test') >>> touch('buildout.cfg', data=data) >>> ls('.') buildout.cfg

run the buildout first time so wget our zope instances:

>>> sh('buildout bootstrap')

System Message: ERROR/3 (<string>, line 270)

Inconsistent literal block quoting.

buildout bootstrap Creating directory '/tmp/buildout.test/bin'. Creating directory '/tmp/buildout.test/parts'. Creating directory '/tmp/buildout.test/develop-eggs'. Generated script '/tmp/buildout.test/bin/buildout'. <BLANKLINE> >>> sh('bin/buildout') bin/buildout ... Installing instance1. Generated script '/tmp/buildout.test/bin/instance1'. Generated script '/tmp/buildout.test/bin/repozo'. Installing instance2. Generated script '/tmp/buildout.test/bin/instance2'. <BLANKLINE> <BLANKLINE>

Now let's create an egg in the src directory:

>>> sh("paster create --no-interactive -o src -t plone_app com.mustap.www namespace_package=com namespace_package2=mustap package=www")

System Message: ERROR/3 (<string>, line 290)

Inconsistent literal block quoting.

paster create --no-interactive -o src -t plone_app com.mustap.www namespace_package=com namespace_package2=mustap package=www ... ...setup.py egg_info <BLANKLINE>

Ok, so now that we have an egg, lets run the buildout in offline mode. We should get a link file in develop-egg, a zcml slugs in parts/instance1/etc/package-includes and parts/instance2/etc/package-includes and a line with the path to our egg in the bin/instance1 and bin/instance2 files.

First we check that there is nothing of the previous mentioned things:

>>> ls('develop-eggs')
>>> ls('parts/instance1/etc/package-includes')

System Message: ERROR/3 (<string>, line 305)

Inconsistent literal block quoting.

No directory named parts/instance1/etc/package-includes

>>> ls('parts/instance2/etc/package-includes')
No directory named parts/instance2/etc/package-includes
>>> sh('grep com.mustap.www bin/instance1')
grep com.mustap.www bin/instance1
<BLANKLINE>
>>> sh('grep com.mustap.www bin/instance2')
grep com.mustap.www bin/instance2
<BLANKLINE>

OK, now run the buildout in offline mode:

>>> sh('./bin/buildout -o')

System Message: ERROR/3 (<string>, line 321)

Inconsistent literal block quoting.

./bin/buildout -o ...

Check that we have a correct created buildout. First check that we have a link in the develop-eggs directory:

>>> ls('develop-eggs')

System Message: ERROR/3 (<string>, line 328)

Inconsistent literal block quoting.

com.mustap.www.egg-link

Check that we have our zcml slugs in the package-includes:

>>> ls('parts', 'instance1', 'etc', 'package-includes')

System Message: ERROR/3 (<string>, line 333)

Inconsistent literal block quoting.

001-com.mustap.www-configure.zcml

>>> ls('parts', 'instance2', 'etc', 'package-includes')
001-com.mustap.www-configure.zcml

and in the end check that there is a line in bin/instance1 and bin/instance1 that includes our egg in the path:

>>> cat('bin', 'instance1')

System Message: ERROR/3 (<string>, line 342)

Inconsistent literal block quoting.

#!/usr/bin/python2.4 ... sys.path[0:0] = [ '/tmp/buildout.test/src/com.mustap.www', ... ] ...

>>> cat('bin', 'instance2')
#!/usr/bin/python2.4
...
sys.path[0:0] = [
'/tmp/buildout.test/src/com.mustap.www',
...
]
...

Let's now try the tractor-target-parts option. We create a new buildout.cfg file with an empty tractor-target-parts:

>>> data = data.replace('eggs =', 'tractor-target-parts = \neggs = ')
>>> touch('buildout.cfg', data=data)
>>> sh('./bin/buildout -o')

System Message: ERROR/3 (<string>, line 365)

Inconsistent literal block quoting.

./bin/buildout -o ...

We get the egg link in the develop-egg directory:

>>> ls('develop-eggs')

System Message: ERROR/3 (<string>, line 371)

Inconsistent literal block quoting.

com.mustap.www.egg-link

But no zcml slug in the instance 1 and 2:

>>> ls('parts', 'instance1', 'etc', 'package-includes')

System Message: ERROR/3 (<string>, line 376)

Inconsistent literal block quoting.

No directory named parts/instance1/etc/package-includes

>>> ls('parts', 'instance2', 'etc', 'package-includes')
No directory named parts/instance2/etc/package-includes

Nor a line in bin/instance1 and bin/instance2 with our egg path:

>>> code = cat('bin', 'instance1', returndata=True)
>>> code.find('com.mustap.www') == -1

System Message: ERROR/3 (<string>, line 385)

Inconsistent literal block quoting.

True

>>> code = cat('bin', 'instance2', returndata=True)
>>> code.find('com.mustap.www') == -1
True

But if the tractor-target-parts option is not empty:

>>> data = data.replace('tractor-target-parts =', 'tractor-target-parts = instance1')
>>> touch('buildout.cfg', data=data)
>>> sh('./bin/buildout -o')

System Message: ERROR/3 (<string>, line 396)

Inconsistent literal block quoting.

./bin/buildout -o ...

and we get a zcml slug only in the specified target:

>>> ls('parts', 'instance1', 'etc', 'package-includes')

System Message: ERROR/3 (<string>, line 402)

Inconsistent literal block quoting.

001-com.mustap.www-configure.zcml

>>> ls('parts', 'instance2', 'etc', 'package-includes')
No directory named parts/instance2/etc/package-includes

and only the specified target's control script is updated:

>>> code = cat('bin', 'instance1', returndata=True)
>>> code.find('com.mustap.www') == -1

System Message: ERROR/3 (<string>, line 411)

Inconsistent literal block quoting.

False

>>> code = cat('bin', 'instance2', returndata=True)
>>> code.find('com.mustap.www') == -1
True

Contributors

  • mustapha, Author
  • hannosch, Minor fixes

Subscribe to package updates

Last updated Jan 5th, 2011

Download Stats

Last month:2

What does the lock icon mean?

Builds marked with a lock icon are only available via PyPM to users with a current ActivePython Business Edition subscription.

Need custom builds or support?

ActivePython Enterprise Edition guarantees priority access to technical support, indemnification, expert consulting and quality-assured language builds.

Plan on re-distributing ActivePython?

Get re-distribution rights and eliminate legal risks with ActivePython OEM Edition.