Thursday, December 1, 2011

Moving yolk to github

Today I moved yolk to github. I've committed a few long overdue patches and hope to make a quick 0.4.2 release soon.


In 2006 I added a setuptools ebuild to Gentoo Linux. This amazing new tool allowed us to install Python packages and their dependencies right from the Python Package Index. And make a mess of your system if you use easy_install instead of your distribution's package manager to install Python pacakges. But I had a plan.

As a Gentoo developer and Python package maintainer, I'm responsible for watching bugzilla for Python package bugs. I wanted to add this amazing new tool to Gentoo as an official ebuild, but I had a feeling it may be used by people who didn't want to write an ebuild or wait for a 'version bump' of a package.

Trying to determine which Python packages were installed via a Linux distribution's package manager and which were installed with easy_install became an early goal of yolk. I learned about entry points from reading nose's source and made yolk plugin-based. The first plugin showed which packages were installed via Gentoo's portage system and which were installed via setuptool's easy_install.

After adding basic functions like showing you which Python packages were installed locally, I added XML-RPC support to query the Python Package Index at This let me check if newer versions of packages were available. I was also able to start another project, g-pypi, a tool for creating ebuilds from any Python package in the Python Package Index. This worked great and g-pypi later became a Google Summer of Code project.

I learned a lot from writing yolk. It was one of my first Test Driven Design projects in Python and I really enjoyed learning to code with that technique. I have no doubt the relatively small number of bugs discovered after the earlier releases was due to using TDD.


I've moved yolk to github, grab a fork and dig in.


I'm going to be thinking in terms of how to make yolk more useful with virtualenv. For instance, virtualenv's 'lssitepackages' is like 'yolk -l', but only for that specific virtualenv. I may try to make yolk more virtualenv-aware so you can see which packages in a specific virtualenv have upgrades available, for instance. There's a few things I'd have done differently. All of the lower case commands in yolk query your local Python packages on your system. All of the upper case commands query The Python Package Index. This should have been two separate libraries and a command-line tool. The name 'yolk' doesn't make a lot of sense these days with all the non-egg things yolk does and the decline of the egg package format.