Oakwinter.com :: Code

As promised, and prompted in part by a recent post by Brett Cannon, here’re my thoughts on why unittest sucks.

Reading the docs for unittest, you’d think it would be easily extensible. You see things like TestCase.defaultTestResult(), the many overridable methods on TestResult objects, the apparent flexibility of TextTestRunner, and you get it in your head that it should be pretty easy to make it do whatever you want.

Armed with this impression and your Python-foo, you set off to write a unittest extension that will let you mark certain tests as “TODO”. You want the test harness to count these tests differently than normal tests: TODO tests are supposed to fail, and you want to be notified they start unexpectedly passing.

You tinker a bit, you poke and you prod, and you wind up with your extension, and the whole thing works great. You just wish you hadn’t had to subclass _TestTextResult, TestCase and TextTestRunner to get the job done. You feel like it could have been easier, but you don’t pay it much mind. After all, you only needed the one extension.

A few months later, a different project has a need to run reference-count checks around each test case for a C extension module. Confident from your first experience extending unittest, you head back into the code. A little later, you emerge, bearing the shiny new reference count-checking extension to unittest. You again ended up subclassing _TestTextResult, TestCase and TextTestRunner, but again, it’s just one extension.

An hour later, your boss walks by and says that the ref-counting extension and the TODO extension need to be combined so they can be used together on a new project. No problem, you say; composing the two should be cake.

That thought lasts about as long as it takes to load the extensions in your editor of choice.

unittest might have been intended to be extended, but only in simple ways, and only by one extension at a time. I’ll save you the suspense; to combine the above extensions, you have to write a completely different third extension, which attempts to merge the two functionality sets as much as possible. You want to incorporate another extension, say one that logs the test results to a database? Tough luck.

unittest’s design is fundamentally broken. Little or no attempt was made to separate the different concerns at work here: TestCase instances can determine what result logger to use and how exceptions are to be interpreted. Making TextTestRunner use a subclass of _TestTextResult means subclassing the runner object. TestResult is responsible for converting tracebacks to a textual representation, even though this means that any result classes that want to do introspect the tracebacks end up completely rewriting much of TestResult in the process.

That’s the problem; next time, the solution.

As promised, a discussion of how to use SVK’s pull command:

For the longest time, I would type some variation of

$ svk sync //$project_name/main
$ svk sm //$project_name/main //$project_name/local
$ cd ~/src/$project_name
$ svk update

in order to pull changes from a remote repository to the mirror, merge from the mirror to my local branch, then update my working copy from the local branch.

That much typing sucks, and it was just as I was getting ready to write a macro for this task (in the vein of mymerge and mergeproject) that I discovered svk push. This lead me to investigate pull, which is — as you might have guessed — the opposite of push.

At its core, pull can be thought of as a wrapper around that command sequence above: sync the mirror, merge to the local branch, then update the working copy. The simplest form of pull is this, issued from within a working copy:

svk pull

Like push, you can provide your own working copy path:

svk pull ~/src/project_x/

This has the same effect as if you had cd‘d to the directory and then run the naked pull command.

Also like push, you can provide a depot path instead of a working copy path:

svk pull //project_x/local

This will sync the mirror and merge the new changes, but will not update any working copy.

I’ve been using push and pull for a few weeks now, and I’m thrilled by not having to type out a thousand commands over and over again or write a new macro.

After posting two entries about some custom SVK commands, it seems SVK had already beat me to the punch.

If you recall, I originally wrote the mymerge and mergeproject commands because I was tired of typing this out every time I needed to merge local changes up to the main repository:

svk smerge -I //$project_name/local //$project_name/main

As it turns out, SVK already has a command to do this — and in a more flexible way, to boot: push, an smerge macro like mine.

push takes a single argument, the thing to push changes from. This thing can be either a working copy or a depot path (if you don’t supply this argument, the current directory is used). In either case, SVK will automatically figure out the target for the merge:

• If you supply a working path, SVK will first figure out which depot path you used when checking out the working copy, then…

• If you supply a depot path (or if you’ve fallen through from above), SVK will figure out where you copied the depot path from, then merge to that parent path. (Note that it’s meaningless to use push on a mirrored path; svk push should only be used on depot paths copied from somewhere else.)

Once SVK has figured out the source and target paths, it will perform an incremental smerge. Hmm…this sounds suspiciously like my commands!

So, to translate my own commands into svk push:

svk mymerge $project_name

is the same as

svk push //$project_name/local

while

svk mergeproject

is the same as

svk push

Unfortunately, push’s entry in the SVK book can be summed up as “TODO”, the word that dominates the page. I’ve added this to my todo list, but it’s a fairly low priority at the moment.

In a future entry, I’ll talk about svk pull, the handy-dandy, already-written spelling of another custom command I was close to writing.

Continuing my “Extending SVK for fun and profit” series, I present the mergeproject macro, which builds upon the mymerge command I talked about last time.

As I mentioned in the mymerge article, I name my local and mirrored repositories //$project_name/local/ and //$project_name//main/, respectively. In addition, I follow the convention of giving my checkout paths equally imaginative names, like /home/collin/src/$project_name/.

When last we left our heros, I had managed to cut the command to sync my local repository to the mirrored repository down from a monstrous

svk sm -I //$project_name/local //$project_name/main

to a more lazy-coder-friendly

svk mm $project_name

That’s good, but we can go further.

Since all of my project checkouts follow the same naming conventions, and since most of my svk mm commands are issued from within the project’s checkout directory, there’s no reason for me to type $project_name each time. Some File::Spec incantations should be more than enough to figure this out for me.

After some digging around through SVK’s internals, I present you…mergeproject:

package SVK::Command::Mergeproject;
use strict;
use SVK::Version;  our $VERSION = $SVK::VERSION;

use base qw( SVK::Command::Mymerge );

use SVK::Util qw(splitdir catdir);
use SVK::I18N qw(loc);
use Cwd;

sub parse_arg {
    my $self = shift;
    my @arg = @_;
    return if @arg != 0;

    my $pwd = Cwd::cwd();
    my @dirs = splitdir($pwd);
    for(my $i = 0; $i < @dirs; $i++) {
        my $dir = catdir(@dirs[0..$i]);

        # See if the directory is a valid checkout path
        # If it's not, an error will be raised and $@ will be set.
        # If the directory is a valid checkout path, pass only the
        #  directory name -- ie, not the full path -- up to
        #  Mymerge, which will handle the rest.
        eval { $self->{xd}->find_repos_from_co($dir, 0) };
        unless ($@) {
            return $self->SUPER::parse_arg($dirs[$i]);
        }
    }

    die loc(”Unable to find a checkout path while traversing %1n”,
                $pwd);
}

We use Cwd::cwd() to grab the absolute path to the current directory, then use SVK::Util::splitdir() (SVK::Util autoloads all the useful bits of File::Spec for us) to break the path into individual directory names. We then iterate over the list of directory names, building up longer and longer paths with SVK::Util::catpath(). For example, given the current working directory of /home/collin/src/svnmock/trunk/, we’d look in the following succession of directories for SVK checkouts:

/
/home/
/home/collin/
/home/collin/src/
/home/collin/src/svnmock/
/home/collin/src/svnmock/trunk/

stopping once SVK::XD::find_repos_from_co() reports that we have indeed found one. (In the above example, we’d end up stopping at /home/collin/src/svnmock/, the first directory that SVK can map to a repository.) The second argument of 0 to find_repos_from_co() tells SVK that we’re only interested in whether the checkout maps to a repository.

Once we’ve found a valid checkout path, the last directory in the series (the one that actually holds the checkout) is assumed to be the project name and so is passed up to mymerge.

Let’s recap: we went from this:

svk sm -I //$project_name/local //$project_name/main

to this

svk mm $project_name

to now this (using the mp alias for mergeproject)

svk mp

Hooray, laziness!

If you’re interested in doing something similar, put this code in /usr/lib/perl5/site_perl/*/SVK/Commands/Mergeproject.pm or wherever your SVK command modules happen to be. If you want to use a shortcut (I use mp), you’ll need to add a line to the %alias hash in SVK::Command, something like “mp mergeproject”.

In developing the common test harness for my functional, svnmock and the standard Python module unittest, mainly with respect to how hard it is to combine different extensions to unittest’s TestCase, TestRunner and TestResult classes.

So, I head off to start poking around in unittest’s internals, to see what mucking around I can do in order to be able to compose extensions the way I want to. (Exactly what I’m trying to achieve will be the subject of another post.) I find — to my great horror — that unittest’s test suite consists of the following code:

import unittest

def test_TestSuite_iter():
    '''
    >>> test1 = unittest.FunctionTestCase(lambda: None)
    >>> test2 = unittest.FunctionTestCase(lambda: None)
    >>> suite = unittest.TestSuite((test1, test2))
    >>> tests = []
    >>> for test in suite:
    ...     tests.append(test)
    >>> tests == [test1, test2]
    True
    '''

How’s that for irony.

So: I’ve now spent three or four days going over unittest’s documentation and code, building up a test suite as I go. Work is proceeding approximately like so:

1. Cleaning up the documentation. The old docs were full of typos and grammatical problems, not to mention the blatant factual errors and omissions.

   The bulk of this work is already done: a patch for the docs was accepted and applied to Python’s SVN repository as r51123.

2. Write the test suite. As it stands, I’m up to 121 tests, giving me 60% test coverage, according to figleaf (which proved gratifyingly easy to integrate into the test suite). The bulk of those tests are for unittest.TestLoader, particularly its loadTestsFromName() and loadTestsFromNames() methods.

3. Fix the bugs. Thus far, I’ve uncovered 23 bugs in unittest; some of these are clear-cut and easy to fix, while others will require discussion on python-dev. In addition, I’ve got 14 test cases for functionality that unittest should have had from the beginning, but doesn’t; these will have to wait until Python 2.6.

All my projects use Subversion for revision control, but on my laptop, I use SVK so I can keep working and committing even when away from an Internet connection. (Also: SVK’s merge support and branch tracking beats SVN’s hands down).

(This isn’t an SVK tutorial. For that, you should check out Ron Bieber’s excellent series of SVK tutorials.)

One part of SVK’s everyday workflow is having a mirrored repository, which represents the remote SVN repository, and a local repository, where you normally commit to. You then merge between these repositories — from local to mirrored to push changes to the main repository, mirrored to local to sync with the main repository.

For each project I work on, I name the mirrored and local repositories //$project_name/main and //$project_name/local, respectively. This means every time I want to push changes up to the main SVN repositories, I type svk sm -I //$project_name/local //$project_name/main — automatically merge all changes between branches, incremental commits.

Because that command never changes, and because I’m lazy, I wrote a mymerge “macro” to save myself the trouble of all that typing. Now, instead of that big, long command, I type svk mm $project_name. Much better.

mymerge works by subclassing SVK’s SVK::Command::Smerge class and overriding the parse_args() method. It does some monkeying around with the arguments, then hands control off to smerge.

package SVK::Command::Mymerge;
use strict;
use SVK::Version;  our $VERSION = $SVK::VERSION;

use base qw( SVK::Command::Smerge );

sub options { () }

sub parse_arg {
    my $self = shift;
    my @arg = @_;
    return if $#arg < 0;

    my $depot = $arg[0];

    $self->{incremental} = 1;
    return $self->SUPER::parse_arg(”//$depot/local”,
                                   “//$depot/main”);
}

The $self->{incremental} = 1; assignment is the same as supplying the -I flag to smerge in the original example. The last line does the important argument-mucking before passing control up to smerge.

If you’re interested in doing something similar, put this code in /usr/lib/perl5/site_perl/*/SVK/Commands/Mymerge.pm or wherever your SVK command modules happen to be. If you want to use a shortcut (I use mm), you’ll need to add a line to the %alias hash in SVK::Command, something like “mm mymerge”.