Oakwinter.com :: Code

Not travelling around western Europe has done wonders for Cypress’s TODO list.

Most of my work lately has been going into designing (and redesigning) and implementing the Controller-side watch_module system. Put simply, this subsystem revolves around solving the first problem I outlined in the project introduction: how to ensure things like cache purity in the face of auto-reloaded modules.

This problem took a good while to solve, mostly because of ideological issues. To get into that, we first need a little background:

Cypress was originally conceived as a framework that would host other, higher-level applications. The goal was that these applications could run on top of Cypress without any modification whatsoever, totally ignorant of the fact that Cypress was running below them. Cypress would act as a mediator, sitting between the hosted application and the Python interpreter, intercepting module imports.

This import-interception was how Cypress was going to solve the three issues I mentioned in the project introduction. Cypress would redirect import requests to a specified revision control system (issue #2), periodically querying to find out which modules had changed since the last query and reloading the applicable chunks of code (issue #1); by having each application be hosted by its own instance of some Controller class, any number of applications could run on top of the same Cypress libraries (issue #3).

The part that was getting in the way was the goal of letting applications be ignorant of Cypress. I ran into this problem: say module M caches instances of a class C from module N. What happens when N is modified, changing the definition of class C? Module M’s cache now contains instances of both the old and new version of class C, leading to cache poisoning.

The solution that I’m going forward with tarnishes some of that ideological purity in exchange for more application-side power. If an application needs to know when a given module changes, it can call the watch_module method to tell Cypress, “call this object when anything happens to module foo.bar“. Given a module name and a callable, applications can now be signalled when the named module is edited, renamed or even deleted, giving the hosted code a chance to clear object caches, etc.

A more comprehensive design document is available from Cypress’s SVN repository.

Despite not getting nearly the blog-post luvin’ that some of my other projects get, Cypress is one of the longest-running and most interesting pieces of code I work on. Time to spread the love around.

Cypress: not just another Python web framework.

In fact, despite being written to host a content management system for Community2.com, Cypress isn’t a web framework at all. Cypress is a new kind of framework, one targeted at three specific problems we encountered while developing C2:

• Having to restart Apache sucks. Whenever we change one of the static modules that power C2, we have to restart Apache. Now, I’ve heard that mod_perl is supposed to detect when your modules change and reload them automatically, but I’ve never been able to get it working, nor has anyone I know, and even if we could get it working, we wouldn’t use it.

  Why not just let mod_perl reload the modules? Because mod_perl reloads only the modules that have changed, not those that, say, cache instances of classes defined in the changed modules. This leads to cache poisoning, where your object cache has instances of both the new and old versions of the class. Cypress fixes this.

• Lack of real revision control sucks. Storing code in a database (the way the Everything Engine does) means you lose all the power that comes with a real revision control system. While Everything tries oh-so-desperately to make up for this with a half-assed versioning system, it just can’t compete with the likes of darcs or SVK.

  Cypress moves code out of the database and back under strong revision control, back where it belongs. Cypress is able to pull modules directly from revision control, and thanks to Cypress’ highly flexible architecture, you can have it work with any RCS you can imagine.

• One of the big problems we encountered with C2 is that there were other sites on the same server that used the Everything Engine. Because they all used the same set of modules, this meant that we couldn’t go changing things willy-nilly to suit C2. Cypress, again, fixes this.

I’m hoping to have Cypress ready sometime this decade, which seems to be a pretty slip-proof deadline.

While there’s no official website for Cypress, interested viewers can use the WebSVN viewer to watch the project’s progress.