Thursday, June 12, 2008

Python: A Look Back at Aquarium's Features

I've been in the Python Web world for a long time. When I started, the two dominant competitors were WebWare and Zope. My hat is off to Jim Fulton and Ian Bicking for being around even longer than me! I use Pylons these days, mainly because I know what I'm doing, and I want a framework that doesn't get in my way. However, I've always said that Aquarium had a few tricks up its sleeves that I hadn't seen elsewhere. I finally have names to describe some of them. Looking back at the various releases of Aquarium is a bit entertaining, at least to me. The first release was 0.5 back in 2000.

Release 1.4 in 2004 had a Web server abstraction layer that I called wsadaptor. Back then, the two main APIs I had to contend with were mod_python and CGI. These days, that functionality exists in WSGI, and I'm thankful for that.

Naturally, Aquarium has always had a fair share of interesting libraries. For instance, it had a session container system where you could plug in your own data store implementations in release 1.4 back in 2004. It was inspired by Apache::Session for Perl. These days, Beaker exists to provide this functionality.

Aquarium has always used template inheritance. I learned to separate layouts from templates (which Aquarium calls screens) from my buddy Leon Atkinson's PHP framework called Free Energy. In fact, much of Aquarium was inspired by Free Energy. Later, I copied Perl Mason's approach which allowed arbitrary layers of template inheritance.

In classical inheritance, a subclass gets to call super() when it wants the parent class to do something. In template inheritance, it's the parent template that gets to decide when the child template should do something. It's upside down, because the parent template's HTML has to wrap the child template's HTML. I've always thought this trick was useful even in normal Python. Hence, I wrote an article about it called The Inverse Extension Design Pattern in 2005.

At its heart, Aquarium is a plugin system. You write modules that plug into Aquarium, the framework. It's called Aquarium because it's a transparent structure. I always imagined the modules as little fish swimming around a tank.

Aquarium had a variable called packagePath that let you decide how to find the modules. For instance, if you imported "", the module "foo" could be defined in your app-specific code, your common-look-and-feel code, or Aquarium itself. In this way, you could use packagePath to tie together an arbitrary number of separate projects. Putting everything in the same package hierarchy involved a weird Python trick:
"""Setup the package search path."""
packageType = "database"

from __main__ import packagePath
from os.path import join
__path__ = map(lambda (x): join(x, packageType), packagePath) + __path__
That code is from release 1.4 in 2004. These days, setuptools has its own, nicer plugin system.

Back in 2001, I released Piranha as a subproject of Aquarium. Piranha is a code generator like Prototype in Ruby on Rails. It generated an admin interface. These days, Django's admin interface is even nicer. Here's a snippet of code generating code:
    def _buildHeader(self, nodeType): 
"""Build the header, including the small get methods."""
import time
timeStr = time.asctime(time.localtime(time.time()))
getAttribute = self._libHandler.getAttribute
nodeTypeName = getAttribute(nodeType, "name")
nodeTypeGuiName = getAttribute(nodeType, "guiName")
className = self._classNames.getAddScreenClassName(
return self._libHandler.evalstr('''\
"""This is a screen module for adding @nodeTypeName@ nodes."""

# Created: @timeStr@
# Author: Autogenerated by the AddScreen handler for Piranha
# <>.
# This is an autogenerated file that may contain user defined addons.

from NodeScreen import NodeScreen

class @className@(NodeScreen):
Dynamic imports have always been an important part of Aquarium. From the earliest days, I would dynamically import the screen to show based on parsing the URL. This was in contrast to object publishing systems like Zope, CherryPy, and Quixote. In order to instantiate, for instance, a screen module, I had to pass a ctx object which contained the request information, etc. Hence, any piece of code could call 'self._ctx.iLib.aquariumFactory("widget.mywidget")'. Aquarium would import "aquarium.widget.mywidget", instantiate "aquarium.widget.mywidget.MyWidget", and pass the ctx object.

Remember, that the mywidget module could be in your code or Aquarium's code, it didn't matter. In fact, you could write your own version of any module that came with Aquarium, and your module would take precedence. Hence, you always had an out if you didn't like the way Aquarium did something.

I do believe the above description of aquariumFactory and packagePath qualifies as an overly simplified version of dependency injection. Here's the code from version 0.5 back in 2000. This was back before I knew how to use __import__ instead of exec. Note that back then, I called the "ctx" object, "runData". That was inspired by my buddy Jon Stevens' framework, Apache Turbine.
    def aquariumFactory(self, moduleType, moduleName, constructorArguments):
"""Dynamically import and instantiate an Aquarium class.

moduleType: Some examples are "screen" and "action".
moduleName: This is the name of the module = name of the file = name
of that module's one class.
constructorArguments: This is a tuple of arguments to be passed to
the class's constructor.

This is basically a thin wrapper around "exec" that is useful for
importing and instantiating a given Aquarium class. For instance,
to import and instantiate a screen module whose name is in myScreen,
we'd do:

screenInstance = runData.iLib.aquariumFactory(
"screen", myScreen, (runData,))

exec("from aquarium." + moduleType + "." + moduleName + " import " +
moduleName + "\n")
return apply(eval(moduleName), constructorArguments)
The exec and eval look pretty heinous through modern eyes, but I always made sure to validate user input using regexes.

The strangest thing was that I didn't realize the connection between aquariumFactory and dependency injection until I had a dream last night. I dreamt that I was at some company meeting, and a Java old-timer was explaining dependency injection to me. A light finally went off in my head. Of course, dependency injection provides more. For instance, it creates the object graph automatically via a configuration file, which is more than merely acting as a factory. Nonetheless, I found the connection to aquariumFactory interesting.

Anyway, thanks for listening while I reminisced. Happy Hacking!


Noah Gift said...

I would be curious to know what type of marketing you did for your framework. I am still very convinced that this the single most important factor in a "web frameworks" success. Other open source projects it doesn't matter as much, but with web frameworks it is very critical.

I am also interested about how and when a marketing backlash occurs. If too much is promised, or the buzz gathers too quickly, how long until the projects feels the hurt.

Cool history lesson though, I didn't know Ian was programming web stuff way back then.

Bob Van Zant said...

Hey JJ, I still love Aquarium :-)

Shannon -jj Behrens said...

Haha, me too ;)

Shannon -jj Behrens said...

> I would be curious to know what type of marketing you did for your framework.

Pretty much none. I released it to PyPI which at least announces it on Python-announce. It was known on Web-SIG, but I wouldn't say I spent any time actually marketing it.

Noah Gift said...

hmm, I wonder if they should teach marketing in every CS program. It seems like some of the smartest people I know don't market their work. Look at Ian Bicking he cranks out a ridiculous amount of useful code, but he doesn't have a T-Shirt or "philosophy" yet, just a pile of completely useable tools :) Hmm, what would Ian's marketing slogan be....

It might be cool to get business and marketing degree interns to work with developers to practice their "hype machine" process. That would be one hell of a Google Summer of Code project. There could be YouTube marketing videos, banner ads, software foundations, the works.