Saturday, June 28, 2008

Erlang: Reia

I've been saying for a long time that I'd like to write a Pythonic version of Erlang. Someone beat me to it, which should come as no surprise since I'm always so busy. Hmm, looks like I have a new language to learn ;)

Friday, June 27, 2008

Hosting: RimuHosting

I had to submit a trouble ticket to our new hosting provider, RimuHosting. Two minutes after I submitted the ticket, it was accepted by someone. Check out the stats in the picture. It's 10:30PM. Gees, that makes me happy ;)

Update: The problem was fixed 20 minutes later :)

Tuesday, June 24, 2008

OpenSolaris: Package Management and Joyent

Linux has really, really good package management. You'll realize this the second you try to use another operating system. Ubuntu (thanks to Debian) makes life so easy. It's easy to install all the software I want, and it's easy to make sure it stays updated. Even the kernel, etc. stay updated with practically no effort on my part.

In contrast, managing packages on OpenSolaris is painful. There are multiple package systems, and none of them seems to do everything I want. (In Linux, there are multiple distros, each with their own fully functioning package system.) Reading Joyent's wiki, it seems to be commonplace to fall back to patching and compiling the source by hand. Painful.

Theoretically, Nexenta might fix this situation one of these days, but in the meantime, I have a deadline ;)

Since POSIX is a commodity, the OSs work hard to differentiate themselves. OpenSolaris sets itself apart by being very scalable. Furthermore, DTrace and zfs are supposed to be crazy awesome. Perhaps that's true, but man, I wish I could just get Python 2.5 installed! What's worse is that once I do have it installed, I have no idea how to keep the system updated. Ubuntu sets itself apart by being easy to install and administrate by novice sysadmins like myself, and boy do I feel the difference!

By the way, as I mentioned, I'm using Joyent. I'm not feeling the love today. So much of the documentation on their wiki is stale and misleading. I spent all day trying to figure out how to use Blastwave because that's what the wiki said they used. I finally gave up and submitted a trouble ticket. Support replied (in a timely manner) that Joyent was no longer using Blastwave, but had switched to pkgsrc. Unfortunately, I can't find the pkgsrc directory on the system either. After reading the docs for several hours, I've realized two things: My chances of getting Python 2.5 working and talking to MySQL without immense pain are slim to none, and don't trust Joyent's wiki.

Wednesday, June 18, 2008

Modern Java From a Python Perspective

I had a friend visit me today. He's a Java programmer, but he also knows enough Python, Ruby, JavaScript and Scheme to be dangerous. I asked him to show me around his code so I could see some modern Java. I learned Java back in the 1.0 days, but I stopped paying attention back around 1.4. Here are some of the things I saw.

Generics are as I expected.

Annotations are interesting. There was code like:
public void method() {
"SomeClass" acts like a "tag" for "method". Elsewhere in the code, you can define wrapper methods (i.e. decorators) for all the methods that have been wrapped by "@SomeClass". Clearly, this was the inspiration for Python's function decorators. I had simply never seen them in Java before.

Dependency injection is interesting, as I expected it to be. In general, you accept parameters for everything you need, and you don't worry a lot about how things get instantiated. You avoid using the "new" keyword, and instead let the dependency injection framework instantiate things for you. This takes care of circular dependencies. There was a Java file that configured what gets bound to what. Hence, if I ask for SomeFoo, it'll give me an instance of the right SomeFooImpl. Basically, it's a whole layer to add more dynamism.

IntelliJ was impressive. I asked him to write a module from scratch so I could see his workflow. He right clicked and hit "add new class". IntelliJ created a file and asked if he wanted to add it to Subversion. Then he simply started typing in the implementation for the class. Whenever he typed something like FileWriter, it automatically added the right import line. Whenever he used a class attribute that didn't exist yet, it showed up in red until he told the IDE to create the attribute. Then, he had to click on the attribute to add a getter and setter.

I knew that Java IDEs held your hand a lot. I didn't know that the work flow was to simply start using something and then have the IDE generate the boilerplate for you. That was interesting. I liked how nicely he was able to navigate the code base. Best of all, IntelliJ has really nice Vim keybindings, which he was using.

He showed me Struts 2 and his usage of JSP. Those weren't all that remarkable or new to me.

I returned the favor by showing him some of the code from my current project. He said, "Wow, your Python code is really beautiful," which I took as quite a compliment. He also said that there was clearly a lot less boilerplate and that more functionality was achieved in fewer lines.

Clearly, a lot of useful code is written in Java. glassfish, Lucene, HiveDB, and IntelliJ are just a few things that come to mind. I'm not going to give up Python anytime soon, but it was nice to get a peek into another world.

Tuesday, June 17, 2008

Web: Best Anti-Web Rant

My buddy Mike left a comment on one of my previous posts, UNIX vs. the Web. I'm going to quote the whole thing:
It's hopeless.

It's too easy to throw something together, which "sortta" works, but has no conceptual integrity.

I think another problem is that Open Sores software is, by definition, AT BEST, a Beta, more typically an Alpha. So, it should not be surprising if the quality is, ahh, not as high as it might be.

Like most stuff that gets buzz, Django "sortta" works. Works well enough that it has a user base.

I keep waiting for the Grand Unification of Pylons and Gears - maybe soon enough...

In the meantime, as I suffer with JavaScript, CSS, DHTML, the DOM, XML, and all that _shit_ - well, I'm ready to have heated discussions with every Netscape employee who had anything to do with this disaster.

I'm tempted to blame the Stanfords, CMUs, and MITs for this current mess, because they didn't step up and produce leadership - (CLEARLY, none of their students or graduates were involved in this web disaster!).

So we are forced to deal with the poop of The Inexperienced - and that is NEVER fun!

If people had their wits about them when web standards were being solidified, they would have chosen Lisp as the browser's language, and they would have chosen S-expressions instead of XML/JSON (and, by default, a UNIFIED DOM - what a concept!)

It's enough to make me heave... Something as simple as looking up something in a database, re-formatting into JSON, and getting the client to actually receive it so it can display it requires an absolutely humongous amount of code/servers/etc - just for that!

The Web standards feel like they've been invented by high school students on crack - certainly, no Adult Supervision was involved.

The other thing Web Standards do is help foster the Web Programmer's Full Employment Program - because so much of what a Web developer has to do is deal with trivial mind-numbing bullshit.

The other thing is that the most trivial of applications seem to need these _huge_ Systems Approaches - it's like, nobody ever heard about how to build small, efficient, quickly-written systems.

Apparently, while I was building h/w in the 90s and early part of the 2000s, the s/w world went to hell, and I wasn't paying enough attention to save it. I _deeply_ apologize.

That's the most entertaining thing I've read all week ;)

BTW, Mike has a PDP 11 at his house, which qualifies him as my hero ;)

Thursday, June 12, 2008

Python: A Look Back at Aquarium's Features

I've been in the Python Web world for a long time. When I started, the two dominant competitors were WebWare and Zope. My hat is off to Jim Fulton and Ian Bicking for being around even longer than me! I use Pylons these days, mainly because I know what I'm doing, and I want a framework that doesn't get in my way. However, I've always said that Aquarium had a few tricks up its sleeves that I hadn't seen elsewhere. I finally have names to describe some of them. Looking back at the various releases of Aquarium is a bit entertaining, at least to me. The first release was 0.5 back in 2000.

Release 1.4 in 2004 had a Web server abstraction layer that I called wsadaptor. Back then, the two main APIs I had to contend with were mod_python and CGI. These days, that functionality exists in WSGI, and I'm thankful for that.

Naturally, Aquarium has always had a fair share of interesting libraries. For instance, it had a session container system where you could plug in your own data store implementations in release 1.4 back in 2004. It was inspired by Apache::Session for Perl. These days, Beaker exists to provide this functionality.

Aquarium has always used template inheritance. I learned to separate layouts from templates (which Aquarium calls screens) from my buddy Leon Atkinson's PHP framework called Free Energy. In fact, much of Aquarium was inspired by Free Energy. Later, I copied Perl Mason's approach which allowed arbitrary layers of template inheritance.

In classical inheritance, a subclass gets to call super() when it wants the parent class to do something. In template inheritance, it's the parent template that gets to decide when the child template should do something. It's upside down, because the parent template's HTML has to wrap the child template's HTML. I've always thought this trick was useful even in normal Python. Hence, I wrote an article about it called The Inverse Extension Design Pattern in 2005.

At its heart, Aquarium is a plugin system. You write modules that plug into Aquarium, the framework. It's called Aquarium because it's a transparent structure. I always imagined the modules as little fish swimming around a tank.

Aquarium had a variable called packagePath that let you decide how to find the modules. For instance, if you imported "", the module "foo" could be defined in your app-specific code, your common-look-and-feel code, or Aquarium itself. In this way, you could use packagePath to tie together an arbitrary number of separate projects. Putting everything in the same package hierarchy involved a weird Python trick:
"""Setup the package search path."""
packageType = "database"

from __main__ import packagePath
from os.path import join
__path__ = map(lambda (x): join(x, packageType), packagePath) + __path__
That code is from release 1.4 in 2004. These days, setuptools has its own, nicer plugin system.

Back in 2001, I released Piranha as a subproject of Aquarium. Piranha is a code generator like Prototype in Ruby on Rails. It generated an admin interface. These days, Django's admin interface is even nicer. Here's a snippet of code generating code:
    def _buildHeader(self, nodeType): 
"""Build the header, including the small get methods."""
import time
timeStr = time.asctime(time.localtime(time.time()))
getAttribute = self._libHandler.getAttribute
nodeTypeName = getAttribute(nodeType, "name")
nodeTypeGuiName = getAttribute(nodeType, "guiName")
className = self._classNames.getAddScreenClassName(
return self._libHandler.evalstr('''\
"""This is a screen module for adding @nodeTypeName@ nodes."""

# Created: @timeStr@
# Author: Autogenerated by the AddScreen handler for Piranha
# <>.
# This is an autogenerated file that may contain user defined addons.

from NodeScreen import NodeScreen

class @className@(NodeScreen):
Dynamic imports have always been an important part of Aquarium. From the earliest days, I would dynamically import the screen to show based on parsing the URL. This was in contrast to object publishing systems like Zope, CherryPy, and Quixote. In order to instantiate, for instance, a screen module, I had to pass a ctx object which contained the request information, etc. Hence, any piece of code could call 'self._ctx.iLib.aquariumFactory("widget.mywidget")'. Aquarium would import "aquarium.widget.mywidget", instantiate "aquarium.widget.mywidget.MyWidget", and pass the ctx object.

Remember, that the mywidget module could be in your code or Aquarium's code, it didn't matter. In fact, you could write your own version of any module that came with Aquarium, and your module would take precedence. Hence, you always had an out if you didn't like the way Aquarium did something.

I do believe the above description of aquariumFactory and packagePath qualifies as an overly simplified version of dependency injection. Here's the code from version 0.5 back in 2000. This was back before I knew how to use __import__ instead of exec. Note that back then, I called the "ctx" object, "runData". That was inspired by my buddy Jon Stevens' framework, Apache Turbine.
    def aquariumFactory(self, moduleType, moduleName, constructorArguments):
"""Dynamically import and instantiate an Aquarium class.

moduleType: Some examples are "screen" and "action".
moduleName: This is the name of the module = name of the file = name
of that module's one class.
constructorArguments: This is a tuple of arguments to be passed to
the class's constructor.

This is basically a thin wrapper around "exec" that is useful for
importing and instantiating a given Aquarium class. For instance,
to import and instantiate a screen module whose name is in myScreen,
we'd do:

screenInstance = runData.iLib.aquariumFactory(
"screen", myScreen, (runData,))

exec("from aquarium." + moduleType + "." + moduleName + " import " +
moduleName + "\n")
return apply(eval(moduleName), constructorArguments)
The exec and eval look pretty heinous through modern eyes, but I always made sure to validate user input using regexes.

The strangest thing was that I didn't realize the connection between aquariumFactory and dependency injection until I had a dream last night. I dreamt that I was at some company meeting, and a Java old-timer was explaining dependency injection to me. A light finally went off in my head. Of course, dependency injection provides more. For instance, it creates the object graph automatically via a configuration file, which is more than merely acting as a factory. Nonetheless, I found the connection to aquariumFactory interesting.

Anyway, thanks for listening while I reminisced. Happy Hacking!

Monday, June 09, 2008

Palm: Sprint Dial Up Networking on the Treo 700P

From here:
Treo 700p users, take note! With the Sprint Treo 650, all you needed for Bluetooth DUN was the Vision service added to your normal voice plan. With the Treo 700p, things have changed. Sprint has added special software to the 700p that detects when you're using the Treo to go online with your laptop (as opposed to just surfing on the phone's tiny screen). Sprint calls this kind of access “Phone As Modem,” or PAM, and for some reason they assume that only business users with deep pockets will want it. Therefore, they require you to purchase a special Phone As Modem plan, currently priced at $40 per month in addition to whatever other plan you may already have. That's right: Even if you're paying an extra $25/month for the Power Vision Ultimate Pack, you must still pay another $40/month just to use Bluetooth DUN (no matter how light your usage is). Note that this is the very same feature that comes free (with Vision) on the Treo 650.

Luckily, there is a workaround...
Wow, that sucks!

Fortunately, I get free Internet access at Starbucks thanks to my SBC Yahoo DSL account.

Wednesday, June 04, 2008

Python: any() + generator expression

Here's a nice use of any() with a generator expression:
    def is_url_interesting(self, url):
return any(regex.match(url) for regex in self.interesting_url_regexes)
It's fast because any() shortcircuits the same way "or" does. It's way shorter than the comparable "for" loop. Last of all, I think it's very readable, at least if you're used to reading list comprehensions.