Tuesday, December 21, 2010

Apple: Ouch!

I had my laptop wrapped in a blanket in my backpack when I dropped it on concrete at BART yesterday. It landed on the corner with quite a lot of force and bent a few metal pieces. Everything seems to work, with the exception of the ethernet port. Thankfully, I rarely use that. I'm hoping this is covered by the warranty, although it seems unlikely. The bent aluminum reminds me of body damage on a car.

Wednesday, December 15, 2010

Vim: PeepCode Videos

There are commercial videos on PeepCode that show how to use Vim. They're fantastic! The production quality is very high, so they're worth paying for. I bought both of them, and I learned quite a lot. Even the first one covered a ton of advanced stuff, so I highly recommend it.

If you're just getting started with Vim, I suggest you:
  • Read this blog post from Yehuda Katz. Basically, he suggests you use the GUI to get started and iteratively gain more experience with Vim. I think that's good advice.
  • Next, I suggest you fire up Vim and take the internal tutorial.
  • Last of all, watch Smash into Vim part I.
If you're an advanced Vim user, watch both videos and tell me what your favorite part was ;)

Friday, December 10, 2010

JavaScript: IE and DTDs

I was having a hard time getting Internet Explorer 8 (IE8) working with Socket.IO's "htmlfile" transport (or any other transport for that matter). I was getting weird permissions issues. I finally fixed the problem by adding a DTD to the top of my HTML file, "<!doctype html>". I was just being lazy not having a DTD, but I was really surprised when that fixed my problem. Of course, I'm still having a hard time getting IE7, IE6, and Opera working cross-domain (technically, I'm just using a different port, but eventually it'll truly be cross-domain).

Wednesday, December 08, 2010

Ruby: Why I Prefer Cucumber

I've been struggling to articulate why, in most cases, I prefer high-level integration tests with Cucumber and Webrat (or Capybara) over low-level model, view, and controller tests with RSpec. I think I finally have an example that conveys what I'm trying to say. Consider the following Cucumber test:
Scenario: create a new film unsuccessfully
Given I am logged in as "admin"
And I am on the admin films page

When I follow "New film"
And I press "Create"
Then I should see "There were problems with the following fields:"
And I should see "Name can't be blank"
And I should see "Url name can't be blank"
And I should see "Sort name can't be blank"
And I should see "URL doesn't look like a valid RTMPE URL"
But I should not see "Trailer URL doesn't look like a valid RTMPE URL"
And I should not see "Scene URL doesn't look like a valid RTMPE URL"
This test is short, readable, and easy to write. It doesn't test every possible validation failure, and it's not the only test I have. (In fact, I have some RSpec model tests that test the more esoteric URL validation rules.) However, it does test the model, view, and controller's handling of validation failures, and it even tests that they integrate with each other.

Can you imagine trying to write the same tests by separately testing the model, view, and controller using RSpec? Now, imagine trying to use a separate test for each assertion. That's a lot of code for something so trivial--this ain't rocket science, guys! Finally, remember that when you test the things separately, there's nothing preventing the code from crashing when you put all the pieces together. (For instance, what happens if the controller and view each pass their RSpec tests, but they disagree on the spelling of one of the assigns?)

Is there benefit to testing things separately--absolutely. Is it worth it in this case--absolutely not. I think it's important to remember that at a certain level, our job is to implement features that work. Tests are a means to an end--they help us keep the code working. They don't really have any intrinsic value for the stakeholder. They only have the secondary value of keeping the code working when it is extended.

Just as there is engineering value in implementing features using less code (as long as it remains readable), there is also engineering value in implementing features using less testing code (as long as the tests continue to serve their purpose of preventing regressions).

My point is that Cucumber lets you test more using less effort.

Friday, November 26, 2010

Personal: Wife in Labor

Wife in labor. Built Twilio app to time contractions. Call it to try it out: (866) 948-3615.

Friday, November 19, 2010

Software Engineering: Coping When You Must Repeat Yourself

These days, most software engineers are familiar with the acronym "DRY" which stands for "don't repeat yourself". In general, it's good advice. In a perfect world, no code would ever be duplicated and every piece of "truth" would only exist in one place. However, the real world isn't quite so perfect, and things are far less DRY than you might imagine. The question is, how do you cope?

First let me show you some reasons why you can't always keep it DRY:

Often, the same truth is duplicated in the code, the tests, and the docs. Duplicating the same piece of truth in the code and in the tests helps each verify the other. Generally, the code is more general than the tests (the tests verify examples of running the more general code), but the duplication is there. When you update one (for instance to change the API), you'll need to update the other. This is a case where not keeping it DRY pays off--if you have to update the tests, that's a reminder that you'll also have to update all the other consumers of your API.

Similarly, the API docs often duplicate the truth that is built in to the code. That's because it's helpful to explain the truth to the computer in one way (using very precise code) and explain the truth to the reader in another way (using friendly, high-level English). Every truthful comment duplicates what the code already says, but not every piece of code is easily and quickly readable by human readers--this is especially true in, say, assembly.

Another area where truth is duplicated is in APIs. The function defines a name and an API. The caller uses it. They must agree on these things or the code won't work. If the caller decides to use a different name or a different API, the code will break. Essentially, programmers have decided that it's better to duplicate the name and the API rather than duplicate the contents of the function. This points to a useful trick--sometimes a small amount of duplication saves a large amount of duplication. You'll also see this sometimes in comments when they say "see also..."

Another source of duplication concerns public vs. private. For instance, in C, the same API is duplicated in the .h file and the .c file. Sometimes, the same piece of code must be duplicated in different projects. For instance, one operating system might need to define the same C types as another operating system because there's no easy way for them to share the same header files.

At a higher level, one time I had to add the same function I wrote to two projects. One project was proprietary company code. The other was open source (I had permission, of course). For technical reasons, it was impractical for the company code to import or subclass the open source code, so I was stuck just duplicating it.

Often, you'll need to duplicate the same piece of truth in multiple languages. For instance, think of how many HTTP client libraries there are in all the different programming languages. It doesn't matter how good an HTTP client library is if it's not easily accessible from the programming language I'm currently coding in. Sometimes there will be multiple HTTP client libraries for the same language because they're implemented differently (for instance, syncronously vs. asyncronously).

I mentioned tests before. Often tests duplicate some setup or teardown or perhaps the same pattern of interacting with a function. Refactoring is sometimes appropriate, but not always. It is commonly held that this is one area where keeping it DRY is less important than keeping it simple and isolated. A perfectly DRY collection of unittests that is difficult to comprehend and difficult to debug when something fails is less helpful than a set of simple, isolated unittests that contain a small amount of duplication. If the duplication causes multiple tests to fail, you'll know to keep fixing the tests until they all pass.

The question remains, how do you cope when you can't keep it DRY?

Greppability is very important. (By grep, I mean any tool that can search for a string or regular expression. I don't necessarily mean the UNIX tool "grep".) In highly dynamic languages like Ruby (that have great facilities for metaprogramming, but no static typing or interfaces) and highly factored frameworks like Rails (that use lots of files and levels of indirection), even a brilliant IDE can fail in comparison to a simple "grep tour". If you refactor a class in Ruby, how will you remember to refactor all the mocks of that class? You might have a user of your class that has a mock of your class that still makes use of your old API. The tests might be passing even though the code will assuredly crash. If you use grep, you can update all the callers of your class as well as all the mocks of the class. Grep can also help you find instances of a string in non-code documentation, and it even occasionally works with binary files. My point is, don't underestimate the utility of grep. Rather, you should aim for greppability. A function named "f" is not greppable, but a function named "calculate_apr" is. (By the way, naming all your loop variables "iterator" does not improve greppability, it just wastes time.)

Another way of coping when things aren't DRY is to have cross referencing comments. If you know that you must duplicate the same piece of truth in 5 places, add a comment next to each of those 5 places that refers to the other 5 places. Don't be afraid to duplicate the comment exactly. Your comment can say something like, "If you change this, don't forget to update..."

Another thing that helps mitigate duplication is proximity. Docstrings belong in the code because if a programmer updates one, he'll be more likely to update the other (although even proximity can't always help lazy programmers). If all the API documentation is in a separate file, that file will go stale very quickly.

Parallelization also helps. For instance, this code has a small amount of duplication:
 some_a = 1
some_b = 2
Sometimes you can factor out this duplication. However, in less dynamic languages like C, it may not always be easy to do so. However, parallelization can really help:
 some_a = 1
some_b = 2
Another old trick for coping with duplication is to have one source generate the other. Generating API documentation using javadoc is a good example of this. Sometimes you can use a program to generate code for multiple programming languages. There's another example of "generation" that I sometimes use in Python. I use string interpolation when creating docstrings. For instance, if there's a piece of documentation that should be duplicated in multiple places, string interpolation makes it possible so that I only have to write that piece of documentation once.

Another source of duplication has to deal with the plethora of tools programmers must use. There is the source code itself, a revision control system, a bug tracker, and a wiki. Often times, the same piece of truth needs to be duplicated in all of these places. This is one place where Trac really shines. Once you properly configure Trac, you can reference the bug number in each of your commits. Trac's commit hook will take that commit and add it as a comment in the original bug with a reference to the source code in Trac's source code viewer. Hence, Trac (which is a bug tracking system, a wiki, and a source code viewer) and the revision control system work together to reduce duplication.

It's unfortunate that life isn't always as DRY as you'd like it to be. However, keeping a few tricks in mind can really help mitigate the problems caused by having to duplicate a piece of truth in more than once place. If you have other tricks, feel free to leave them in a comment below.

Thursday, November 18, 2010

JavaScript: Naughty Socket.IO Example

File this under the "things you probably shouldn't do, but are fun anyways" category. Socket.IO is a library for Node.JS that provides Comet using a plethora of different approaches (WebSocket, Flash socket, AJAX long polling, etc.). I hacked the Socket.IO chat demo so that it reads HTML from my terminal and just dumps it to the browser. Hence, I can control people's browsers from my terminal. Insecure? Yeah. Fun? Oh yeah!

Anyway, here's how I hacked the server.js file in Socket.IO's chat demo:
io.on('connection', function (client) {

// Read from /dev/tty and send it to the browser.
var stream = fs.createReadStream('/dev/tty', {encoding: 'ascii'});

stream.on('error', function (exception) {
client.send({announcement: 'Exception: ' + exception});

stream.on('data', function (data) {
client.send({html: data});
And here's how I hacked chat.html:
function message(obj) {
var el = document.createElement('p');
if ('html' in obj) el.innerHTML = obj.html;
Here's what it looks like in my terminal:
sudo ./server.js
18 Nov 10:14:04 - socket.io ready - accepting connections
18 Nov 10:14:06 - Initializing client with transport "websocket"
18 Nov 10:14:06 - Client 5832344139926136 connected

<i>I'm typing this in to control the page.</i>
<script>alert('Oh baby!');</script> This doesn't work with innerHTML, thankfully.

Thursday, November 11, 2010

Jobs: Looking for People to Work With Me

I'm really enjoying myself here at Twilio. We're looking for a few more people, and I wonder if any of my readers would like to come work with me.

Twilio makes it easy for normal web developers to write voice and SMS enabled applications. If you don't know what I mean, try calling my app: (888) 877-7418. By the way, Jeff Lindsay, the SHDH house guy, is here too.

Here are the positions we're hiring for:
  • DevOps Engineer
  • Senior Software Engineer
  • Core Team
  • Software Engineering Leader, Organizer, Mentor
  • Customer Advocate
  • Developer Evangelist
  • Product Manager
We use a mix of Python, Java, PHP, and Ruby. We're in San Francisco. We just closed a second round of funding, but we also make a lot of money.

Here are the actual job postings. Contact Joanna Samuels for more information.

Tuesday, November 09, 2010

Books: Digital At Work: Snapshots From The First Thirty-Five Years

I just finished reading Digital At Work: Snapshots From The First Thirty-Five Years.
"Digital At Work" tells the story of the first thirty-five years of Digital Equipment Corporation [DEC] and illuminates the origins of its unique culture. First person accounts from past and present members of the Digital community, industry associates, board members, and friends - plus a wealth of photos from Digital's archives - trace the company's evolution from the 1950s to present.
In short, I really enjoyed it. By reading this book, I was able to vicariously experience the growth and history of one of the most significant companies in the history of computing, and it definitely left an emotional impact.

I think one of the most interesting things about Digital was its culture. Some people might call it chaos. Other people might call it a meritocracy. It was definitely in the MIT tradition. It wasn't uncommon to get into shouting matches over which approach to take. Good ideas were always more important than what little company hierarchy existed.

Here are a few random, interesting quotes I jotted down while reading the book. I left out the page numbers because you can just search for them in the PDF:
Fortune magazine’s report in the late 1950s that no money was to be made in computers suggested the word itself be avoided in Digital’s first business plan.

If you had to design a modern computer with the tools we had, you couldn’t do it. But to build the first computer was an eminently doable thing, partly because we could design something that we could build.

Many of Sketchpad’s capabilities were sophisticated even by the workstation standards of the 1980s. “If I had known how hard it was to do,” Sutherland said later, “I probably wouldn’t have done it.”

Six MIT students, including Alan Kotok and Peter Samson, bet Jack Dennis, who ran the PDP-1, that they could come up with their own assembler in a single weekend. They wrote and debugged it in 250 man-hours, and it was loaded onto the machine when Dennis came to work on Monday. This was the sort of job that might have taken the industry months to complete.

Success depended on extraordinary personal commitments, often creating high levels of personal stress. “The atmosphere has always been that of small groups of engineers with extremely high energy, working hard and aggressively for long, long hours-always on the edge of burnout,” says Jesse Lipcon. “That can be both positive and negative.”

“We didn’t have much experience,” says Cady, “but we were energetic, enthusiastic, and too dumb to know what we were doing couldn’t be done. So we did it anyway."

And, of course, we disagreed with much of what the original committee had done. So in the best Digital tradition, while creating the impression that the specs were frozen and we were just fixing some bugs, we surreptitiously went around changing many things, simplifying the protocols as much as we could get away with.

Primarily, architecture is the ability to take complex problems and structure them down into smaller problems in a simple, tasteful, and elegant way.

“It worked out that there were about a million lines of code in each new version of VMS,” says Heffner. “The first version was about a million lines of code, and by the time we got to Version 5, there were 5 million lines of code. We’re talking about a really heavy-duty operating system here, with more functionality than the world at that time had ever known.

We’d assign new kids to a senior person who would look after them, like an apprentice. Managing a good software engineer is like raising a kid-you want them to get into a little bit of trouble, but you don’t let them burn down the house.

[In Galway, Ireland] we were the only nonunion shop around, we paid well, and we did a lot of employee training so people could move up to higher-paying jobs very quickly. The hierarchy between workers and management was invisible.

This book is wonderful...As for people just entering the computer field, they will get a sense of how wonderfully uncomplicated things were, how exciting and liberating the challenges were, and how much actually got done.

Thursday, November 04, 2010

Linux: The Tiling Window Manager I Wish I Had

Every year or two, I switch to a tiling window manager such as xmonad or dwm. Inevitably, I switch back to GNOME after a couple weeks. Sometimes it's because the window manager doesn't fit in with the rest of my GNOME desktop (it used to be non-trivial to get xmonad to work with GNOME's panel). Sometimes it's because of bugs related to having a weird window manager (NetBeans used to freak out with xmonad, and Flash refused to go full-screen). Every time I try again, a bunch of things have improved. xmonad even had a project aimed at making it more accessible to GNOME users. Still, I think the biggest problem I have is that tiling window managers make some assumptions that just don't work out for me in practice.

I use more than just terminals. I still like to use things like GVim, the GIMP, Google Chrome, a graphical chat client, etc. In fact, I even get a real kick out of writing GUIs. Some tiling window managers assume you're going to live in a terminal (which is partially true), and they only give special attential to GUIs like the GIMP as an afterthought. The ramifications of that attitude tend to be frustrating in practice.

Ease of use is important. I'm as good at reading man pages as the next guy, but it's even better when I don't have to. Furthermore, every minute that I spend reading man pages is time I'm not a) getting my real work done b) actually using the software. It's really helpful if I can switch to a new window manager without spending a day trying to memorize all the key bindings or struggling to get it to work on my Ubuntu GNOME desktop. Don't get me wrong--I love hot keys, but it's better when I can learn them as I'm using the software (Firefox is like that). Furthermore, graphical configuration utilities are helpful, but don't go overboard. Whenever possible, I shouldn't have to configure it to do the right thing; it should do the right thing by default.

I want to see my desktop background. I'm a little bit on the obsessive compulsive side. Hence, my desk is always as empty as possible. I feel uncomfortable when it's messy. In the same way, when my desktop has only a couple windows open and I can see the desktop background, I feel like things are calm and under control. If my entire desktop is covered with text, I feel like I'm out of control. There's no reason why by default my 27" monitor needs to be completely covered in order for me to edit a single file. (Note, I make heavy use of virtual desktops in order to keep each individual desktop sparse and well organized.)

I want window decorations. More specifically, since I have a 27" monitor, I can afford to have more than a single pixel in order to separate windows. If there are nice borders between windows, it looks cleaner and more under control. Having text from two windows be only a single pixel apart makes me feel uncomfortable. Nice raised, lowered, etc. borders have been an important tool for UI designers for several decades--use them. Furthermoe, I want buttons. Buttons help you get started using software you're unfamiliar with. I know xmonad has done some work on this. If the buttons have mouse-over text containing the hot-keys, that's even better because it means I can learn the software and become faster as I'm using the software.

Windows should only expand when I ask them to. I don't need a confirm dialog to cover my entire 27" monitor. Most GUI windows know how big they should be by default, so it's best to respect that by default. This is really important with the GIMP. However, it's also important with things like GVim. By default, I want my terminal and GVim to be 80 columns wide. However, I want buttons on the bottoms and sides of the window so that I can tell the window manager that I want the window to maximize horizontally or vertically. For instance, I often want GVim to maximize vertically, but not horizontally. I want my terminal to be 80x24 unless I need to read some log file, in which case I'll press the buttons to maximize the window.

Overlapping windows are not the end of the world. A window that's too small is useless. In fact, when I use GNOME, I often stagger my windows. The windows should tile as long as there's room, but when there's not enough room, they should start to stagger. As long as I can see part of the window in order to recognize it and click on it to select it, that's fine. I also don't feel that it's strictly necessary to stick to a grid; think of how you pack a box with random items, and you'll know what I mean.

Small is not always better. My laptop has 4 gigs of RAM. Hence, it doesn't really matter if the window manager fits in 4k or 8k of RAM. It doesn't even matter to me if it takes 100 mb of RAM. Lynx is cool, but most of the time I use Google Chrome. Don't get me wrong; I like small, simple, efficient software as much as the next guy, but I also like software that's smart, friendly, and helpful. Let's face it, these days, my computer is much faster and has a lot more memory than I do, so let's optimize software for me, not the computer. Of course, let's not forget that large software can still be conceptually simple.

What I have described doesn't perfectly fit the model of a tiling window manager. What I have described is a normal window manager that has the personality of a tiling window manager. The things I like most about tiling window managers is that they are a) innovative b) helpful. I think those same characteristics would continue to be true for a hybrid window manager that behaved somewhere between a normal window manager and a tiling window manager.

Wednesday, November 03, 2010

JavaScript: A Second Impression of NodeJS

When I first heard about NodeJS, my reaction was, "Why would I use JavaScript on the server when there are similar continuation-passing-style, asynchronous network servers such as Twisted and Tornado in Python already? Python is a nicer language. Furthermore, I prefer coroutine-based solutions such as gevent and Concurrence." However, after watching this video, Ryan Dahl, the author of NodeJS, has convinced me that NodeJS is worthy of attention.

First of all, NodeJS is crazy fast. Dahl showed one benchmark that had it beating out Nginx. (However, as he admitted, it was an unfair comparison since he was comparing NodeJS serving something out of memory with Nginx serving something from disk.) It's faster than Twisted and Jetty. That last one surprised me.

Dahl argued against green thread systems and coroutine-based systems due to the infrastructural overhead and magic involved. He argued that he doesn't like Eventlet because it's too magical both at an implementation level and also because he doesn't like multiple stacks. As I said, I'm not at all convinced by his arguments, but it reassures me that he was at least able to talk about such approaches. When I brought them up during Douglas Crockford's talk on concurrency, Crockford just gave me a blank, dismissive stare.

Dahl argued that by using callbacks for all blocking calls, it's really obvious which functions can block. As much as I dislike "continuation-passing-style", he makes a good point.

Dahl argued that NodeJS has an advantage over EventMachine (in Ruby) and Twisted (in Python) because JavaScript programmers are inherently comfortable with event-based programming. Dahl also argued that JavaScript is well suited to event-based programming because it has anonymous functions and closures. Ruby and Python has those things too, but Dahl further argued that it's very easy to accidentally call something in Ruby or Python that is blocking since it's not easy to know if something blocks or not. In contrast, NodeJS is built from the ground up so that pretty much everything is non-blocking.

NodeJS has built in support for doing DNS asynchronously, and it supports TLS (i.e. SSL). It also supports advanced HTTP features such as pipelining, chunked encoding, etc.

NodeJS has an internal thread pool for making things like file I/O non-blocking. That was a bit of a surprise since I wasn't even sure it could do file I/O. The thread pool is only accessible from C since he doesn't feel most programmers can be trusted with threads. He feels most programmers should only be trusted with the asynchronous JavaScript layer which is harder to screw up.

Dahl still feels that it's important to put NodeJS behind a stable web server such as Nginx. He admits that NodeJS has lots of bugs and that it's not stable. He's not at all certain that it's free of security vulnerabilities.

In general, Dahl believes you'll only need one process running NodeJS per machine since it is so good at not blocking. However, it makes sense to use one process per core. It also makes sense to use multiple processes when you need to do heavy CPU crunching. At some point in the future, he hopes to add web workers a la HTML5.

NodeJS has a REPL.

Dealing with binary in NodeJS is non-optimal because dealing with binary in JavaScript sucks. NodeJS has a buffer class that sits outside V8. V8's memory management makes it impossible to expose pointers since memory may be moved around the heap by the garbage collector. Dahl would prefer to deal with binary in a string, but that's not currently possible. Furthermore, pushing a big string to a socket is currently slow.

Dahl works for Joyent.

Although I still feel Erlang has a real edge in the realm of asynchronous network servers, Erlang is difficult for most programmers to adapt to. I think NodeJS is interesting because it opens up asynchronous network programming to a much wider audience of programmers. It's also interesting because it allows you to use the same programming language and in some cases the same libraries on both the client and the server. I'm currently looking at NodeJS because I want to use socket.io for Comet aka "real time" programming. Only time will tell how NodeJS works out in practice.

Friday, October 29, 2010

Telephony: Blatantly Open Source Phone Trees

As I mentioned in my last post, I just launched Teladventure, a phone based adventure game. (The phone number is (888) 877-7418.)

Anyway, I added a "learn more" menu option to the phone tree. It says:
Teladventure was created by Shannon -jj Behrens using Ruby on Rails and Twilio. Please feel free to submit feedback to the author by emailing him at j,j,i,n,u,x,at,g,mail,dot,com.

Teladventure is open source. You can get the source code from git,hub,dot,com,slash,j,j,i,n,u,x,slash,t,e,l,adventure.
(The commas are there to help the text to speech engine.)

Why am I bringing this up? When was the last time you called a phone number and got a phone tree that told you that it was open source and gave you the URL to download it? ;)

Teladventure: A Phone-based Adventure Game

I just finished building a phone-based adventure game. The number is (888) 877-7418 (which is a toll free number). The code is here: http://github.com/jjinux/teladventure. Give it a shot!

I used Ruby on Rails, behavioral-driven development (with Cucumber), and Twilio. However, these same tricks work in Python too since Twilio is language agnostic.

Check out my previous post about using behavior-driven development to build phone trees.

Tuesday, October 26, 2010

Ruby: Phone Trees

I just finished building my first, fully-functional, non-trivial phone tree. This is exciting because it only took me two days. It has a bunch of submenus, it records and plays back content from the user, and it's even pretty friendly about confirmations.

I built it using Ruby on Rails and Twilio which is where I just started working. Building a telephone app using web technologies is very comfortable.

What's most amazing is that I used behavioral driven development using Cucumber in order to develop the app. In fact, I didn't even call my app until I had finished building it. It was mostly correct the first time I called it. Using Cucumber to walk through all the user-flows in the phone tree was a big win!

Note, I didn't use one assertion per test as I warned against here, but rather wrote very full integration tests that walked around the phone tree quite a bit. Here's an example test that actually interacts with the phone tree:
Scenario: delete a friend, but cancel
Given I already have 5 friends
When I enter "delete_friend" on the edit friend menu for the friend at position 3
Then I should get a valid TwiML response
And it should say "You are about to delete"
And it should play "name_3"

When I follow the redirect
Then I should get a valid TwiML response
And it should say "If this is correct"
And it should say "If this is incorrect"

When I enter "incorrect"
Then I should get a valid TwiML response
And it should say "The delete has been canceled."

When I follow the redirect
Then I should get a valid TwiML response
And it should say "Main menu."
Unfortunately, I can't share the entire source code for this application yet or the phone number since this is a commercial application I'm building for a customer. However, as soon as I finish my pet project that I'm building on the side, I'll post its phone number and its source code. I'm anxious to share a bunch of tricks that I came up with for making it easier to develop phone trees using Ruby on Rails and Cucumber.

Wednesday, October 13, 2010

Rails: Fat Tests Considered Unharmful

How many assertions should a test have? For instance, how many times should you use ".should" in an RSpec test? How many times should you use "Then" in a Cucumber test?

I have seen some developers work very hard to never have more than 1-2 assertions per test. They consider tests that are more than 3 lines long to be a code smell, and they tend to break up tests so that there is one assertion per test. I'd like to suggest that this conventional approach has some downsides that are too often overlooked.

First of all, what are the upsides of super skinny tests?

If you put 10 assertions each into their own test, then you can get very fine-grained results concerning which tests pass and which tests fail. That's useful. However, fine-grained results aren't all that crucial. I actually don't care all that much whether one test fails or ten tests fail. It doesn't really tell me how many bugs there are. Either way, it could be a single misspelling. Any number of tests failing is bad, and I don't regain confidence until all the tests are passing.

The next benefit of super skinny tests is that it helps prevent one assertion from affecting the outcome of another assertion. There's nothing more frustrating than tests that fail if you run them one way and pass if you run them another way. However, super skinny tests don't always address this problem. For instance, if you have data cached at the class level, that data will persist even between tests. Furthermore, a lot of tests are so simple you don't really need to worry very much about them affecting one another. For instance, if you check that a string contains "foo", that won't realistically have any affect on another assertion that checks whether it has "bar".

One more benefit of skinny tests is that they result in a comforting feeling. What's better 1000 tests or 1500 tests? The gut reaction is that 1500 tests is better, and I'll admit that I like lots of dots on the screen just as much as the next guy. However, if the 1500 tests only contain 1500 assertions whereas the 1000 tests contain 3000 assertions, it may be that the 1000 tests test more. I don't think there's any shame in adding a few more assertions to an existing test if you think that it adds real value to that test. Let's face it, testing isn't really about who can create the largest number of tests. It's about catching bugs that arise as the code is extended or refactored.

There's an old saying that says the best tests are the ones that get run. There's a hidden cost to super skinny tests that often gets overlooked. Consider the following hypothetical, functional RSpec test:
describe FilmsController do
before(:each) do
@film = Factory :film
get films_path

it "should have content" do

it "should set flash correctly" do

it "should have the right title" do

it "should set the right value for such and such assign" do

it "should set the right value for some other assign" do
What's wrong with this set of tests? Well, first of all, I prefer Cucumber integration tests over RSpec functional tests, but let's put that aside from a moment. There is a before filter that runs before each test. Hence, each test has to create a new film record and then regenerate the whole page. Generating the page probably results in a bunch of queries too. Let's assume 10 tests, 1 setup query per test, and 7 queries per page load. That's 80 queries for 10 assertions.

However, none of these assertions is all that complex. Furthermore, they're all simple enough that they are extremely unlikely to have side effects on one another. We could replace the before(:each) with a before(:all), but that turns out not to work out so well in practice. I suspect it's because each test runs in its own transaction.

Alternatively, we can collapse this into one test:
describe FilmsController do
it "should generate a proper index" do
@film = Factory :film
get films_path
test the content...
test the flash...
test the title...
test such and such assign...
test some other assign...
Not only does this remove a lot of boilerplate, it also cuts down the number of queries to 8. That means the tests run faster. When the tests run faster, they're more likely to be run as you develop. (I assume you already have integration testing that will run all the tests once you check the code in, but there's still value in running the tests as you develop.)

In conclusion, I think it's best not to go too overboard keeping your tests skinny. If two assertions require different setup, certainly put them in separate tests. However, once you have everything setup, if you can think of a few extra assertions that would add value to the test, don't be afraid to add them to the same test. I even like to occasionally add some assertions as I'm doing the setup just to make sure that the computer and I are on the same page (although I think it's best not to re-test things that have already been tested elsewhere). Certainly, I wouldn't recommend humongous tests that are 100s of lines long, but neither should every test be constrained to a single assertion.

Wednesday, October 06, 2010

Personal: Mostly Offline for a Week

Hey everyone,

I'm going to go mostly offline for about a week in order to recharge. If you need me, call me.

Next week, I start my new job at Twilio! :)

Best Regards,

Saturday, October 02, 2010

Personal: Fandor is Launching

The site I've been working on for the last year, Fandor, is finally ready! If you like independent and international films, you might want to check it out. Fandor is a subscription service for streaming independent films on demand. As we move into a new phase, we are inviting our friends and family to sign up for a one-month free trial.

To sign up, go here and fill out the quick subscription form: fandor.com

This is the first version of the site, so we anticipate many changes to come. Please don't hesitate to give feedback. Your comments and questions will be very helpful as we shape the service. By the way, no it wasn't my idea to require a credit card before you can look around on the site ;)

Have fun!

Thursday, September 30, 2010

Tailing Whitespace

Here's how to get rid of trailing whitespace from all your files:
# Strip trailing whitespace from files that end in .as.
# Change this to match your programming language.
# I'm not sure if the -i flag is supported by all OSs.
find . -name '*.as' | xargs sed -i 's/ *$//g'

# See if the changes look reasonable.
git diff

# There should be no changes that aren't whitespace changes.
git diff -w

Wednesday, September 15, 2010

Quotes: A Recruiter on Scala and Erlang Jobs

I got the following from a recruiter today:
Scala (and erlang while we're at it) roles are like UFO's, there are always sightings of them, but I just don't think they're real. :)
That's a pretty funny quote ;) It just reminds me of the fact that jobs are a trailing indicator of a technology's success. For instance, there are currently five jobs for COBOL on Craigslist, which is actually the same as the number of Scala jobs on Craigslist.

Tuesday, September 14, 2010

Quotes: Mark Twain on Math

"In the space of one hundred and seventy-six years the Mississippi has shortened itself two hundred and forty-two miles. Therefore ... in the Old Silurian Period the Mississippi River was upward of one million three hundred thousand miles long ... seven hundred and forty-two years from now the Mississippi will be only a mile and three-quarters long. ... There is something fascinating about science. One gets such wholesome returns of conjecture out of such a trifling investment of fact."
-- Mark Twain

Quotes: Max Levchin on Startups

"The very first company I started failed with a great bang. The second one failed a little bit less, but still failed. The third one, you know, proper failed, but it was kind of okay. I recovered quickly. Number four almost didn’t fail. It still didn’t really feel great, but it did okay. Number five was PayPal."
-- Max Levchin

Saturday, September 11, 2010

Scala: Using Akka with Play

Here's a screencast of using Akka in the context of Play. Akka's motto is, "Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Actors". The screencast shows how to implement a hit counter using STM (software transactional memory). It's acting as a library within the context of a Play application. That surprises me (in a good way) since I was afraid you'd have to run Akka out of process. The screencast shows how STM is able to maintain correct results under a heavy, concurrent load. It looks pretty painless. There's another screencast showing how to do the same thing with actors, and then another that shows how to do the same thing for a cluster of servers using remote actors.

Java: The Play Framework

I just watched a screencast for the Play framework. It's motto is:
Finally a Java framework made by Web developers. Discover a clean alternative to bloated enterprise Java stacks. Play focuses on developer productivity and targets RESTful architectures.
All I have to say is wow!

I plan on using Play using Scala instead of Java, but nonetheless, I'm completely amazed at how friendly, efficient (to develop), and polished it is. The API looks very much like Rails, and so does the development process. For instance, you just edit files and hit reload. Compile errors are shown directly in the browser, using a beautiful interface. The MVC approach and file layout looks like Rails, however there's an admin interface that looks like Django.

I've heard multiple people say that you can go from not knowing Play at all to having something working to show to your boss in about two days. Best of all, Play has put a lot of effort into its Scala bindings. I hate to sound like a fan boy, but I'm really excited!

Friday, September 10, 2010

Scala: Lift vs. Play! For Web Development

If you're interested in doing Web development in Scala, have a look at my Stack Overflow question and even more importantly this thread that I started on the Bay Area Scala Enthusiasts mailing list.

In it, there are some choice quotes from David Pollak such as:
In terms of Lift and older browsers, Lift doesn't support them (or at least it doesn't support them well.) At the end of the day, in order to use a Lift app, you need a modern browser (IE 6+, Firefox 1.5+ or WebKit-based [Chrome, Safari])...

Put another way, with 99%+ of the apps people are writing in Lift, they will live in a single JVM...

[When I asked whether a user's session (such as the contents of his shopping cart) was lost when new code was deployed to the server because of the statefulness of Lift, David said] Most of the site deployments that I do in production are during well defined maintenance windows in which the entire service is shut down. I realize that there's a class of services for which that's not acceptable, but the vast majority of sites (the bottom 98% or so) are going to be cool with the maintenance window. If that's not acceptable, there will be a commercial Lift Cluster Manager...

As a practical matter, the deployment and crash scenarios that folks raise are premature optimizations because in the real world, nobody notices (or cares to complain) when Lift sessions stop and are restarted elsewhere.
I'm feeling a little down since I was flamed pretty badly in that thread. Obviously, people in the Python world either like or at least tolerate me a lot more than people in the Scala world.

Anyway, my current plan of action is to learn the Play! framework and Akka. Both look very interesting and very polished. I'm hoping to find a job coding in Scala, but since I've ruled out Lift, my prospects are looking even more slim ;)

Thursday, September 09, 2010

Scala: Scala for Newbies Night at Linkedin

There's going to be a Scala for Newbies meeting at Linkedin at 7PM, Monday, September the 13th at Linkedin in Mountain View. Here's the full announcement.

Saturday, September 04, 2010

Personal: I'm a Wireless Black Hole

My wife and I both have Samsung Moment cell phones running Android on Sprint's network. I always complain that I can't get reception in the house, whereas her reception is always fine.

Today, we did an experiment. She had 4 bars on her phone. She passed her phone to me, and it dropped down to 0 bars. I gave it back, and she had 4 bars again. I decided to sit in her seat. I still only got 1 bar.

What the heck? The only thing I can think of is that she's really small, and she's a bit on the anemic side, whereas I'm fairly large and I have an abundance of iron in my blood. Does that even matter?

We just conducted the experiment again, standing next to each other, with the same results. The way we hold the phone doesn't seem to make a difference. We're both barefoot.

Saturday, August 28, 2010

Rails: reset_session and Webrat Don't Play Nicely with One Another

All the Rails security guides say that you should call reset_session after the user logs in or logs out. This clears out the session and forces a new session ID to be created. It seems there have been a few Rails bugs related to reset_session over the years.

In my login action, I call reset_session and then put a nice message in flash. When I actually use the website, I can see (via Firefox) that I'm getting a new session ID, and I can see my flash message. However, when I write tests for those two things, the flash message gets lost, and I don't get a new session ID in my cookies. It's almost as if the new session is being ignored, and the old session is being used.

I submitted a bug to Webrat about this, but it turns out it's an issue in Rails. This issue is present in version 2.3.8. If this is affecting you, there's a workaround here. I implemented the workaround, and it worked like a charm :)

Friday, August 27, 2010

Ruby: Blooger

I needed some sample code for an interview, so I built a website called Blooger. It's a site (like Blogger) where people can go create a blog to blog about their booger stories. It even has functioning Atom feeds :)

Here's the source code. I used Cucumber, RSpec, Webrat, and factory_girl to do behavior driven development. I used Authlogic for authentication. I used BlueCloth for Markdown support in order to format the blog posts. The README in the source has more details.

Saturday, August 21, 2010

Books: Programming Scala

I just finished reading Programming Scala. In short, I really liked it.

The first few chapters are breathtakingly fast. Some of the middle chapters are kind of slow, but are still worthwhile. Scala is a fairly large language (unlike, say, Scheme or C), and the book is a fairly dense 400 pages. I found it helpful to read slowly and take notes.

As for Scala itself, I really like it! Scala is a nice mix of Java, C#, Erlang, Haskell, Ruby, and Smalltalk. You can treat it as a "better Java", or you can treat it as a more enterprise-friendly Haskell. Either way, it's exactly what I was looking for: a language with reasonable syntax, an ML type system, and a decent set of real world libraries. I know that the Haskell community is working hard in this direction as well. I think Scala stands a very good chance at being a work-friendly, programmer-friendly language.

I'm a little afraid that its type system may be too large and too complex for a lot of programmers. To be fair, I think that C++ is too large and complex for most programmers too. I'm also afraid of subsetting--i.e. the situation where every team picks a different subset of the language to use. This is very common in multiparadigmatic languages like C++. On the other hand, some multiparadigmatic languages like Python seem to avoid this problem.

There's a lot to learn in Scala--covariance, contravariance, parameterized types, abstract types, self-type declarations, lazy values, by-name parameters, DSL-friendly abbreviations, path dependent types, oh my! And there's more! I ended up with 50 pages worth of notes.

I'm pretty sold on Scala. Now, all I need is a startup to hire me to write it ;)

Saturday, August 14, 2010

Personal: Looking for Work

I'm looking for a new job. As much as I've enjoyed my time at my current company, I'd like to move on to something new.
  • I'd like to give Scala a shot, if possible. For the last year, I've been doing Ruby on Rails, and before that I was doing Python. I've been learning Scala in preparation of making the transition.
  • I'm only looking at companies in San Francisco or the East Bay.
  • I prefer behavior driven development and really clean code. I'm really bad at working on sloppy code bases with no tests and no documentation.
  • I prefer Linux.
  • I have a predilection toward startups.
Here's my resume. Thanks!

Saturday, July 31, 2010

Scala: Tweet Tweet Tweet

I'm almost done reading Programming Scala. I've really enjoyed the book as well as Scala.

One of the lead developers is Alex Payne who is a lead developer at Twitter. I'm very impressed with his stuff, but having read the following piece of code from the book, it really makes me curious what the Twitter code base looks like ;)
def saveTweet(tweet: Tweet) = tweets ::= tweet

Thursday, July 29, 2010

Flash: I Like Wowza

I like the Wowza Media Server better than Adobe's Flash Media Server.
  • It costs less.
  • So far, it seems less buggy.
  • Plugins for it are written in Java, which is more pleasant than server-side ActionScript. Server-side ActionScript is very different than ActionScript 3 (which I actually like).
  • The API seems nicer.
  • It has a very nice admin and monitoring system--Java's standard--jmx.
  • The server can simultaneously support streaming the Adobe way (RTMP, etc.), the Microsoft way (smooth streaming), and the Apple way (HTTP live streaming). It supports all three including bitrate switching all from the same source files.
The one missing feature that it doesn't support is SWF verification. It does support something called secure tokens, but I can make a pretty decent argument that SWF verification is more secure (at least for my use case).

Credit goes to Dan Aronson for a lot of this content.

Wednesday, July 28, 2010

Software Engineering: Premature Featurization

"Premature Featurization" is my term for the tendency to implement a plethora of features before they've even been requested by real users just in case a user might want them.

It's related to creeping featurism. Like premature optimization, it leads to difficult-to-maintain code bloat.

Product managers and managers in general are especially prone to premature featurization. When experienced developers themselves fall prey to premature featurization, they may be suffering from second system effect.

To some degree, the opposite of premature featurization is creeping elegance, which is where an existing piece of code is polished excessively at the expense of other factors such as the schedule or real world features.

It should be pointed out that agile software development aims to prevent premature featurization in that it forces coders to code only those features that the stake holders have requested as they request them. However, it tends to be an enabler for premature featurization in that it allows stake holders to successfully request an unlimited number of features since the programmer doesn't have to design a complete, minimalistic, cohesive design ahead of time.

Whereas time management is occasionally used by project managers to organize a developer's time as he attempts to implement a premature list of features, the goal of task management is to constrain the list of features requested to the minimal set possible.

Saturday, July 24, 2010

Linux: Retro Desktops

I like messing around with my desktop. I decided to give it a retro Windows look. I used the Redmond theme, set all my fonts to Monospace 10, used a boring gray background, and tweaked my editor and terminal colors.

Now I know that Windows 2000 didn't use monospace fonts for everything, and it supported backgrounds other than a drab grab, but doing it this way sets a pleasant, calm, nostalgic mood.

What's strange about this 1 AM post is that a) I've been a Linux user for so long that I've never even used Windows 2000 b) I'm using Mint Linux which aims at offering a very elegant, polished look--which I've totally screwed up ;)

By the way, what I'd really like is a desktop that looks like the original Macintosh. That'd be pretty neat looking for this MacBook running Linux ;)

Ok, yes, I probably need to get a life, but I wonder if there's anyone else out there who likes retro looking desktops.

Friday, July 23, 2010

Apple: Cleaning a White MacBook

In a moment of reckless abandon, I decided to clean my white MacBook today. I wash my hands almost every time I use my laptop, but oil from my hands still builds up and attracts dirt. This leads to very irritating grime. Ordinary water won't due the trick, so I decided to tempt fate and use something with bleach. I'm pretty sure that voids my non-existent warranty.

I sprayed some Easy Off BAM Multi-Surface Power Cleaner with Bleach onto a damp paper towel and then rinsed out the paper towel a couple times. I squeezed as much water out of the paper towel as possible, and then I wiped my keyboard, etc. clean. I didn't put the cleaner on the screen--I'm not that brave ;) Then, I took a microfiber towel, wet it, and wiped down the whole laptop. Then, I used a dry part of the same towel and dried it completely.

The wet microfiber towel always does a good job with the screen. The cleaner did a great job with the grime. Some of the keys are still shiny (presumably from hand oil), but it's now very clean. I performed the entire operation without turning off the laptop.

So far so good. I'll post an update if my laptop suddenly melts tomorrow or if the dirt returns within a week.

Thursday, July 22, 2010

An Email Scam

My wife encountered an email scam that I had never seen before. She got an email from a friend's email address:
I'm caught up in a real mess and i need your help. I'm sorry I didn't inform you about my trip,I had a trip to the Wales United Kingdom and a bizarre thing happened to me.I was mugged at gun point last night, it happened at the park of the Hotel were i lodged but thank God i wasn't hurt, the muggers carted away with all my belongings excluded my passport. Cell,credit card,cash and some important documents are all gone. I was able to make contact with the UK Police and i was directed to the Embassy, but they seems to be taking things too slow.I need your help so urgently.. My flight leaves pretty soon but i am having problems sorting out the hotel bills also getting my ticket straightened out. Ineed your help. I need a quick loan of ($1,800) to get things fixed out here,I promise to refund as soon as i get back home.. please reply asap. so i can tell you what to do and how to get the money to me..Hope to read from you soon..
It smelled fishy, but the email headers were legit. My wife decided to call her friend. By chance, she didn't pick up and we had to leave voice mail, which had us worrying that it my be real.

My wife responded to the email saying:
I didn't know you were out of the country! Just to make sure no one hijacked your e-mail account, can you tell me how you know me?
Within a minute, she got a response:
My email was not hacked,l know this sounds weird and you wouldn't believe me,i'm really stuck out here in the UK and it's so devastating at the moment.l wish i could call but i don't have access to phone at the moment ,I have nothing left on me and i'm glad that i still have my life and my passport cos it would have been worst if they made away with my passport.I am full of panic now and the police only asked me to write a statement about the incident and directed me to the embassy,i have spoken to the Consulate here but they are not responding to the matter effectively,I really need your help to get myself out of this place.
I realized that the result came too quickly, but more importantly, we were involved in an actual Turing test ;) Since the email didn't answer the question we asked, we assumed it was fake.

A few minutes later, my wife's friend finally called back. She wasn't in the UK. We told her about the scam and told her to change her password, and we would see if the robot would continue responding. It turns out, she was locked out of her account--it had been hijacked. Gmail has a process to recover a hijacked account, and the last we heard, she was going through that process.

Crazy. I wonder if we're going to see more scams like this.

Monday, July 19, 2010

Personal: Working at Google--the Hard Way

I've always wanted to go work for Google the hard way--i.e. to work at a company and get acquired by Google. Metaweb just got acquired by Google. This is the second company that I worked at that got acquired by Google--after I left. The first was Mochi Media. Ugh.

Congrats to the guys at Metaweb! They have some really exciting technology.

Monday, June 28, 2010

Ruby: Additional Attributes in ActiveRecord

I'm a bit surprised that this returns a string instead of an int:
Book.find_by_sql("select name, 5 as some_number from books").first.some_number
My buddy Shailen was the one who pointed this out to me.

Monday, June 21, 2010

Ruby: rails_xss

There's a new plugin called rails_xss:
This plugin replaces the default ERB template handlers with erubis, and switches the behaviour to escape by default rather than requiring you to escape. This is consistent with the behaviour in Rails 3.0.

Strings now have a notion of "html safe", which is false by default. Whenever rails copies a string into the response body it checks whether or not the string is safe, safe strings are copied verbatim into the response body, but unsafe strings are escaped first.
This is a nice feature that I used to like in other templating engines, so I'm glad Rails now has it as well. I just updated my code to use it, and things went relatively smoothly. I deleted all the useless calls to "h()" and added "raw" or "html_safe" in all the right places. Thank goodness for tests ;)

Saturday, June 12, 2010

A Few Simple Ruby, Rails, and MySQL Tips

Sometimes it makes sense to store the value of a query in a variable within MySQL. You might do this if you're avoiding ActiveRecord because you're in a data migration. Here's how:
    execute %{SELECT (@setuid_id:=id) FROM roles WHERE title = 'setuid'};
execute %{SELECT (@admin_id:=id) FROM roles WHERE title = 'admin'};
Similarly, sometimes it's helpful to use the results of a query in order to insert several rows at the same time. MySQL has INSERT...SELECT syntax that can be used like this:
    execute %{
INSERT INTO roles_users (role_id, user_id)
SELECT @setuid_id, user_id
FROM roles_users
WHERE role_id = @admin_id
The above query finds all the admins and gives them the setuid role as well.

If you're a Python coder coding in Ruby, don't forget that ["a", "b", "c"] is more simply written %w(a b c).

You probably know that you can lookup rows via ActiveRecord finder methods like Role.find_by_name(name). However, did you know that you can also find multiple records at the same time via Role.find_all_by_name(names_array)?

Remember that Role.find_by_name(name) will return nil if there is no such record, whereas Role.find_by_name!(name) will raise an exception if there is no such record. Unfortunately, there is no "!" version of Role.find_all_by_name. Hence, if you need to make sure that every name resulted in a record, you'll have to look at the number of records returned.

Thursday, June 10, 2010

IDEs: Netbeans

As I've mentioned before, I use Netbeans with the jVi plugin. It's enough IDE to improve my productivity without being so much that I feel overwhelmed. Using the jVi plugin makes a Vim fanatic like me feel right at home.

Just in case it's helpful, here's my NetBeans setup (in outline form):
Setup NetBeans:
Downloaded the Ruby pack.
It's a shell script. Run it as root. Don't install GlassFish.
It installs to /usr/local/netbeans-6.8.
To uninstall: /usr/local/netbeans-6.8/uninstall.sh
There is a menu item to run it.
After install:
Installed the Python plugin:
Jython distribution.
Installed the PHP plugin.
Installed the Cucumber Features plugin.
Installed the nbgit plugin by downloading it manually.
Installed the jVi plugin by downloading it manually.
Setup jVi:
Tools >> Options:
Buffer Modifications:
bs: 2
sts: 2
sw: 2
Control-Key Bindings:
Click Control-V to enable rectangle edit.
Unclick Control-W so I can close windows quickly.
Set the default font to Monospace 12:
Make sure you're not setting the whitespace category.

Ruby: Using reset_session in Rails with Cucumber and Webrat

I filed the following bug:
All the Rails security guides say that you should call reset_session after the user logs in or logs out. This clears out the session and forces a new session ID to be created. It seems there have been a few Rails bugs related to reset_session over the years. However, I'm now worried that there's a conflict somewhere related to Rails' testing infrastructure and/or Webrat.

In my login action, I call reset_session and then put a nice message in flash. When I actually use the website, I can see (via Firefox) that I'm getting a new session ID, and I can see my flash message. However, when I write tests for those two things, the flash message gets lost, and I don't get a new session ID in my cookies. It's almost as if the new session is being ignored, and the old session is being used.

I'm sorry, but I can't tell exactly where the problem is. I know that there's special Rails code that handles session cookies when you're doing integration testing.

I'm using actionpack-2.3.8 and webrat-0.7.1.
Here's how I did my best to work around the problem.

First of all, I'm storing my sessions in the database. Hence, I created an ActiveRecord model to manually delete session records:
class Session < ActiveRecord::Base
# Rails' reset_session has been nothing but trouble for us. I'm taking
# matters into my own hands. This code will have to change if we stop
# putting our sessions in the database.
def self.nuke_session(session_id)
Since I'm using authlogic, my logout action looks something like this:
def destroy

# Avoid session fixation attacks. This may seem redundant, but it was
# necessary to make the tests pass.
session[:test_that_this_disappears] = 'ok'
session_id = cookies[ActionController::Base.session_options[:key]]

redirect_to :action => :logged_out
My login action is something like this:
def create
@user_session = UserSession.new(params[:user_session])
unless @user_session.valid?
return render :action => :new

# Avoid session fixation attacks by resetting the session.
@user_session = UserSession.new(params[:user_session])

# For some reason, flash, reset_session, Cucumber, etc. don't get along,
# so I have to pass the flash message via a parameter.
redirect_to(root_url(:login => 1))
Instead of setting a message via flash, I pass a query parameter.

I check for this parameter in my ApplicationController:
before_filter :check_for_login_message
# For some reason, flash, reset_session, Cucumber, etc. don't get along.
# Hence, after login, we have to pass the flash message via a query parameter
# rather than via flash.
# If I have to do this kind of thing again, I'll create a lookup table full
# of messages.
def check_for_login_message
flash.now[:success_message] = "Login successful!" if params[:login]
Finally, I have this in a feature file:
# This test should work.  In fact, it does work when you use your browser.
# However, there's a bug somewhere between Rails and Webrat that prevents
# it from working when you use Cucumber. Somehow, reset_session is broken.
# Scenario: logging in should invalidate your session cookie
# Given I am on the homepage
# When I look at my session cookie
# And I am logged in
# Then I should have a different session cookie
That test relies on these steps:
# This test should work.  In fact, it does work when you use your browser.
# However, there's a bug somewhere between Rails and Webrat that prevents
# it from working when you use Cucumber. Somehow, reset_session is broken.
# Given /^I look at my session cookie$/ do
# @cookie = cookies[ActionController::Base.session_options[:key]]
# end
# Then /^I should have a different session cookie$/ do
# new_cookie = cookies[ActionController::Base.session_options[:key]]
# new_cookie.should_not == @cookie
# end

Tuesday, June 01, 2010

Linux: Qimo for Kids

I just installed Qimo, an Ubuntu-based distribution for kids. If you already have Ubuntu, just run "apt-get install qimo-session".

I just started poking around, but I like it already. I had Edubuntu previously. For some reason, this seems simpler, neater, easier, and more engaging. I think Edubuntu has everything Qimo has, and more, but Qimo presents it in a more compelling manner.

By the way, Qimo doesn't provide parental controls, so you'll have to install something extra if you're into that sort of thing.

Saturday, May 29, 2010

ActionScript: JavaScript Event Handlers for JW Player

JW Player has a rich JavaScript API. One of the things you can do is to create JavaScript event handlers for ActionScript events. However, there are a few things that can bite you in the butt if you don't keep them in mind.

First of all, as the documentation points out, you can't set an event handler until JW Player is ready for it. If you set it too early, it won't work, and you won't even get an error message. If you have a function called playerReady, it'll automatically be called by JW Player. That's a great place to setup your handlers.

Next, when JW Player calls playerReady, it's supposed to pass an obj containing the ID for the HTML DOM object. In my experience, it doesn't. Hence, you have to lookup the object manually. See my previous post about the fact that you can't use jQuery's $() to lookup the object, but should instead stick with document.getElementById.

Last of all, remember that when you give JW Player your JavaScript callback, you have to pass the function's name as a string. It feels natural to pass a function pointer or even to have an inline function, but that doesn't work (nor will you get an error message).

In summary, here's the code:
function playerReady(obj) {
"TIME", "handleTimeEvent");

function handleTimeEvent(e) {

ActionScript: jQuery and ActionScript Callbacks

Let's suppose you have a SWF, and you're calling callbacks on that SWF from JavaScript. The following works:
The following doesn't work, and there is no error message:
I often think of $() as a way to use document.getElementById, but since it returns a jQuery shim, some things don't work as expected. This one took me quite a while to figure out.

Thursday, May 20, 2010

Neuroscience: Burn-out Visible in the Brains of Patients

I just found this on Hacker News: Burn-out visible in the brains of patients. Since I've suffered from burnout for about a decade, this comes as no surprise to me.

Try to do pushups until you can't do any more. Now, wait a minute, and then do 50 more pushups. That's the best way I can explain what burnout feels like--my brain just feels like jello a lot of times.

I'm sure a lot of other programmers have to deal with this just like I do.

Tuesday, May 18, 2010

Apple: iPad and Emacs

Someone asked my boss's buddy Art Medlar if he was going to buy an iPad. He said, "I figure as soon as it runs Emacs, that will be the sign to buy." I think he was just trying to be funny, but his statement is actually fairly profound.

It's well known that submitting iPhone and iPad applications for sale on Apple's store is a huge pain--even if they're free and open source. Apple is acting as a gatekeeper for what is and isn't allowed on your device. I heard that Apple would never allow a scripting language to be installed on your iPad because it would allow end users to run code that they hadn't verified. (I don't have a reference for this, but if you do, please post it below.) Emacs is mostly written in Emacs Lisp. Per Apple's policy, I don't think it'll ever be possible to run Emacs on the iPad.

Emacs was written by Richard Stallman, and it practically defines the Free Software movement (in a manner of speaking at least). Stallman's vision for the future of computing is very open, and Apple's vision for the future of computing is very closed. Hence, it's ironic that Emacs, which is such a profound part of Free Software history, can't ever run on the iPad.

Hence, I think there's a profound truth when Art Medlar said, "I figure as soon as it runs Emacs, that will be the sign to buy."

Tuesday, May 11, 2010

Books: The Little Schemer

I just finished The Little Schemer. I liked it. It's a very short book. Most of my time was spent doing the exercises, which were very worthwhile. There were a few sections I had to read twice, but the book was far more accessible than say SICP. If you've never coded in Lisp or Scheme or if you're looking for some good exercises for your coding fu, I highly recommend this book. Happy hacking!

Saturday, May 08, 2010

Python: The Halting Problem

I've been reading The Little Schemer, and I particularly enjoyed learning about the halting problem. I enjoyed learning about it so much that I figured I'd take the time to explain it in Python so that other Pythonistas could enjoy it with me ;)

Let's suppose I have two functions:
def does_stop():
return True

def does_not_stop():
while True:
does_stop does stop. does_not_stop does not stop. How do I implement the following function:
def will_stop(f):
"""Return True if the function f eventually stops, and False otherwise."""
Let's suppose for a moment that I can implement such a function. Now, let's look at this function:
def just_might_stop():
return will_stop(just_might_stop) and does_not_stop()
Does just_might_stop stop or does it continue forever? That is, what is the value of "will_stop(just_might_stop)"?

Well, I don't know yet, but let's suppose that "will_stop(just_might_stop)" returns True, i.e. that just_might_stop does stop. Well, looking at the definition of just_might_stop, it should "return will_stop(just_might_stop) and does_not_stop()". Since we stipulated that "will_stop(just_might_stop)" is True, then the function will execute "does_not_stop()", and hence, just_might_stop will never stop.

This time, let's suppose that "will_stop(just_might_stop)" returns False, i.e. that just_might_stop does not stop. Looking at the definition of just_might_stop, it should "return will_stop(just_might_stop) and does_not_stop()", i.e. it should return False. However, that means that just_might_stop did stop, even though we stipulated that "will_stop(just_might_stop)" returns False.

It turns out that whether will_stop(just_might_stop) returns True or False, it still leads to a contradiction. Hence, it turns out that it's not possible to implement will_stop.

Fun, huh? It's yet another version of "this statement is false".

Scheme: My Y Combinator

I've been reading The Little Schemer, and it posed an interesting question: how can you create a recursive function without having the ability to "define" a name for it? For instance, in Scheme, how can you create a simple, recursive function to calculate the length of a list without having the ability to use "define"?

Here's my approach:
((lambda (my-length l)
(my-length my-length l))
(lambda (my-length l)
((null? l) 0)
(else (add1 (my-length my-length (cdr l))))))
'(1 2 3 4 5))
The outer function is "(lambda (my-length l) ...)". It takes a reference to a function that it calls my-length. It calls that function "(my-length my-length l)". Hence, that function, which I call "my-length", receives the list as well as a reference to itself, "(lambda (my-length l) ...)". Since it receives a reference to itself, it's able to call itself recursively.

It turns out the real answer is called the Y Combinator. It's written as:
Y = λf·(λx·f (x x)) (λx·f (x x))
It was created by Haskell Curry. To use the Y Combinator to solve the above problem, you'd write:
((lambda (le)
((lambda (mk-length)
(mk-length mk-length))
(lambda (mk-length)
(le (lambda (x)
((mk-length mk-length) x))))))
(lambda (length)
(lambda (l)
((null? l) 0)
(else (add1 (length (cdr l)))))))
Notice that my version is shorter, but the Y Combinator version is more elegant because the actual length function isn't burdened with having to recursively pass a reference to itself.

If you didn't understand any of the above, don't sweat it. I read it twice, and I still barely get it ;) However, I think I now understand why Paul Graham used that name for his startup incubator, Y Combinator. Paul Graham is a Lisp guy, and his Y Combinator is a startup meant to recursively launch other startups ;)

Wednesday, May 05, 2010

Humor: An Environmentally-friendly Desktop

In an effort to be more environmentally conscious, I've decided to make my desktop more green. And although my sympathies lie with California's Central Valley farmers, I've also decided to show my support of the delta smelt by switching to a shell called "fish".

Do I think my actions will solve global warming? No--but it's important for each of us to do our part!

Tuesday, May 04, 2010

Linux: fish: The Friendly, Interactive Shell

I'm trying out a new shell called fish. Here are some screen shots and here is a great article to help you get started.

Usually, I'm a zsh user. fish aims to be as powerful as, say, zsh, but a heck of a lot easier to use and a lot easier to learn. So far, that's proven true. The shell provides intelligent tab completion, syntax highlighting, helpful error reporting, integrated documentation, etc.

Fish is inspired by Bourne shell syntax, but is not compatible with it. Specifically, a lot of ugly things have been cleaned up and made more regular. I do think that the syntax is nicer, although it takes a while to get the hang of if you're already a shell expert. About the only inconvenient part of switching to fish is that I can no longer copy and paste complicated bash one-liners from various places online.

The documentation is excellent. However, you might find this cheat sheet helpful for getting started:
: Get help with using fish.
Searching through history:
: Type in the search string, and press the up arrow.
: Open a file using the proper application.
if true; echo hello; end
for i in a b c; echo $i; end
switch $you; case '*'; echo hi; end
function hi; echo hello; end
rm $file:
: You don't have to quote $file, even if it has spaces.
: Double quotes do variable substitution. Single quotes don't.
: Nesting doubles in singles or singles in doubles doesn't hurt
: anything. Both forms allow a minimal escaping, such as \', \\,
: or \$.
set smurf blue
: This is variable assignment. They didn't use the "=" syntax
: because in fish, *everything* is a command. Use -e to erase a
: variable, -l to set a variable locally, -g to set a variable
: globally, and -u to set a variable universally. Setting a
: variable universally applies to all fish sessions and will
: even survive a reboot. Use -x when setting a variable to export
: the value of that variable. It is convention that exported
: variables are in uppercase and unexported variables are in
: lowercase.
echo (ls)
: Use parenthesis for subshells.
f1; and f2
: This is like f1 && f2 in bash.
ls **.as
: Find all the .as files recursively.
: Redirect standard error to a file. ^^ is used for appending to
: a file.
function ll; ls -l $argv; end
: This is the replacement for aliasing and "$@".
: This is $! in bash.
: This is the directory for all your functions. They must use a
: .fish extension.
echo input.{c,h,txt}
: Echoes: input.c input.h input.txt
echo {$USER}meister
for i in (seq 10); echo $i; end
: Counts to 10. Actually, so does "seq 10" ;)
set my_array a b c; echo $my_array[1]
: Prints "a". Note, arrays are 1-based, just like seq.
count $my_array
: Returns the number of items in the array.
for i in (seq (count $my_array)); echo $my_array[$i]; end
: Here's how to loop over an array.
for i in $PATH; echo $i; end
: This is the easier way to loop over an array.
The startup file is ~/.config/fish/config.fish.
: Quickly lists the current directory, or the directory your
: cursor is over.
: Adds "| less". Think "pager".
Fish supports even more Emacs-like commands--even a killring.

Wednesday, April 21, 2010

Ruby: An Introduction to Behavioral Driven Development with RSpec and Cucumber: Version 2

Tonight, I'll be giving an "encore" presentation of my talk An Introduction to Behavioral Driven Development with RSpec and Cucumber to the San Francisco Ruby Meetup Group. This is an introduction to behavioral driven development in Rails using Cucumber, RSpec, Webrat, and factory_girl.

Happy testing! :-D

Thursday, April 15, 2010

Ruby: Modifying Class Attributes in Cucumber and RSpec

Imagine you're building Amazon.com, and you have a rating system. You might not want to show an average rating to the user until there's been some minimum number of ratings. Of course, you'll want to test that that feature works. However, in other tests, you might want to disable that feature by setting the minimum number of ratings required to 0.

More generally, I sometimes run into the situation where I have a constant in my model that I might want to change, but only for a specific test. Instead of using a constant, you can use a class attribute. However, since class attributes aren't part of a database transaction, they don't get reset after every test. That's a problem since one test that changes the class attribute might mysteriously break another test that relies on that class attribute. I had a problem where one feature file would pass when run alone, but it wouldn't pass when all the features files were run. Here's how to make it all work.

In your Rating model, create your class attribute as follows:
# Rating.min_num_ratings is the minimum number of ratings before
# we show the ratings average to the user. We don't want a single bad
# rating to forever scar a book's reputation.
# I'm setting this up as a class accessor so that you can set it to 0 in your
# tests. Beware that Cucumber and RSpec don't reset this after every test
# since this isn't part of a database transaction. Hence, if you change
# Rating.min_num_ratings, you must change it back after your test. In
# RSpec, you can do this with a begin/ensure block. In Cucumber, you can use
# the @afterwards_reset_min_num_ratings annotation.
cattr_accessor :min_num_ratings
self.min_num_ratings = DEFAULT_MIN_NUM_RATINGS
Your RSpec tests should look like:
it "should do something useful when the minimum number of ratings has been hit" do
Rating.min_num_ratings = 2
# Test that it does something useful.
Rating.min_num_ratings = Rating::DEFAULT_MIN_NUM_RATINGS
Just to be sure I didn't forget to reset min_num_ratings anywhere, I have the following at the bottom of my .spec file:
it "should still have the default min_num_ratings" do
Rating.min_num_ratings.should == Rating::DEFAULT_MIN_NUM_RATINGS
Cucumber is a bit harder since you can't use a begin/ensure block. I started by adding a step:
# If you use this step, you *must* use the @afterwards_reset_min_num_ratings
# annotation on your scenario in order to reset Rating.min_num_ratings after
# your test.
Given /^the min_num_ratings has been set to (\d+)$/ do |min|
Rating.min_num_ratings = min.to_i
Then, I made use of the step and the annotation in my tests:
Scenario: users can vote, and their vote affects the average
Given there is only one book
And the min_num_ratings has been set to 2
And a user has already given the book 2.0 stars
And I am logged in as a user
When I follow "Moby Dick"
And I choose by xpath "//input[@name='rating' and @value='1.0']"
And I press "Rate"
Then I should see "Moby Dick"
And I should see "Thanks for rating!"
And the CSS class for the flash message should be "success"
And I should see an average rating of 1.5 stars
Last of all, I created a new file, features/support/hooks.rb, with the following:
# This will only run after scenarios tagged with
# @afterwards_reset_min_num_ratings.
After("@afterwards_reset_min_num_ratings") do
Rating.min_num_ratings = Rating::DEFAULT_MIN_NUM_RATINGS
I know this is a bit esoteric, but this same approach can be used anytime you have just a few tests that require test-specific configuration.

Tuesday, April 13, 2010

ActionScript: Dynamic Streaming

There's an article here about dynamic streaming in Flash Media Server 3.5 using Flash 10's new QoS features. It's a three-part series, and I just spent several hours reading it.

It talks about a class that Adobe provides called "DynamicStream". It doesn't come with ActionScript 3.0--it's a separate download. I was hoping to integrate this class into JW Player in order to do more intelligent bitrate switching using Adobe's new QoS features.

Let me explain why I'm frustrated.

First of all, this article was a three-part series, each with several pages. Hence, it was very long--much longer than I think was necessary. Part 3 covered "Integrating dynamic streaming with existing video players". Since it was the final page in the entire series, I expected it to be very exciting and useful. Instead, it was very mediocre. I can summarize, "Hey, this stuff doesn't work in Flash 9, so you'll have to do something else to support older players. Here, read this long code sample to learn how to parse the Flash player version number." There's a lot of code like:
private function _verifyServerVersion( p_version:String ):void
var fmsVersion:Number = Number( p_version.split(",", 2).join(".") );

trace("fmsVersion: " + fmsVersion);

if ( fmsVersion >= _targetFMSVersion )
_isServerCapable = true;
_isServerCapable = false;
At the very least, that if/else could have been written:
_isServerCapable = (fmsVersion >= _targetFMSVersion);
However, I think it's a flaw in ActionScript that you have to parse the version numbers manually at all.

Next up, I looked at the DynamicStream class, and it had multiple different coding styles:
if (...) {
if(...) { ... }
When I see stuff like that, I lose my confidence in the code. The file is 1049 lines long, by the way.

Having said all that, what really frustrates me is that the code doesn't take into consideration the widths for the various encodings. Currently, JW Player will pick the best encoding based on width and bitrate. That way, you can have different encodings for different size players. This is important for several reasons. First of all, there's no use pulling down an HD video for a 480x320 player until the user goes full-screen. It eats up bits that could be going to quality instead of size. Furthermore, it's better to avoid scaling on the client since it looks better to scale only once during encoding.

Hence, I'm more than a little disappointed by the DynamicStream class, and I'm stuck wondering how the heck I'm going to pull the useful bits out of this 1049 line file in order to integrate them into JW Player. Ugh, painful.

Oh, did I mention Silverlight takes care of bitrate switching entirely automatically? Double ugh.

Wednesday, March 17, 2010

Science: Gravity as a Communications Mechanism?

Does gravity move at the speed of light? For instance, if I were able to change some energy into matter, how long would it take for other matter to begin to feel the attraction of the matter I created? I assume someone out there knows the answer.

If you could convert energy to matter and back again in an amazingly concise manner, and do it at a specific frequency, and if you could detect such changes in gravity at a specific frequency, you could conceivably use gravity as a communications mechanism. That's not likely to happen during my lifetime, but it does make for some interesting science fiction.

Saturday, March 06, 2010

Books: ActionScript 3.0: Visual QuickStart Guide

I finished reading ActionScript 3.0: Visual QuickStart Guide. All-in-all, it wasn't bad. I needed to learn ActionScript quickly, but not extremely deeply, and it helped me do that.

This is the first time I've read a visual quickstart guide. The book was only about 300 pages, and each page was cut in two columns. The inner columns have code, and the outer columns have prose. Hence, there's only about 150 pages of prose. The code tended to be very basic and very redundant. Unsurprisingly, the author was a non-software engineer who learned how to code in ActionScript. I sort of expected that. The book wasn't perfect, but I definitely recommend it if you're a software engineer who needs to learn ActionScript in a hurry.

That's enough for the book review. Now, let me say a few words about ActionScript. First of all, ActionScript 3.0 is a compiled language. ActionScript started life as ECMAScript, but by the time they hit 3.0, it had morphed into a pleasant mix of JavaScript and Java. It has static types like Java, but you can do a lot of the same tricks that you can do in JavaScript, such as treat objects as hashes and use closures. Like JavaScript, ActionScript revolves around event handling. Anyway, as I said, I think ActionScript 3.0 is pleasant.

Now, let me cover a few of the things that were interesting enough for me to write down while I was reading the book.

ActionScript 3.0 has three types of numbers: int, uint, and Number. Numbers are floats, and match the same number type that JavaScript has.

One way to think of Flash is as a programmable version of Photoshop optimized for animated gifs. That's pretty much how it started life. Over time, they added layers and timelines. These days, ActionScript is a full featured language. Early on in the book, I couldn't understand how this graphical environment and this programming language worked together. Now I understand that Adobe Flash CS 4 is basically something like a paint program with a timeline that you use to create movies. ActionScript is bolted onto the side of it so that you can control things programmatically. However, early on, the most important use of the programming language was simply to control the timeline of the movie. (I could be getting some of this wrong since I've only been coding in ActionScript for a few months.)

Using the open source Flex SDK, you can compile Flash from the command line. You can even shun Adobe Flash CS 4 entirely and code everything in Vim. Although there are rather nice Flash IDEs, I felt less out of my element by sticking to Vim. See my other three posts: Linux: Installing the Flex SDK, Vim: Editing ActionScript, and Linux: Installing the Debug Version of Flash on Ubuntu.

Whereas in Java you might write "FooBar fooBar = new FooBar();" in ActionScript you write "var fooBar:Foobar = Foobar();". I think I have a mild preference for ActionScript's syntax.

There is a widget hierarchy in ActionScript. As you go down the hierarchy, the widgets gain more behavior such as being able to accept events or being able to contain other widgets. MovieClip is pretty far down in the hierarchy and is used for almost everything. MovieClip doesn't actually have anything to do with playing videos. It's just a widget that can handle events, contain other widgets, and live in its own timeline.

In C, to cast an int, you might write "(int) n", but in ActionScript, you write "Number(n)".

TextFields in ActionScript do support HTML, but its very weak. They also support CSS, but that's pretty weak too.

"someArray.forEach(someFunction);" works.

Just like JavaScript, you can write 'if (typeof(s) == "string")'. However, to test what type of object you have, you can also write 'if (someObj is MovieClip)'.

Use "for (var i:String in someObj) { trace(someObj[i]); }" to loop over the indexes of an object.

Use "for each (var value in someObj) { trace(value); }" to loop over the values of an object.

Use "navigateToURL" (mentioned on p. 205) to make the browser go to a new web page.

The way you dynamically load data from a web site varies wildly depending on whether you're loading sound, a DisplayObject, or data. I'm pretty sure these things were built by different engineers at different times.

In summary, if you're familiar with JavaScript and Java, ActionScript will seem extremely straightforward. I managed to code quite a bit before picking up a book at all.

Monday, March 01, 2010

Python: Is there a Python Toolbox Website?

Is there something like Ruby Toolbox for Python? Unlike PyPI, Ruby Toolbox feels highly curated, which I really, really like. Notice it even has graphs to show you the relative popularities of the various packages.

Half the reason I go to PyCon is so that I can know which libraries are good, and which ones aren't so good. Ruby Toolbox helps me cut to the chase, so it'd be great to find something like that in the Python world too.

Friday, February 26, 2010

Ruby: Interesting Things from the Rails Book

Here are a few interesting things I stumbled across while reading the Rails book recently:

Use debug(object) to dump an object to the screen in a template.

Here's how to use the debugger to debug your web app. First, "gem install ruby-debug". Then, start the server via "./script/server --debugger". Now, add "debugger" in your code.

Use "rake --tasks" to see the full list of rake tasks available.

Use "rake --describe task" to describe a particular task.

David Heinemeier Hansson said this on page 261, "Learn to cope with the panic attacks of unexplained errors."

To create a new, struct-like class, use "Rating = Struct.new(:name, :rating)".

ActiveSupport adds a ton of useful methods to the String class that alleviate some of the pain associated with Ruby strings. (I like the way Python treats characters and string slices a bit better.) For instance, it adds .from and .to for slicing.

Use "config.active_record.schema_format = :sql" to configure Rails to dump the schema to schema.rb in SQL instead of Ruby.

Use "config.action_view.field_error_proc" to configure how Rails outputs fields when there is an error.