Monday, November 30, 2009

Linux: Help, my Mouse is Possessed!

I have a MacBook running Ubuntu 9.04. Every once in a while, the mouse moves by itself. I always assumed it was my palm grazing the trackpad. If you have VertEdgeScroll turned on (which is the default), it's really easy for your right hand to touch the right side of the trackpad and cause your window to scroll. I figured OS X was just better at palm detection.

However, I recently bought a really nice external monitor, keyboard, and mouse. I was sitting reading some code. I was nowhere near my trackpad. In fact, I wasn't even typing or moving the mouse, although my hand was resting gently on my mouse. (If you're in the mood for a good conspiracy theory, keep in mind that it's a Microsoft mouse and an Apple keyboard.) All of a sudden, my mouse started moving. It's not a bluetooth mouse. None of my friends are around to be playing tricks on me (that I know of). It moved three inches right, and then down and to the left, and then all the way to the right. What the heck?!?

Web: Redirecting the User Back After Some Workflow

Here's a common Web application workflow. A user gets to a page. He doesn't have access to the page because he isn't logged in. He gets redirected to the login page. He logs in. Perhaps he even has to create an account first. Afterwards, he gets redirected to the page he was trying to go to in the first place. Users find this sort of behavior helpful.

Here's another. The user is trying to buy something. He gets redirected to Google checkout (disclaimer: I've never used Google checkout). After he finishes checking out, he gets redirected back to the origin site. Perhaps he now has access to download the thing he just bought.

Thanks to the authlogic tutorial, I already had code to take care of the login workflow. However, I now find myself needing to do more workflows like it. The user tries to go to a page. However, he must do this other thing. Afterwards, he can go to the original page. What's the best way to handle the problem generically?

Here are the constraints that come to mind. I don't want two workflows stepping on each others toes. (I'm not even sure workflows have toes!) Hence, if one workflow leads to a page, and then the user has to do another workflow, he should get redirected at the end of the inner workflow and then get redirected at the end of the outer workflow. However, I don't expect that the user will need to recurse. That is, I don't expect one workflow to lead to itself again in a recursive sort of way.

With all that in mind, I took what authlogic gave me and expanded on it. In my base class, ApplicationController, I have:

def store_location(session_key_suffix, url=request.request_uri)
session[session_key_for_return_to(session_key_suffix)] = url

def redirect_back_or_default(session_key_suffix, default)
key = session_key_for_return_to(session_key_suffix)
redirect_to(session[key] || default)
session[key] = nil

def location_stored?(session_key_suffix)


def session_key_for_return_to(session_key_suffix)
Here's how to use it to handle the login case:
  def require_user
unless current_user
flash[:error_message] = "You must be logged in to access this page"
redirect_to new_user_session_url
return false
After the user logs in, I call:
  redirect_back_or_default(:after_login, root_url)
Based on my tests, the above works. It can handle nested workflows. It can even handle a user working on two different workflows in two different tabs. I can envision some esoteric ways to mess the above code up, but I'm going to give it a couple weeks and see how it turns out in practice.

Saturday, November 28, 2009

ActionScript: Dates

I had to parse an RFC 2822 date today in ActionScript. I couldn't find any equivalent of strptime in order to allow me to parse dates in a variety of formats. It turns out that ActionScript can do it automatically:
new Date("Sat, 28 Nov 2009 10:15:02 +0000");
The Date constructor will attempt to guess at the correct format, and it'll raise an exception if it can't.

The next problem I had was that I was using Date.getTime() to return the number of milliseconds since midnight January 1, 1970, universal time. Even though getUTCYear() said that my Date object was for 2009, I kept getting a value somewhere around January 12, 1970. The value returned by getTime() was just too small!

I finally figured out that getTime() returns a Number, not an int. If you store the value in an int, you'll be completely hosed, and there aren't even any warnings to help you out. It's obvious in retrospect--a 32-bit int can't possibly hold the number of milliseconds since 1970 given a date in the year 2009.

Wednesday, November 25, 2009

ActionScript: Flowplayer vs. Open Source Media Framework

I've been playing around with Flowplayer, which is an "Open Source (GPL 3) video player for the Web". I decided to check out the Open Source Media Framework (formerly known as Strobe), which "enables developers to easily assemble pluggable components to create high-quality, full-featured playback experiences." In short, if you need a Flash video player today, stick with Flowplayer. The OSMF just isn't ready for prime time. It's a large framework written by a large company (Adobe). It's not a simple, hackable video player that you can throw on a production website in a couple of hours.

The website reeks of corporate speak. For instance, here is part of the page that talks about plugins:
One the primary goals for OSMF is to provide a set of standard APIs for video ecosystem service integration so that player integration is no longer a barrier in the market. The framework will have a plugin API that allows for integration of client-side code for the full range of ecosystem services, including media delivery, media composition for advertising, event reporting for audience measurement, and other rich experiences. The plugin API is designed to support dynamic plugins as SWFs and integrated plugins as SWCs in a way that makes it straight forward for the same provider to support both scenarios.
When I read the document for writing a plugin, it sounded more like a functional spec than real documentation:
Summary of Features: User Story: As a plugin provider, I need examples for how to create a media plugin and ideally a basic composition plugin. I also need the ability to specify when a specific version of my plugin works with a specific version of the Strobe framework. If a player tries to load a version of my plugin that the player won't support, the framework should be aware and pass info through for graceful error handling. I also want to be able to support multiple versions of my plugin.
I clicked to read the Developer Documentation, and the only thing they had was "On-line API Documentation". That means they ran a tool like JavaDoc to pull the comments out of their source code. In fact, the tool mixes the API documentation in with the normal ActionScript API docs:
Overview of the ActionScript 3.0 Language Reference
including the Open Source Media Framework API
The one thing the OSMF has going for it is that the source code is very regular and clean. It's large and Java-esque, but at least the style and comments look regular. Flowplayer's code isn't bad, but I wouldn't call it "polished".

The OSMF requires Flash 10, whereas I was hoping to stick with Flash 9 since the adoption rate is higher. Here's a comment from another poster:
While I am sure OSMF can do well with small content media companies, I find it hard to believe that large media content companies will join before flash player 10 penetration is about 96% - 98%. The reason OSMF compile for FP10 is that OVP uses some of the new features in FP10 to do dynamic switching etc, it can be modified so OVP doesn’t uses the FP10 capabilities and will position the framework better.
Based on looking at this blog post, the OSMF doesn't even provide default player controls. That's a heck of a lot of code to not even provide player controls!

I don't know a lot about ActionScript, but I do know a lot about open source. Open source projects tend to tend to go nowhere unless they address the following:
  1. The code must be easy to build and do something useful by default.
  2. If it's a library, the documentation must make it very easy to get started. Framework API docs don't count.
  3. The code must be stable. (I haven't run the OSMF, so I can't comment on that.)
  4. It must be easy to hack the code to accomplish what you want.
  5. It must be easy to contribute back to the project. (I can't even look at the existing OSMF bugs without creating an account.)
Frameworks start out life with 2 strikes against them because they don't do anything useful by default, and they take a lot of time to understand. In contrast, applications (like Emacs or Firefox) do do something useful by default, but are still straightforward to customize.

In summary: It requires Flash 10. It doesn't have player controls by default. It's written like a massive Java framework. It lacks good documentation to help you get started. It does have nice source code, but at this point, it's simply not a viable alternative to Flowplayer.

Tuesday, November 24, 2009

Linux: Convert Mac to UNIX Line Endings

I'm working with a project that has a bunch of ActionScript (.as) files with old Mac line endings (i.e. CR instead of LF). For some reason, I see a bunch of references to mac2unix online, but I don't see anything readily available in Debian packages.

I want to convert to UNIX line endings, but without screwing up DOS files, etc. That is, I want it to be somewhat intelligent. This is what I came up with. It finds all the ActionScript files, determines if they're using just CRs, and then converts them to UNIX format:
for i in $(find . -name '*.as' -exec file '{}' \; |
grep 'with CR line terminators' |
sed 's/:.*//g'); do
echo $i;
perl -pi -e 's/\r/\n/g;' "$i";
By the way, yes, that'll fit on one line ;)

Saturday, November 14, 2009

Linux: Making Your Desktop Calm

Have you ever noticed that the more you tweak your desktop theme, the more it drives you crazy? Maybe it's just me--I'm pretty compulsive obsessive. For most of my career, I've tried to create an interesting, exciting, cool-looking desktop. I know that a lot of you are probably saying, "Duh!" right about now, but I figured out I was actually more productive when I made my desktop more boring.

I've heard people say that if you paint a child's room bright yellow, it makes them more hyperactive. I've also heard people say that if you paint a prisoner's cell soft pink, he'll be more sedated. I'm not sure if those things are true, but they seem reasonable.

I've always sought after the perfect wallpaper. However, the cooler the wallpaper is, the more it drives me crazy. For some reason, I've always found the default wallpaper in OS X to be very calming (with the exception of OS 10.5).

Recently, while using Ubuntu, I switched to having no wallpaper at all. It was initially an accident, but I realized I liked it. I just stuck with the weird tan color that is the default if you turn the wallpaper off. After a few hours, I realized it had a calming effect on me--which is important since I'm so compulsive obsessive.

Recently, I noticed my designer buddy Albert had a plain gray background on his MacBook Pro. I asked him why he had chosen not to use a wallpaper. He said that it was important for his desktop to be neutral. That makes sense that you wouldn't want your desktop to conflict with the design you were creating. I asked him which color gray he was using. It turns out, he wasn't using a plain gray, but instead he had added just a hint of red to it. He said it made the gray more "warm".

"Calm" and "warm" are two words that I had never even thought of when trying to describe the perfect desktop. I'm curious, has anyone else out there figured out a desktop that they find "calm" and "warm"? By the way, the background that I'm currently using is "#636161" which is a bit darker than what Albert originally picked, "#978989".

Linux: My xmonad Setup

Since I already blogged about why I think xmonad is interesting, I thought I would take the time to blog about my xmonad setup. I'd like to thank arjuna del toso for his instructions, because that's how I got started.

Start by installing all the software:
apt-get install xmonad libghc6-xmonad-contrib-dev libghc6-xmonad-dev dwm-tools feh
Now, edit /usr/share/xsessions/xmonad.desktop so that it executes "xmonad.start" instead of "xmonad".

Then create /usr/local/bin/xmonad.start:

xrdb -merge .Xresources

gnome-panel &
syndaemon -d -t

# feh --bg-scale /usr/share/backgrounds/warty-final-ubuntu.png &
# xsetroot -solid "#978989" # Lighter gray.
xsetroot -solid "#636161" # Darker gray.

# This must be started before seahorse-daemon.
eval $(gnome-keyring-daemon)

# This is all the stuff I found in "Startup Applications".
/usr/lib/gnome-session/helpers/at-spi-registryd-wrapper &
bluetooth-applet &
sh -c 'test -e /var/cache/jockey/check || exec jockey-gtk --check 60' &
# sh -c "sleep 30; exec /usr/lib/evolution/2.26/evolution-alarm-notify"
# gnome-keyring-daemon --start
# /usr/bin/canberra-gtk-play --id="desktop-login" --description="GNOME Login"
# /usr/lib/gnome-session/helpers/gnome-session-splash
sh -c "sleep 60 && python /usr/share/gnome-panel/" &
nm-applet --sm-disable &
sh -c "sleep 1 && gnome-power-manager" &
sh -c "sleep 31 && system-config-printer-applet > /dev/null 2> /dev/null" &
# /usr/lib/vino/vino-server &
update-notifier --startup-delay=60 &
# gnome-at-visual -s

exec xmonad
You might wonder how I came up with all of that. I started with the xmonad documentation. Then, I opened up "Startup Applications" when logged into my GNOME desktop in order to figure out what I wanted to recreate. Some of the things I decided I didn't need. Some of the things had to be tweaked to run outside the context of a normal Ubuntu environment.

Make the script executable:
chmod +x /usr/local/bin/xmonad.start
Now, create ~/.xmonad/xmonad.hs:
import XMonad hiding (Tall)
import XMonad.Hooks.ManageDocks
import XMonad.Hooks.SetWMName
import XMonad.Layout.Circle
import XMonad.Layout.Magnifier
import XMonad.Layout.HintedTile

myLayout = avoidStruts $ hintedTile Tall ||| hintedTile Wide ||| Circle
||| magnifier Circle ||| Full
-- hintedTile listens to application hints, so as not to break gVim.
hintedTile = HintedTile nmaster delta ratio TopLeft
nmaster = 1
delta = 3/100
ratio = 1/2

myManageHook = composeAll
[ className =? "Gimp" --> doFloat ]

main = do
xmonad $ defaultConfig
{ manageHook = manageDocks <+> myManageHook <+> manageHook defaultConfig

-- I used to use: avoidStruts $ layoutHook defaultConfig
, layoutHook = myLayout

-- This hack is necessary to make Java GUIs like NetBeans work. See the FAQ.
, logHook = setWMName "LG3D"
To test that you got the syntax right:
ghci ~/.xmonad/xmonad.hs
Note that I decided not to use xmobar, which is a text-based status bar. It's just not my thing, although you might want to read arjuna del toso's instructions and give it a shot.

Last of all, there was one more incredibly esoteric hack that was necessary. If you using xmonad + NetBeans + jVi, then using ":w" to save your file will cause NetBeans to lock up. Weird, I know. The workaround is to edit your jVi preferences to tell jVi not to handle Control-s, but rather to pass it to the IDE. That way, you can use Control-s to save your file instead of using ":w".

Make sure to read the guided tour and check the xmonad FAQ if you have any issues.

What Makes a Good Operating System?

There are a lot of good operating systems these days. How do you pick one? If you're going to pick Linux, how do you pick which distribution to use? Here are the criteria that are important to me:
  • It must support my hardware. It doesn't matter if it's the best operating system in the world, if I can't run it on my hardware, it does me no good. An operating system that doesn't support my wireless card just isn't a viable alternative.
  • It must be painless to install the software I want and uninstall it when I don't want it anymore. I like keeping a clean, "repeatable" system, so building software by hand without being able to uninstall it is something I avoid whenever possible.
  • It must be easy to update. I like to apply security patches as soon as they come out, and I like running modern versions of all my favorite software. Installing a new kernel in production under Ubuntu is trivial. Running "make world" in production isn't.
  • It must be FOSS, because that's the future I want to see. Being "mostly" open source like OpenSolaris just doesn't cut it for me.
Criteria involve trade offs. You probably have your own criteria, but these are the criteria that have lead me to be such a fan of Ubuntu.

Linux: xmonad

xmonad is a tiling window manager. If you don't know what I'm talking about, take a peek at one of the screencasts. I've been using xmonad for the last couple weeks. It's been a couple years since I tried it last, and it's really improved:
  • It's now a lot easier to install on Ubuntu.
  • It's now a lot easier to integrate with panels such as gnome-panel or xmobar.
  • It's now a lot easier to try out various layouts, and there are more layouts to choose from.
Every time I try out a tiling window manager, I am reminded of the fact that I fundamentally disagree with the premise:
  • They think that maximizing a window as much as possible whenever possible is useful. I think that's true with terminals and chat windows, but less so for many other windows. For instance, I always want GVim to be 80 columns wide.
  • They think that minimizing windows to very small sizes is more acceptable than allowing windows to overlap. I disagree.
  • They think that forcibly resizing windows won't break them. Unfortunately, forcibly resizing windows makes the Gimp look terrible, and it often cuts off the last line of text in GVim.
  • They think that I don't care about having an integrated desktop environment such as the one GNOME provides. In reality, I like the idea of using weird window managers as part of my GNOME desktop. I really like all those things that Ubuntu puts on my gnome-panel such as the network manager, update manager, mixer, etc.
Fortunately, of all the tiling window managers, xmonad is the most understanding of my needs:
  • It works with the gnome-panel.
  • It has layout managers that can take hints from applications so as not to chop off the last line of text in GVim.
  • It can create exceptions for certain applications, such as the Gimp, placing them in a "floating" layer that is more suitable for such applications.
  • It has layouts that allow windows to overlap in useful ways.
Hence, xmonad is my favorite among all the tiling window managers that I've tried. There are a couple premises that I fundamentally agree with:
  • Having the computer help you manage your windows using smart algorithms is a good idea.
  • Trying bold, new ideas in user interface design is a great idea.
That last point is important. There's a great saying in The Myths of Innovation: "Don't worry about people stealing your ideas. If they're any good, you'll have to cram them down people's throats!" Not every new idea is good, but trying out lots of new ideas is very good.

xmonad exemplifies what I'd like to see more of in the Linux world. Linux has always been a little behind in terms of user interface design. Instead of cloning Windows or OS X, I really like the idea of creating our own new, interesting, and clever approaches to user interface design. Do you remember the time when neither Windows nor OS X had virtual desktops? I like having things that those guys don't have, which is why I'm happy to see the innovation that xmonad is providing.

Linux: Minimizing Memory Usage

It's still fashionable in certain Linux circles these days to pick a leaner distro in order to minimize resource utilization. Advocates of Arch Linux and Gentoo like to point out that Ubuntu is a bit heavy on RAM usage. As a Linux old timer, I'm sympathetic to the yearnings for a leaner, meaner past. Even Linus Torvalds admits that Linux is getting a bit bloated these days.

However, does switching to something like Arch Linux or Xubuntu really help? Consider the fact that I have a dual core MacBook with two gigs of RAM. It'd be one thing if I had an ancient machine, but I actually have relatively beefy hardware. Also, consider what top says when I sort by RAM usage:
top - 03:32:53 up 10 days,  4:03,  3 users,  load average: 0.37, 0.37, 0.41
Tasks: 154 total, 2 running, 149 sleeping, 0 stopped, 3 zombie
Cpu(s): 12.9%us, 5.4%sy, 0.0%ni, 81.5%id, 0.0%wa, 0.2%hi, 0.0%si, 0.0%st
Mem: 2026720k total, 1834468k used, 192252k free, 189524k buffers
Swap: 3559984k total, 36252k used, 3523732k free, 710924k cached

29545 jj 20 0 591m 310m 31m S 21 15.7 55:11.01 firefox
31458 root 20 0 972m 216m 58m S 8 10.9 86:57.34 Xorg
9677 jj 20 0 762m 118m 19m S 0 6.0 0:17.02 java
32600 jj 20 0 248m 55m 18m S 4 2.8 10:54.55 skype
32602 jj 20 0 109m 44m 20m S 0 2.3 7:03.57 pidgin
31311 jj 20 0 49784 36m 3468 S 1 1.8 1:48.45 ruby
1129 jj 20 0 54408 27m 13m S 1 1.4 4:27.88 gvim
31600 jj 20 0 34152 22m 8100 S 0 1.2 0:47.06 notify-osd
31563 jj 20 0 37368 20m 13m S 0 1.0 0:30.85 gnome-panel
1119 jj 20 0 55848 19m 10m R 2 1.0 1:13.64 gnome-terminal
31644 jj 20 0 73664 16m 10m S 0 0.8 0:08.64 gnome-power-man
31627 jj 20 0 46588 16m 11m S 0 0.8 0:00.74 mixer_applet2
31632 jj 20 0 29372 16m 8472 S 0 0.8 2:03.24 indicator-apple
31588 jj 20 0 26880 15m 9.8m S 0 0.8 0:09.21 nm-applet
31602 jj 20 0 28128 14m 10m S 0 0.7 0:03.17 update-notifier
31624 jj 20 0 27528 14m 10m S 0 0.7 0:07.55 fast-user-switcher
Firefox is eating up 310MB of resident memory. Oops! Time to restart it! NetBeans is eating up a whopping 762MB of virtual memory, although only 118MB of it are resident. Even GVim is using 27MB of resident memory. I have no clue why X is taking up so much memory. Despite all that, I still have massive amounts of memory free for the OS to dedicate to buffers and cache.

The fact of the matter is I need Firefox with Firebug, etc. in order to do my job, and using NetBeans (with the jVi plugin) makes me more productive than using GVim or Emacs alone. Remember the days when people joked that Emacs stood for "Eight Megs And Constantly Swapping"? It's hard to imagine an integrated development environment taking up only 8MB. Even my terminal eats more than that these days!

Hence, my point is that I could switch to Arch Linux, but that's optimizing the wrong thing. NetBeans and Firefox are the ones that eat the most RAM.

Tuesday, November 10, 2009

Python: Concurrency

With all the excitement surrounding Tornado Web Server, I'd like to mention that I wrote a great article last year on Python Concurrency, with an emphasis on the various approaches to writing Web servers.

By the way, has anyone played with gevent or Eventlet? Like the proprietary version of stackless Python we used at IronPort, they offer asynchronous networking without the need to use continuation passing style.

Happy Hacking!

Friday, November 06, 2009

My Friend Committed Suicide

He was a brilliant programmer, and he had everything going for him. He was very successful.

I'm crushed because I know I could have helped if only he had given me a chance. He never did.

We in the programming world aren't always the most emotionally balanced. I know of three others who took their lives in the programming world. I've hinted at this before on my Bipolar Lisp Programmer post. To compound matters, our society has been moving away from personal interaction and responsibility for decades, leading to a culture that is toxic.

Mother Theresa said that the greatest poverty that she ever saw was to see people who felt unloved. If your friends are feeling unloved, please reach out to them. We are each far more loved than we think. In the programming world, it's so easy to get caught up in petty struggles, like Pylons vs. Django, Ruby vs. Python, free software vs. open source, Linux vs. pretty much everything else ;) What we forget is that we're all people with hopes and dreams, fears and insecurities. We're all trying to change the world, but as Mother Theresa said, "In this life we cannot do great things. We can only do small things with great love."

Look out for each other, but if you have no where to turn, contact me! Call me directly at (925) 209-6439. Don't consider the middle of the night an inconvenience. When else am I going to work with five kids around ;) I will do everything I can to help. I don't want to wake up on another day to find another brilliant mind missing from this great community of thinkers. We are a community and we are only as great as the individuals that thrive and share with one another in that community. Everyone contributes, whether in small ways or large, and every loss is felt by more people than we consider in our moments of darkness.