Tuesday, December 11, 2012

New Dart Web UI Codelab

Originally posted on Dart News & Updates:

If you've been waiting for the right time to learn how to use Web Components in Dart, now's your chance! Continuing where we left off with the Google IO 2012 Dart Codelab, I've created a Dart Web UI Codelab which focuses on Web Components and the new Web UI library.

In this codelab, I cover:

  • How to set up pub
  • How to use Web Components
  • How to use dynamic templates and two-way data binding (inspired by Model-driven Views)
  • How to build an application with multiple Web Components
  • Where to get more information about Web Components and the Dart Web UI package
  • What to do if you get stuck while trying to build an app using the Dart Web UI package

We've already created videos, blog posts, and articles about how to use Web Components in Dart, but this codelab is pretty exciting in that it walks you through building a complete, albeit small, application using Web Components and the Dart Web UI library. So give it a try and tell us what you think!

We invite you to join the Dart Web UI mailing list, ask questions on Stack Overflow, or file feature requests on dartbug.com.

Monday, December 10, 2012

An Approach to Illiterate Programming: doc-code-merge

Reposted from Dart News & Updates:

It's very common to embed code examples in technical documentation. However, doing so is kind of painful. First of all, the environment you use to edit the documentation may not be the same as the environment you use to write code. For instance, I generally use Dart Editor to write Dart code, but I use Sublime Text 2 to edit documentation written in HTML or Markdown. Secondly, it's difficult to run unit tests or use dart_analyzer if the code is trapped in documentation. This is a big problem for us on the Dart project because we move pretty quickly, and we want to make sure our documentation is always 100% correct. I've created a tool called doc-code-merge to solve this problem.

doc-code-merge is the opposite of what Donald Knuth labelled Literate Programming. doc-code-merge lets you write your documentation and your code examples completely separately. When you run doc_code_merge.dart, it makes a copy of your documentation with all the examples merged in. The Pragmatic Programmers mentioned using a similar tool when writing The Pragmatic Programmer: From Journeyman to Master, so the idea has probably been around for a while.

Using doc-code-merge is easy. Start by writing some documentation such as:

Then, write some code such as:

Now, run doc_code_merge.dart:

This results in:

Easy peasy! doc-code-merge automatically handles things like indenting code blocks when generating Markdown and HTML escaping code when generating HTML.

Although doc-code-merge is written in Dart, it'll work with almost any programming language and with almost any type of documentation as long as the documentation is written in text such as HTML or Markdown. We're hoping to make use of it for the Dart web site, dartlang.org, and for our book on Dart, Dart: Up and Running.

If you're interested in using doc-code-merge, check out the README. As always, we invite you to join the Dart mailing list and ask questions on Stack Overflow.

Friday, December 07, 2012

Dartisans Ep. 16: Dart and Web Components Reloaded

In this episode of Dartisans, Dimitri Glazkov (one of the godfathers of Web Components) gives a presentation on Web Components. Also, John Messerly and Siggi Cherem (who helped build the dart-lang/dart-web-components library) give a presentation on using Web Components in Dart. A lot of things have changed since our last episode focused on Web Components, and this was a must-see episode if you're interested in Web Components and/or Dart!

Tuesday, November 20, 2012

The Untouchables

There are some technologies that have such strong "street cred", they are effectively untouchable. Anyone who dares imply that they are deficient in any way is instantly labelled an uneducated moron. It is impossible to have a realistic conversation about these technologies, and it's not very common for enthusiasts of these technologies to admit their failings. Let me give some examples:

  • Linux is the best operating system.
  • Common Lisp is the best programming language ever invented.
  • Haskell code is shorter, more elegant, and more correct than code written in any other language.
  • Emacs is the best editor for real programmers.
  • Apple creates the most user friendly user-interfaces.
  • Tiling window managers lead to the highest degree of user productivity.
  • Ruby on Rails is the best web framework written in any language.
  • The only way to write good software that meets its users needs and is delivered on time is to embrace agile software development and use test-driven development.

Of course, there are problems with each of these things:

  • It can be very difficult to get hardware accelerated 3D, wireless drivers, and suspend working reliably in Linux depending on the hardware you have and which version of which distro you are using.
  • Common Lisp has a lot of historical baggage, and it lacks the breadth of community library support that some languages such as Python, Ruby, and C have.
  • Haskell code can be very terse, and advanced Haskell code can be difficult for even intermediate Haskell programmers to understand.
  • IntelliJ has many advantages over Emacs when it comes to editing Java.
  • Not everyone prefers Apple user interfaces. My wife always complains when I make her use OS X.
  • Many of the tiling window manager fans that I know spend an inordinate amount of time configuring and tweaking their window manager, presumably because it doesn't yet do exactly what they want.
  • Ruby on Rails isn't the best approach for real-time applications or applications that must have very, very low latency.
  • Agile software development is not a good fit when it is impossible to iterate, such as when you're building software that must be completely done and completely correct the first time it is used (e.g. pacemakers and satellite software).

There are also some technologies that have such negative street cred that it's difficult to praise them in certain circles. For instance:

  • It can be hard for a Linux advocate to admit that Microsoft has ever done anything good or innovative.
  • It can be difficult for a Python or Ruby enthusiast to admit that Java has any advantages whatsoever.

Linux, Common Lisp, Haskell, Emacs, Apple, tiling window managers, Ruby on Rails, agile, and TDD each have amazing amounts of street cred, and I've spent a lot of time learning playing with all of them. What I discovered is that the world isn't always so black-and-white. It's really helpful when people can honestly admit the weaknesses in technologies they like as well as the strengths in technologies they don't like. I'm not saying that all technologies are equally good. I'm just saying that it would help if we could be more realistic.

Saturday, November 17, 2012

Humor: Metamocking

There are a lot of very powerful concepts in the world of software engineering that have "meta" in their name. Just consider metaprogramming and metaclasses. I've spent some time thinking about what "metamocking" might look like. What would it mean to mock a mock?

First, some background. Let's say I have the following function (written in Dart):

void helloWorld() {
  print("Hello, World!");
}

How do I know if it works correctly? Since it doesn't return anything, but rather changes the state of the outside world (specifically, the stuff printed to my terminal), it's not so easy to test.

Mocking to the rescue! I need to add a bit of dependency injection so that the test can pass in its own code for printing and so that I can mock things properly:

library hello_world;

const defaultPrinter = const Printer();

class Printer {
  const Printer();
  printValue(String s) => print(s);
}

void helloWorld({Printer printer: defaultPrinter}) {
  printer.printValue("Hello, World!");
}

main() {
  helloWorld();
}

Now, let's write a unittest that makes use of mocking to test the helloWorld function:

import 'package:unittest/unittest.dart';

import 'hello_world.dart';

class PrinterSpy extends Mock implements Printer {
  PrinterSpy() {
    when(callsTo('printValue', "Hello, World!")).thenReturn(true);
  }
}

void main() {
  group('helloWorld', () {
    test('prints "Hello, world!"', () {
      var printerSpy = new PrinterSpy();
      helloWorld(printer: printerSpy);
      printerSpy.getLogs(callsTo('printValue'))
        .verify(happenedExactly(1))
        .verify(alwaysReturned(true));
    });
  });
}

Woah! That's a lot of code to test such a simple function! I feel pretty comfortable about "class PrinterSpy" because it's pretty short, but I'm really worried about the test itself. How do I know that the test itself is using printerSpy correctly?

Once again, mocking to the rescue! Let me start by pulling out the heart of the test into a separate function so that I can make use of dependency injection again:

import 'package:unittest/unittest.dart';

import 'hello_world.dart';

class PrinterSpy extends Mock implements Printer {
  PrinterSpy() {
    when(callsTo('printValue', "Hello, World!")).thenReturn(true);
  }
}

void testPrintsHelloWorld({PrinterSpy printerSpy}) {
  if (!?printerSpy) {  
    printerSpy = new PrinterSpy();
  }
  helloWorld(printer: printerSpy);
  printerSpy.getLogs(callsTo('printValue'))
    ..verify(happenedExactly(1))
    ..verify(alwaysReturned(true));
}

void main() {
  group('helloWorld', () {
    test('prints "Hello, world!"', () => testPrintsHelloWorld());
  });
}

Now, I can create a mock for my mock to make sure that I wrote it correctly. Unfortunately, this is fairly challenging since you can't use a Mock to mock a method named getLogs since Mock has its own definition of that function. Nonetheless:

import 'package:unittest/unittest.dart';

import 'hello_world.dart';

class PrinterSpy extends Mock implements Printer {
  PrinterSpy() {
    when(callsTo('printValue', "Hello, World!")).thenReturn(true);
  }
}

void testPrintsHelloWorld({PrinterSpy printerSpy}) {
  if (!?printerSpy) {  
    printerSpy = new PrinterSpy();
  }
  helloWorld(printer: printerSpy);
  printerSpy.getLogs(callsTo('printValue'))
    ..verify(happenedExactly(1))
    ..verify(alwaysReturned(true));
}

class PrinterSpySpy extends PrinterSpy {
  LogEntryList getLogs([CallMatcher logFilter,
                        Matcher actionMatcher,
                        bool destructive = false]) {
    log.add(new LogEntry(null, "getLogs",
        [logFilter, actionMatcher, destructive], null));
    return super.getLogs(logFilter, actionMatcher, destructive);
  }
}

void testTestPrintsHelloWorld() {
  var printerSpySpy = new PrinterSpySpy();
  testPrintsHelloWorld(printerSpy: printerSpySpy);
  printerSpySpy.getLogs(callsTo('getLogs')).verify(happenedExactly(2));

  // Verifying the number of times verify is called is left as an
  // exercise for the reader.
}

void main() {
  group('helloWorld', () {
    test('prints "Hello, world!"', () => testPrintsHelloWorld());
  });
  
  group('testPrintsHelloWorld', () {
    test('uses printerSpy correctly', () => testTestPrintsHelloWorld());      
  });
}

As you can see, I was able to verify that my mock executed correctly. Metamocking is certainly an interesting, albeit advanced technique for software engineers who consider themselves true experts in the art of software testing.

However, I should mention that it does have a couple flaws. For instance, who's to say that the mock for my mock is itself correct? In theory, running a mock should serve to test the mock, but one can never be too sure! Furthermore, despite the fact that I've written a lot of test code, I have no idea if helloWorld will truly print "Hello, World!" to my terminal since I never tested the actual print function. Mocking is like that sometimes ;)

At this point, I should probably wrap up. I'll finish with a proper definition of metamocking. The astute reader may have guessed by now that metamocking (i.e. mock mocking) actually has two meanings:

  • To create mock objects in order to test your mock objects.
  • To mock (i.e. poke fun at) those people who rely on mocking too much :)

My next blog post will be on metametamocking. In it, I plan on mocking this blog post and explaining that it was just a hopelessly bad and overly elaborate pun ;)

Friday, October 26, 2012

Dart: Lazily Evaluating Static Variables Leads to Wackiness

It is now possible in Dart to initialize top-level (i.e. static) variables using any expression. The expression gets evaluated lazily. When I first heard about this, my instant reaction was fear because laziness and imperative programming don't really get along. At the time, I hadn't seen the blog post where Gilad admitted that plainly.

Hence, I spent a couple hours that night trying to come up with ways this could lead to wackiness. I asked Gilad about it, and he said, "Laziness and statefulness can cause scary problems, I know. You can get into these Hesienbugs, where things only show up when you look, so as you debug the problem disappears."

Just to prove the point, I came up with the following Dart program:

import "dart:io";
import "dart:math";

var varA = changeVarB();
var varB = 1;

int changeVarB() {
  varB += 1;
  return 0;
}

main() {
  print("$varB");
  var seed = new Date.now().millisecondsSinceEpoch;
  var nextBool = new Random(seed).nextBool();
  if (nextBool) {
    varA;  // This looks like a no-op, right?
  }
  print("$varB");
  varA;
  print("$varB");  
}

Can you guess what this prints? Half the time, it prints 1, 1, 2. The other half the time, it prints 1, 2, 2. The first time I print varB, it's value is 1. However, the first time varA is evaluated, changeVarB() gets called (i.e. evaluated lazily). From then on, varB's value will be 2.

Of course, being able to initialize top-level variables to things that aren't constant is really useful, so I'm not sure I'd advocate changing this. Nonetheless, it's something I think you should be careful with.

Thursday, October 25, 2012

Dart: My Recent Dart Talks

I recently gave two talks on Dart, one at reflections projections|2012 at the University of Illinois at Urbana-Champaign and one at DevOps Live in Plano, Texas. Both of them were recorded. The DevOps Live talk is slightly newer since I updated it for the M1 release of Dart. If you've never seen one of my introduction to Dart talks, now's your chance!

reflections|projections 2012 talk

DevOps Live talk

Wednesday, October 24, 2012

Trip to "reflections projections|2012"

I gave a talk and a codelab on Dart at reflections projections|2012 at the University of Illinois at Urbana-Champaign. I can't possibly explain how well organized the conference was, how friendly the students and alumni were, or how much I enjoyed hanging out with everyone there! A big shout out to Kurtis, RJ, Metro, Zed, Danielle, Caitlyn, Ciara, etc.! Since I did manage to attend a few talks, let me blog about those. If you want, you can watch the videos.

Garry Tan from YCombinator

This was a talk on creating startups.

He worked at Microsoft. He co-founded Posterous.

"Do everything. Don't box yourself in[to a particular skillset]."

The more you know, the better you'll be able to guage the qualities of cofounders.

Studying entreprenuership != doing it.

Work is for learning.

If you plan on doing a startup, don't run out and buy a really nice, brand-new car. Money is capital.

Microsoft made $650,000 per year per employee while he was there.

Work is a transaction. You create the value. They give you a fraction of the value in exchange for stability...and a place to learn.

Be contrarian: Don't rely on third-hand accounts of what happened 6 months ago. (He criticized the media.)

All great ideas seem stupid initially.

He showed a Venn diagram where there's a sweet spot where something looks like a bad idea, but it's actually a good idea.

A great pitch is a self-fulfilling prophecy.

Everything sucks. Seek a problem, not technology for its own sake.

Ship a quantum of product: a discrete, indivisible manifestation of a product that solves a problem that people have.

When you're small, act small: Do unscalable things. Don't act bigger than you are.

When you're small, you can be human. Answer every email. Be personal. People notice.

reddit and justin.tv came out of the first YCombinator batch.

They're spawning companies that are worth billions.

They only focus on building software and getting users.

Dinner is the only thing that's required of YCombinator participants.

Every angel and VC in the valley comes out to watch Demo Day. People generally raise $300k to over a million.

To get PR, go to a publication, and email them, "We want to give you an exclusive. Do you want it? Yes or no?" Go after the most junior writer there.

He joined YCombinator because he got a little burnt out.

When you're starting, it takes all your working hours. However, you can only sustain that for 3-6 months. At some point, you have to take it easier. Flow makes the time go by really fast.

Getting other co-founders involved is important. If you can't get another extremely talented person to go into it with you, then maybe what you're doing is really is crazy.

At this point, YCombinator isn't about the money they give you. It's about the connections.

Mark Twain said, "If at first you don't succeed, try, try again. But then quit. There's no use being a damn fool about it."

The most frequent question he hears is "How do I hire?" He said to go after your friends. Vote on who you think is the smartest, and then go sit on their doorstep. Posting a job posting and expecting to get an amazing hacker--that rarely happens. Don't start doing that until you've been around for quite a while.

The other question he hears the most frequently is "How do I fire my co-founder?" He said to talk to a lawyer. Make sure your stock options have a vesting schedule.

He thinks KickStarter is a good idea.

In general, focus on getting to market. Generally, don't focus on patents. They're really expensive.

I don't think a 10-page business plan is important. No one does that anymore. However, a 3-4 sentence pitch is key.

The Art and Algorithms of Pixar's Brave

This talk was given by Danielle Feinberg, Directory of Photography for Lighting.

The only way I can explain what they do is "wow"!!! Unfortunately, we weren't allowed to take pictures of anything. This has really given me a new appreciation for what they do.

Debian: 19 Years of Free Software, Do-ocracy, and Democracy

This talk was given by Stefano Zacchiroli, the Debian project leader.

They now have support for running kFreeBSD. This was added about 2 years ago. It's important for running ZFS. I remember wishing back in 2001 that Debian would make it possible to swap in new kernels, especially FreeBSD.

Ubuntu is 15-30 times more popular than Debian.

Dropbox Notification Servers

Dropbox can handle 1.7 million clients connected per machine. Those servers are written in C++. They use epoll (in edge-triggered mode). They use libevent. Each process has 170k connections. Limiting the amount of memory used per connection is critical. Avoiding SSL where possible is also critical.

They get about 70% CPU utilization. IO bandwidth is their biggest bottleneck.

They make clever use of shared memory and swapping open file descriptors. That way, they can restart a server process without losing all the open connections.

They originally wrote things in Twisted, but they couldn't get past 5000 connected clients. Twisted and Python are both kind of slow.

They then wrote it in C++, then went back to C, and then went back to C++.

They have about 60 engineers.

They server about 100 million users.

They want to be the "filesystem of the Internet".

They say that the fact that they're platform agnostic is one of their biggest competitive advantages.

They don't do client-side encryption. They use one key to encrypt everyone's data.

They've had a couple security incidents. There are tradeoffs for how secure you need to be.

Their main data is stored in S3.

All their web stuff is in Python. They use Pylons, but they have evolved very far away from Pylons. They also have their own templating system.

MySQL really doesn't like supporting more than 100 open connections.

When they receive a DMCA takedown, they don't try to delete it from the user's account entirely. They just ban the item from being publicly shared.

A lot of people at Dropbox came from Google.

My talk: Structured Web Programming: An Introduction to Dart

Here's a link to my talk.

There were two other talks going on at the same time as mine. One of them was by Zed Shaw. I got about 65 people. This proves conclusively there are still 65 people in the world who don't know that Zed Shaw is a more entertaining speaker than I am :)

I also did a codelab. Things were going very well, but about half way through the codelab, most of the people suddenly left. Apparently, there were a panel starting, and some of the guys who built "Portal" were on it :)

Nonetheless, I'm really, really glad I made the trip down to UIUC!

Personal: Life Beyond Startups

I've gone to a lot of talks recently focused on startups. Personally, I've spent more than a decade at more than 10 different startups, and I certainly think startups are a heck of a lot of fun. However, I'm beginning to think they're a bit overrated. Some of the talks I've gone to lately almost suggest that doing your own startup is the only path to happiness and personal fulfillment. Let me dispel that myth.

First of all, not everything can be accomplished by two guys and a spare garage. Apple started out in a garage, as did the Wright brothers. However, the sleek MacBook Pro that I'm currently typing on bares little resemblance to the Apple I. There have been thousands of people needed to make Apple what it is today. The same is true of Google as well. Google is way more than just a search engine these days!

The Wright brothers may have been the first ones to successfully put a man in an airplane, but a Boeing 747 isn't something that can be accomplished by two guys and a garage. Similarly, it took a lot more than two guys and a garage to put a guy on the moon. The freeway that I drove on to get to work today may have been envisioned by some guy in Germany in the 1930's, but actually building the freeway took a lot more effort than is possible by one man. I definitely admire Linus Torvalds for creating Linux, but it has taken thousands of skilled hackers to build and package all the software present in a modern Linux distro.

A lot of these talks also seem to suggest that people at startups are happier and co-workers are smarter. Sometimes that's true, but it's definitely not always true. I've been very unhappy at a lot of startups, especially when I was burnt out from being overworked, and I wasn't sure there was going to be enough money left to pay my rent! I may be a bit biased considering I work at Google, but I think that my current co-workers are some of the smartest, friendliest, and happiest people I've ever worked with. I know I'm definitely enjoying myself!

Everything has to start somewhere, and it's a lot of fun to start new things. However, there are many very worthwhile things that take a massive amount of manpower to accomplish. So let's not forget all those people who aren't famous who nonetheless made it possible to fully accomplish many of the most monumental and influential things all around us!

Sunday, October 21, 2012

Video Games: TV Latency

I was having a lot of problems with latency on my TV, and my wife and I finally figured out how to fix it. I bought a used TV on Craigslist because my old CRT started smoking. The new one was originally purchased at Best Buy. It's an Insignia model number NS-32L550A11.

When I plugged in the Wii, the latency was so bad it was driving my entire family crazy. Here are the steps that my wife and I took to finally fix the problem:

  • First of all, I decided to buy a component adapter / cable to connect the Wii directly to the TV using a component connection. This made the picture look a lot better.
  • We resynced all of the Wii remotes and checked their batteries.
  • Then we went to the Wii settings menu and switched the screen to wide screen mode, and we set the resolution to 480p.
  • Then we made sure that the TV was in game mode. Other versions of this TV called this DCM mode.
  • We tried moving the sensor bar and checking the sensitivity.
  • The most important thing we did was reset the component settings in the TV menu to factory settings. I had seen other tips online that resetting the TV to factory settings would fix the problem. Resetting just the component settings was enough to get rid of the latency issues.

Now my wife can play her Wii Fit without wanting to kill the TV :)

Tuesday, October 09, 2012

Software Engineering: Corporate Wormholes

Imagine you have two teams that are located in different offices. It's sad when you lose the ability to just walk up and talk with someone, especially when you notice that they're just walking by. I think spur-of-the-moment meetings are underrated these days. They can really help speed up your team velocity!

Although I can't take credit for the idea, I think the solution is what I like to call "corporate wormholes". Setup a computer at each office with an always-on video connection with the other offices. Hence, you can always see when there are people in the other office. You can chitchat when they walk by, etc. If you have 10 small offices, then you can reach out to all the offices quickly if it's important enough to interrupt them (which should be rare, of course). Naturally, you don't need a wormhole to every part of your organization. You only need them for teams that are working on the same thing, but at different localities.

In general, corporate wormholes are a great way to build close-knit teams that are forced to work very far from each other.

Tuesday, October 02, 2012

Linux on the Desktop

I'm a Linux guy at heart, but I use a Mac. For a long time, I've had a weird love hate relationship with Apple laptops. I was feeling a bit nostalgic today, and I stumbled across these articles:

It reminds me of The UNIX-Haters Handbook, which as a fantastic read :) Warning, do not read any of the above if you're easily offended!

Who would have thought that Linux would be more successful in the mobile world than on the desktop? Who could have guessed that one of the most successful Linux on the desktop projects would actually embrace trusted platform modules and restrict access to the shell by default (unless you put it in developer mode)? What a strange world we live in!

Wednesday, September 26, 2012

Dart + HTML5 = Happy Web Developers

Jason Brooks and I ported a bunch of samples from HTML5 Rocks to Dart. Check out the project announcement to see how to use Dart with HTML5!

Monday, September 24, 2012

Dart: An Algorithm to Unindent Code

Have you ever had to write an algorithm to unindent a block of code consistently? Figuring out the maximum amount of whitespace among all the lines and then removing that amount of whitespace is non-trivial. Remembering to ignore lines that only have whitespace makes the problem even harder. Here's some Dart code that I wrote to do it. I do believe the performance is O(num_lines * max_line_length), which I think is optimal. It should be easy to port this to other languages if you need it:

/**
 * Remove the indentation from the given lines.
 * 
 * Only remove as much indentation as the line with the least amount of
 * indentation.
 */
List<String> unindentFilter(List<String> lines) {
  // Make a copy so that we can modify it in place.
  lines = new List<String>.from(lines);
  
  // Make sure there is at least one line.
  if (!lines.isEmpty()) {
  
    // Figure out how much indentation the first line has.
    var indentation = new List<String>();
    for (String char in lines[0].splitChars()) {
      if (char == " " || char == "\t") {
        indentation.add(char);
      } else {
        break;
      }      
    }
    
    // Figure out the least amount of indentation any of the other lines has.
    for (var i = 1; i < lines.length; i++) {
      String line = lines[i];
      List<String> chars = line.splitChars();
      
      // Lines that only have whitespace should be set to "" and ignored.
      var whitespaceOnly = true;
      for (var char in chars) {
        if (char != " " && char != "\t") {
          whitespaceOnly = false;
          break;
        }          
      }
      if (whitespaceOnly) {
        lines[i] = "";          
      } else {
        
        // If the line has something other than whitespace, see how its
        // indentation compares to the least amount of indentation we've
        // seen so far.
        for (var j = 0; j < indentation.length && j < chars.length; j++) {
          String char = chars[j];
          if ((char != " " && char != "\t") ||
              char != indentation[j]) {
            
            // We found a new line with less indentation.
            indentation.removeRange(j, indentation.length - j);
            break;
          } 
        }
      }
    }
    
    // Loop over all the lines, and remove the right amount of indentation.
    for (var i = 0; i < lines.length; i++) {
      String line = lines[i];
      
      // Ignore blank lines.
      if (line != "") {
        
        // Otherwise, trim off the right amount of indentation.
        List<String> chars = line.splitChars();
        List<String> unindented = chars.getRange(indentation.length, chars.length - indentation.length);
        lines[i] = Strings.concatAll(unindented);
      }
    }
  }
  
  return lines;
}

Here are some tests:

test("unindentFilter unindents code", () {
  expect(merger.unindentFilter(["  1",
                                "  2"]),
         equals(["1",
                 "2"]));
});

test("unindentFilter unindents code where the first line is indented the most", () {
  expect(merger.unindentFilter(["\t    1",
                                "\t  2",
                                "\t    3"]),
         equals(["  1",
                 "2",
                 "  3"]));
});

test("unindentFilter does nothing for unindented code", () {
  expect(merger.unindentFilter(["1",
                                "2",
                                "3"]),
         equals(["1",
                 "2",
                 "3"]));
});

test("unindentFilter handles empty lists", () {
  expect(merger.unindentFilter([]),
         equals([]));
});

test("unindentFilter does not try to handle inconsistent indentation", () {
  expect(merger.unindentFilter(["\t1",
                                "  2",
                                "    3"
                                "        4"]),
         equals(["\t1",
                 "  2",
                 "    3"
                 "        4"]));
});

test("unindentFilter handles really awkward short lines", () {
  expect(merger.unindentFilter(["    1",
                                "2"]),
         equals(["    1",
                 "2"]));
});

test("unindentFilter handles blank lines and lines with only indentation", () {
  expect(merger.unindentFilter(["  1",
                                "",
                                " ",
                                "    2"]),
         equals(["1",
                 "",
                 "",
                 "  2"]));
});

Friday, September 14, 2012

3 Articles Every Hardcore Dart Developer Should Read

I answer a lot of questions about Dart. Some of my friends are even working on their own programming languages, and they tend to ask really hard questions. In the past, I've struggled with some of those questions. However, I finally got around to reading three articles that answered a lot of those tough questions:

Reading these three articles really helped me appreciate Dart at a much deeper level. If you're a language geek like I am, I highly recommend them!


Wednesday, August 29, 2012

Java: Helloooooooo, World!

I was talking to my buddy, Peter von der Ahé, today about optimizing startup time. He asked me to guess how many classes the JVM had to load for a simple hello world application written in Java. He also asked me how long such a program would take to run. I didn't know the answers, so he quickly typed it out on my friend's MacBook Pro.

Here's how to see how many classes are loaded:

java -verbose:class Hello

That results in 594 classes:

[Opened /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar]
[Loaded java.lang.Object from /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar]
[Loaded java.io.Serializable from /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar]
[Loaded java.lang.Comparable from /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar]
...
[Loaded java.util.Collections$UnmodifiableMap from /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar]

It takes roughly 0.2 seconds to run hello world:

time java Hello
Hello, World!

real 0m0.212s
user 0m0.221s
sys 0m0.051s

Kind of interesting. I know that this is the world's stupidest benchmark, but I decided to write the same thing in Python. It takes 0.04 seconds.

Dart currently takes 0.02 seconds.

Ruby 1.8.7 takes 0.009 seconds.

C takes 0.006 seconds.

(Warning: bad jokes coming...)

Of course, the real reason C is so fast is because it doesn't have any class ;-)

As I said earlier, this is a totally stupid benchmark. Never pick a programming language based on how long it takes to run hello world. What happens if you have to write a program that says something other than hello world, such as "hej, verden"? ;-)


Tuesday, August 21, 2012

Books: Memoirs Found in a Bathtub

I just finished listening to Memoirs Found in a Bathtub by Stanislaw Lem. It was recommended to me by someone long ago, but I can't remember who. All I can say is WTF!?! Fortunately, Wikipedia has a better summary:

Set in the distant future, Memoirs Found in a Bathtub is the horrifying first-hand account of a bureaucratic agent trapped deep within the subterranean bowels of a vast underground military complex. In a Kafkaesque maelstrom of terrifying confusion and utter insanity, this man must attempt to follow his mission directives of conducting an "on-the-spot investigation. Verify. Search. Destroy. Incite. Inform. Over and out. On the nth day nth hour sector n subsector n rendezvous with N."

The narrator inhabits a paranoid dystopia where nothing is as it seems, chaos seems to rule all events, and everyone is deeply suspicious of everyone else. In danger of losing his mind, the protagonist starts keeping a diary, and it is this diary which details only a few days in his life that is ultimately found by a future society and given the title Notes from the Neogene. Memoirs Found in a Bathtub is this distant voice from the past, this Notes from the Neogene.

Here's a quote from the book:

"I'll tell you. You're young, but you're one of us, and I'm one of us, so I'll tell you. Everything. Now, say someone's one of us. . . but he's also—you know—you can tell, right?"

"He's not—one of us," I said.

"Right! You can tell! But sometimes—you can't tell. You think someone's one of us, but they got to him and then he wasn't any more—and then we got to him, and he was—but he still has to look like he isn't, that is, like he only looks like he is! But they get wise to him and—now he isn't again, but he has to look like he isn't—or we'll get wise—and that's a triple!"

Tuesday, August 14, 2012

Python: Party Playlist Picker

At long last, Jeffrey Posnick and I finally got around to open sourcing a project that we started as my starter project at Google. Here is the blog post we posted on the YouTube API Blog:

When Google+ first came out, we were really excited about the idea of writing some open source code to showcase how you could integrate YouTube with Google+. After some brainstorming, we came up with the idea of letting people collaboratively edit a playlist in realtime, and we started hacking!

We started working on Party Playlist Picker in early 2011. We’ve always wanted to release our experiment as an open source project, and you can now download the code to see one example of integrating Google+ functionality to enrich a YouTube application.

Party Playlist Picker is a Google App Engine application written in Python. It uses a Google+ share button so that partygoers can invite their friends to come edit a playlist in realtime. That feature is based on the Google App Engine channel API. It makes use of the gdata-python-client library to talk to YouTube and the google-api-python-client library to talk to Google+. It’s currently using version 1 of the YouTube API (because that’s what gdata-python-client uses), and we used OAuth 2 for authentication. In fact, we made good use of the Python client library to manage the OAuth 2 tokens for us automatically.

There’s also a lot of jQuery-based JavaScript goodness. For instance, you can search for videos without reloading the page, and you can simply drag them to add them to the playlist. If multiple people edit the page at the same time, the list of videos is updated in realtime whenever a change is made. You can also watch videos right on the page. Last of all, it keeps track of who’s edited the playlist, and whether they’re currently editing it.

There are a bunch of subtle HTML5 tricks thrown in as well. For instance, the application supports voice search, and we made a clever use of gradients to fade overflow text in the video descriptions.

Here’s a screenshot of the application:

If you’re interested in learning more, you can play around with a running version of it, or you can download the source code and hack away! If you have any questions, post them to our developer forum.

Cheers,
--Shannon -jj Behrens and Jeffrey Posnick, YouTube API Team


Friday, August 03, 2012

Dart: My Quest to Program Dart on a Chromebook

They gave me a cool new Chromebook at work. Now, the sensible thing to do is to use it as a convenient web browser and nothing more. However, I'm not a very sensible person, and for some reason, I dislike having more than one computer at a time. Since I'm a member of the Dart team, I figured I should be prepared to use Dart Editor at any time of the day or night. Hence, I embarked on a multi-day journey, driven by my own obsessive compulsive disorder, to somehow figure out a way to run Dart Editor on a Chromebook.

By far, the easiest approach is probably to install Ubuntu on it. I didn't want to take that approach since I really wanted to use ChromeOS as well, and I hate dual booting.

The next approach is to put the machine in developer mode and install various things within the existing Linux distro. There are some pretty good instructions on that here. They involve setting up a build server, etc., which I balked at--I want fewer computers, not more computers!

My next idea was to install a full Linux distro in a directory and chroot into it. I thought I was the first one to come up with this idea, but apparently, several other people have had this idea as well (which is the way ideas usually work). There are pretty good instructions on how to do that here. By the way, that page covers the other approaches as well. I spent a couple days with this approach, and I made a lot of progress. However, I never got things to work just right. I faced a few major hurdles:

  • The Chromebook is already running a window manager, but it doesn't work as a normal X11 window manager. Dart Editor doesn't behave very well without a normal X11 window manager. Furthermore, getting another window manager installed and running is non-trivial, and it makes ChromeOS look ugly.
  • There's a lot of confusion about 32-bit vs. 64-bit binaries. If I understand correctly, the Chromebook is a 64-bit device running 32-bit binaries. There's also some confusion about 32-bit vs. 64-bit in the Dart Editor world which threw me off.
  • I was having a hard time getting all the right dependencies to get Dart Editor and Dartium running.

Eventually, I discovered that some companies...ahem...don't like it if you put a Chromebook into developer mode and then try to connect to the corporate network. That kind of makes sense since putting a Chromebook in developer mode circumvents many of the nice security features the Chromebook provides. Hence, even if I did manage to get any of the above approaches working, I wouldn't be able to use it at work.

My next approach was to look at Cloud9 IDE. It looks interesting, but it doesn't support Dart, and it definitely doesn't support Dart Editor.

My next approach was to try "Chromoting" into a Linux box in the cloud using the "Chrome Remote Desktop" extension. I figured that would be easy to get working. Best of all, this doesn't require any hacking of the Chromebook. However, it has a few major drawbacks:

  • It doesn't work when I'm on a train (which is one day a week).
  • It doesn't work when I'm on a plane. It may sound strange, but almost all of the time I've spent learning Dart was while on planes flying around to give various YouTube API talks.
  • If I use chromoting to give a talk on Dart, and my network connection goes down (which tends to happen every time you give a talk), I wouldn't be able to use Dart Editor.
  • Chromebooks don't currently have a VPN solution.
  • Chromebooks do support SPDY Proxy, but that doesn't currently work with Chromoting.

Even with all those drawbacks, I knew Chromoting was the only solution likely to work for my particular set of constraints, so that's what I got to work. I had to install a 64-bit JDK and the 64-bit version of Dart Editor, but I eventually got things working. Hence, the picture above is a picture of me Chromoting into my Linux box and running Dart Editor.

Ok, now that I've spent way too much time on this, and it's still not something I'd want to use on a daily basis, maybe it's time I get back to work ;)

Update September 11, 2012: I tried installing ChrUbuntu 12.04 on it using these instructions. I got Dart Editor working, but my wireless wouldn't work reliably.

I've heard that there are occasional driver issues when running 64-bit Linux on this device, so next I tried following these instructions. They walk you through installing ChrUbuntu 11.04 and upgrading twice to 12.04. Sure enough, this fixed my wireless problems. It temporarily broke my trackpad, but plugging in a mouse and rebooting fixed that problem. However, after following those instructions, I couldn't get Dart Editor to work. Whenever I tried to run it (from the command line), it would say, "Cannot find DartEditor". This is despite the fact that I was specifying the path to the executable. I think this is yet another weird 32-bit vs. 64-bit problem in the JRE.

The fact that Chromebooks have a funky EFF BIOS makes them really great for running ChromeOS securely but not so convenient when you're trying to run a normal version of Linux. Having been defeated twice more, I got a USB drive and installed ChromeOS from scratch and exited developer mode.

Update February 26, 2013: I was hoping that "emerge chromeos-dev" was going to be the path to success, but it turns out that it probably won't be, at least in the short term. See this forum post. Right now, Crouton looks to be the most viable approach. Also see this blog post.

Update February 27, 2013: Success! I got it to work, thanks to Crouton, which is based on using a chroot. See my newer blog post.

Monday, July 30, 2012

Ruby: Quizzimoto

I finally open sourced Quizzimoto. Quizzimoto is an application that lets you build video-based quizzes. My goal was to do something interesting in the educational space using Ruby on Rails and YouTube APIs. I used Quizzimoto as a sample application in my tutorial, Google I/O 101: Using Ruby on Rails and YouTube for Education.

If you're interested, here's the source.

PHP: Phycocauth

Phycocauth is a sample project that combines the following: PHP, YouTube, OAuth2, CodeIgniter, the CodeIgniter Youtube API Library, and the Google APIs Client Library for PHP.

I'm releasing this proof of concept to show that it's possible to integrate the CodeIgniter Youtube API Library with the Google APIs Client Library for PHP. Once version 3 of the YouTube API is ready, the CodeIgniter Youtube API Library will no longer be necessary. All you'll need is the Google APIs Client Library for PHP which will be able to handle OAuth2 as well as the YouTube API.

See application/controllers/example.php for the most interesting piece code.

Friday, July 27, 2012

Books: Content Rules

I just finished listening to Content Rules: How to Create Killer Blogs, Podcasts, Videos, Ebooks, Webinars (and More) That Engage Customers and Ignite Your Business. In short, it was good. If it's part of your job to talk to the world using social media, it's worth reading.


Friday, July 20, 2012

Books: What Technology Wants

I just finished listening to What Technology Wants by Kevin Kelly, the guy who started "Wired" magazine. What an incredible book! Here's Amazon's description:

In this provocative book, one of today's most respected thinkers turns the conversation about technology on its head by viewing technology as a natural system, an extension of biological evolution. By mapping the behavior of life, we paradoxically get a glimpse at where technology is headed-or "what it wants." Kevin Kelly offers a dozen trajectories in the coming decades for this near-living system. And as we align ourselves with technology's agenda, we can capture its colossal potential. This visionary and optimistic book explores how technology gives our lives greater meaning and is a must-read for anyone curious about the future.

In short, it's a fascinating, holistic, comprehensive philosophy of technology. Kelly covers things such as:

  • The history of technology since pre-human times
  • The history of technology from the perspective of a hydrogen atom starting at the big bang
  • How technology can be thought of as a new stage of life in the history of the universe
  • How completely dependent we are on technology for our very survival these days
  • Why a declining population a few decades from now might result in a slowdown in progress

All I can say is that I couldn't put it down, and I highly recommend it!


Books: Peopleware: Productive Projects and Teams

When I worked at Xmarks, I had a boss named Penny. Before coming to Xmarks, she worked as a manager in operations at Amazon. Apparently, she went several years with no employee turnover. Obviously, she was a very good manager. I asked her what her secret was. She told me that I should read a book called Peopleware: Productive Projects and Teams:

Demarco and Lister demonstrate that the major issues of software development are human, not technical. Their answers aren't easy--just incredibly successful...

Fast forward 5 years, I finally took her advice. What a fantastic book! It was first released in 1987, and an updated version was released in 1999. However, it's just as relevant today. I really think this book ranks up there with The Mythical Man-Month. I've spent so much time over the course of my career reading technical books and learning new programming languages in order to perfect my craft and increase my productivity. However, I was oblivious to the fact that so many impediments to productivity and project success weren't technical at all--they were "peopleware" problems. I would have been way better off reading this book earlier in my career. I highly recommend it!


Thursday, June 21, 2012

Books: What Would Google Do?

I just finished listening to What Would Google Do? It was okay. I think the book was really about what Jeff Jarvis would do. For instance, he argued very strongly for some things that he admitted Google doesn't do. In general, I think he gets a lot of stuff right, but he also overstates some things, and he even gets some things wrong. If you really want to understand Google's ethos, I think In the Plex: How Google Thinks, Works, and Shapes Our Lives is more on target.

Tuesday, June 19, 2012

Using Ruby on Rails and YouTube for Education

The other tutorial that I created for Google I/O this year just went live. In "Using Ruby on Rails and YouTube for Education", I explain that Ruby on Rails, YouTube, education, and TDD are like a peanut butter, honey, and banana sandwich! I'm particularly proud of the section that teaches you how to do test-driven development with RESTful web services, including a "live" (for some definition of live) coding session. Enjoy!

YouTube for Your Business

I did two tutorials for Google I/O this year. The first one, "YouTube for Your Business", just went live. It's amazing that it took weeks of effort and 8.5 hours of filming to produce these two videos, but they're less than 45 minutes long combined! Anyway, if you're interested in learning about all the different ways you can use YouTube for your business, including both technical and non-technical examples, check it out!

Thursday, June 14, 2012

Books: In the Plex: How Google Thinks, Works, and Shapes Our Lives

I just finished listening to In the Plex: How Google Thinks, Works, and Shapes Our Lives. I liked it. While it may not be as good as Steve Jobs' biography or Steven Levy's other book, Hackers, I still really enjoyed it. It really helped me put my experiences at Google into context. It's amazing how little Google has changed, how much it continues to work on, and how much Steven Levy was allowed to get an inside look at Google.

Thursday, June 07, 2012

Books: Steve Jobs

I listened to Steve Jobs' biography while commuting to and from work. It was incredibly interesting, and extremely well written. I highly recommend it.

Wednesday, May 23, 2012

Async: To Be or Not To Be

Just because I have to use a callback-oriented style on the client doesn't mean I want to use a callback-oriented style on the server. Now, before anyone gets all upset and tells me that I don't know the difference between async and a kitchen sink, let me explain :)

The client is necessarily an event-oriented place. If I don't know which button the user is going to press, it makes a lot of sense to use a different callback for each button. The server is different. If I'm waiting for the result of a database query before I can continue processing a request, it sure is convenient to just block and wait.

My key point is that it's important to separate what style you want to code with and what performance and scalability characteristics you want. You shouldn't necessarily pick a callback-oriented style just because you want the performance and scalability characteristics of asynchronous networking APIs.

My favorite two examples are gevent and Erlang, but Go is similar. When you code using gevent or Erlang, your code looks like synchronous, blocking code. However, below the covers, they use asynchronous networking APIs. Now, before anyone tells me that it's impossible, buggy, or that it'll never work, let me point out that these tricks have been in production for decades at Ericsson, Yahoo Groups, and IronPort Cisco.

Furthermore, I should point out that asynchronous networking APIs aren't a perfect fit for every problem. For instance, if your goal is to send 10 gigabytes of information to another server, it turns out that synchronous networking APIs will actually outperform asynchronous networking APIs. The reason asynchronous networking APIs are so popular is because they can handle a larger number of clients than synchronous networking APIs can and because they use less memory than a large number of threads, which each have to have their own stack. gevent and Erlang can handle a large number of clients, don't use up much memory, and don't require a real OS-level stack per client.

So what's my problem with the callback-oriented style? I find it a lot harder to read. I've coded projects in Twisted, Node.js, etc., and I prefer the gevent approach. You get roughly the same performance and scalability characteristics, but with much easier to read code. Of course, what's readable to me may not be readable to other people. I've met people who are perfectly happy using Twisted Web 1 and don't think that callback-oriented code poses any real challenge.

If you're interested in hearing more about my thoughts on async and concurrency, check out my other blog posts, which include a link to my Dr. Dobb's Journal article on Python concurrency.

Tuesday, May 22, 2012

HTML5 Dev Conf

I went to HTML5 Dev Conf yesterday. The following is a loose collection of my notes.

First of all, the place was packed! Even the second-tier talks didn't have any standing room left over.

The Backbone.js talk was especially packed. Not only could I not get into the room, I couldn't even get anywhere near the door. In general, Backbone.js was an extremely hot topic. However, most people admitted that they hadn't used it.

Douglas Crockford's Keynote

JSON has no version number. "Everything with a 1.0 gets a 1.1 and a 2.0, and it's all crap until you get to 3.0, so JSON has no version number."

He talked about the first browser, Mosaic. Some company patented an <embed> tag to embed content in a rectangle in the browser. He said that the company was a patent troll, and they went after everyone trying to make money. They figured their patent covered JavaScript, image tags, etc. Apple settled. M$ fought them for years. Yahoo finally led the defense and won.

Crockford was using an Alienware laptop.

Crockford is a calming speaker, but it's easy to overlook the fact that he has strong, unusual opinions.

Crockford cancelled his Facebook account because of their privacy policy.

Crockford is against software patents (and possibly against all patents?).

Yahoo sued Facebook because of patent infringement. That's why he left Yahoo. He works at PayPal now.

Crockford spent some time bashing Yahoo. However, he was also hopeful that the company could be turned around.

He said Yahoo had a globally scalable NoSQL database in 1995.

"Yahoo is profitable and has always been profitable."

Yahoo's business is to do "Anything [on the web] for a dollar."

Yahoo is organized into properties. It's 100 companies that share a cafeteria.

They "misuse" agility.

Crockford kept answering his cellphone during his talk. At first I thought it was a joke, but I'm pretty sure it wasn't. It wasn't even someone important.

He doesn't have a degree in CS. He has a degree in broadcasting.

He thinks that Yahoo should:

  • Optimize for customer value.
  • Create a culture of excellence ("Yahoo doesn't have that.").
  • Fork Node.JS and create Ynode.
  • Desilo the company.
  • Focus on improving and integrating Yahoo Groups (it's basically 12 year old software).
  • Fix the research group.
  • Bring back Jerry Yang as chief Yahoo (instead of CEO).

He ended the talk by saying, "Don't forget your semicolons!"

He called himself "Doug Crockford".

He said, "I love CoffeeScript," but also expressed certain reservations about it.

Behind the Scenes of "Cut the Rope"

This talk was given by Giorgio Sardo who does Windows evangelism at Microsoft Corp.

He showed PacMan on an really large board (80,000 user generated boards all connected).

"IE9 Hardware Accelerated HTML5."

"Cut the Rope" was originally a native iOS application. They rewrote it in HTML5.

It was 40,000 lines of code. They translated it directly to JavaScript. They went from OpenGL to Canvas.

They're using setInterval for animation. They have one image with all the different animation states.

The rope was the hardest part. It's made of 20,000 triangles. There's another 10,000 triangles for the shadow. It's easier on HTML5 because they can use bezier lines.

He used a JavaScript profiler to fix his performance problems. It saved the project.

He's using Microsoft Visual Studio (of course).

He's using jQuery.

They worked on the project for 12+ weeks.

The original game developers are in Russia.

Windows 8 makes it easy to take a web app and make it a desktop app.

There's a Windows 8 store.

Benefits of CoffeeScript

For some reason, the speaker didn't show up. I offered to give an introduction to Dart. Someone else offered to give an introduction to CoffeeScript. After doing our respective short introductions, we held a panel-like discussion in which we answered questions from the audience.

CoffeeScript has an emphasis on conciseness, and it is inspired by Python and Ruby. Dart has an emphasis on building extremely large, client-side applications, and it is inspired by JavaScript and Java.

I had always heard that CoffeeScript had a weird scoping issue. If you have a local variable, and someone introduces a module-level variable of the same name, your local variable will start referring to the module-level variable instead of shadowing it. The CoffeeScript panelist said that he worked around this problem by using a different version of CoffeeScript that has ":=" syntax for setting non-local variables.

CoffeeScript places an emphasis on translating to JavaScript in very predictable ways. Dart places an emphasis on avoiding "puzzlers" (such as those present in the "Java Puzzlers" book).

CoffeeScript generates JavaScript that it is easy to debug. Dart can be debugged natively using a version of Chromium that has Dart support built in.

CoffeeScript takes JavaScript and makes it "even more dynamic". Dart takes JavaScript and adds optional static typing in order to facilitate building larger projects.

Overall, the panel discussion was friendly and enlightening.

Cross Platform Mobile and Web

This was a talk from a guy at LinkedIn on how they handle developing for the web and multiple mobile platforms.

Unfortunately, I missed the first 15 minutes.

Their iOS application is a mix of native and web code. It's 20% native code and 80% web code.

They use native code for all infinite lists because it's too hard to do infinite lists using web code.

They decided PhoneGap was not a good fit for them. One of the reasons was that they wanted to decide whether to use native code or web code on a case-by-case basis.

On Android, they wrote more native code than they did on iOS. About 80% of the code was native. However, they're slowly transitioning more toward HTML5.

For testing, they're using many projects, including: Vows, Robotium, Selenium, FoneMonkey, GHUnit, Hudson, PhantonJS, and some internal stuff.

They ship everything twice a week.

The apps and server deploy independently.

They're using Node.js. They upgrade it frequently.

A single Node.js instance can achieve 150qps and only requires 50mb.

LinkedIn uses Java, Scala, Python, Ruby, Node.js, etc.

IntelliJ can step through Node.js code thanks to the WebKit debugger.

They don't have a good debugging solution for mobile. They rely on logging.

They use MVC on the client side.

A "screen" is a "div to div move."

A "page" requires a "whole new browser page load."

A single page can have multiple screens. Each screen has a controller and a model.

They want to reduce the number of frameworks they use and increase the number of libraries they use. A framework calls you, whereas you call a library.

They use Backbone.js. It:

  • Provides routing from screen to screen.
  • Controls screen lifecycle.
  • Controls view updating based on model changes.
  • Has model construct for validation.

They're very tied to Backbone.js.

They use several libraries including: Zepto (to manipulate the DOM), iScroll (for inertial scrolling), and Underscore (which provides collections helpers, binding constructs, and templating).

They use SASS.

They embrace raw JavaScript instead of using CoffeeScript, etc.

They use Closure to minify and optimize their JavaScript. It also compiles their templates.

A bundle is a list of screens, templates, etc.

They cache bundles, images (?), etc. into local storage. They use MD5 hashes for files, and they only have to download diffs of files when they make changes. They said that mobile browser caches were very fickle, so they prefer to use offline storage to manage their own cache.

They use Jasmine as well as Selenium.

They use Capistrano.

Responsive Web Design: Conquering the Web, One Screen at a Time

This talk was mostly geared toward designers.

Responsive Web Design emphasizes web designs that work well on a variety of web and mobile devices. It came out of the "fluid layout" movement. However, these designs use CSS and other tricks to adapt themselves to look good everywhere, even on mobile.

Break points define when you should switch to a different layout. Try resizing your window drastically to see if there are break points that cause the application you're using to switch to a different layout.

Layouts are the thing that change the most. However, you might want to change the type (i.e. font) as well.

On mobile, you should probably use 14px Arial for your font.

You might want to change which image you show based on the screen size. The way to do this hasn't completely been resolved. There's some new "picture" element they're playing with in the standards bodies (the W3C?).

"Starting with the desktop may be an increasingly backwards way of thinking about a Web product." -- Luke Wroblewski

He said half of your users are mobile.

Opera is the most popular mobile browser world wide (I assume this refers to cellphones that aren't running Android or iOS).

He said QA is very important. You have to test on many different devices.

Focus on your content!!!

He claimed that Flash doesn't work reliably on Android.

mediaqueri.es has a lot of examples of sites that use responsive web design.

There is a fold, but ignore it. Focus more on horizontal sizes, not vertical.

These are the screen widths you should consider: 320px (important), 480px (sideways iPhone), 640px, 768px (important; older iPads), 960px (important), 1184px, and 1400px.

Sweat the small stuff such as type (i.e. font), navigation, and links.

44px is a good rule of thumb for a nice navigational link.

Design in tiles. Tiles should be flexible. They can be swapped.

If you want to learn more, search for "media queries".

See also developer.mozilla.org/en/CSS/Media_queries.

Media queries let you specify different CSS for different situations.

Examples:

@media screen and (max-device-width: 810px) {
  div {
    background: blue;
  }
}

<link rel="stylesheet" media="screen and (color)" href="foo.css">

There are many media features you can use such as width, device-width, orientation, color, etc.

See also: responsive.is

"Working on the web...is defined by its transcience, often refined or replaced within a year or two." -- Ethan Marcotte

He uses SASS. He doesn't like LESS because it's based on JavaScript.

Twitter has a framework called Bootstrap. He doesn't use it.

Adding a Server-side Backend to Your JS-powered App

This was a pitch for StackMob.

StackMob, in my own words, is a platform as a service offering for pure client-side applications written in JavaScript. They provide the server-side infrastructure so that you don't have to.

StackMob uses Backbone.js.

He said that when you use the Facebook API and the Twitter API, it's easy to forget that those companies have massive backends.

StackMob's goal is to create a server-side backend for your JavaScript-powered application.

They provide many backend services such as user auth, a datastore, security, GeoSpatial APIs, push support, static file hosting, social integration, custom server-side code, and analytics.

They're a "mobile backend platform".

They are "giving everyone access to 'big company' backend power."

They are "lowering barrier of entry."

The backend is accessible via a JavaScript API built on top of a RESTful API.

It's based on OAuth2, Ajax, and Backbone.js models.

They've integrated persistence to StackMob into their Backbone.js-based API.

They have a nice looking dashboard.

It looks like their datastore is schemaless.

They use a mix of MySQL and MongoDB.

You can fetch an object as well as all of its related objects, up to depth of 3. I'm worried about the security implications of retrieving things you shouldn't have access to.

I'm also worried that someone can hack your JavaScript to do things that your app wouldn't want you to do. I don't think StackMob has any way to differentiate trustworthy JavaScript from non-trustworthy JavaScript.

They're using OAuth2.

Public/Private keys identify your app's requests.

They're going to add access controls "very soon". This would allow you to share access to certain objects.

I really don't think they have a viable, comprehensive security model yet.

They're a Java/Scala shop.

They have a way for you to run your Java on their servers.

They don't have an approach to server-side validation in the SDK yet.

If you use a password field, they will explicitly not give it back to you when you load the user object.

They deal with cross-domain issues by hosting your static files.

They support custom domains.

They're integrated with GitHub. I.e. they can update your code when you push to GitHub.

They give you separate database environments for development and production.

They encrypt your data before storing it in the database.

It's free for limited use. They charge for higher volume use.

They have support for using S3.

He used reveal.js for his slides.

0-60: Animating the Web

This talk was given by Jeremy Kahn who works for YouTube. (I've never met him.)

Animation is used to draw a connection between two separate things.

"Flash is not the future of the web."

CSS3 has powerful APIs for smooth animations on the web.

Ideally, we want 60 frames per second. Flash's default frame rate is 12fps.

Tweening is a technique where you move from a start position to a stop position over a period of time. The goal is to let the computer do the hard work of making the animation between the start position and stop position work. Tweening is a popular technique in Flash. It's also supported by jQuery's animate function.

Keyframing is a little more complex than tweening. Keyframing lets you have any number of points in between the start and the stop.

Here are examples of the two techniques using jQuery:

// Tweening
$ell
  .css({ left: -700 })
  .animate({ left: 0 }, 2500);

// Keyframing
$ell
  .keyframe(0, { left: -700 })
  .keyframe(2500, { left: 0 });

Do not use setInterval for animation. If your callback takes too long, all sorts of bad things will happen.

It's better to use setTimeout.

It's even better to use requestAnimationFrame. It tries to call itself 60 times a second.

When you think about requestAnimationFrame, take into consideration the time delta, and make sure you have a terminating condition.

You can use MVC informally to break up your animation code into separate functions.

You can do animation using only CSS3 (i.e. without using JavaScript).

There is CSS syntax for keyframes (it uses @).

CSS has support for easing.

Ceaser is a tool for visually generating bezier curves for CSS easing.

Use different easing functions for X and Y to get interesting curves in your animations.

JavaScript animation:
Pros: flexible, runs anywhere
Cons: Everything lives in the JavaScript thread

CSS animation:
Pros: Better performance / smoother (it uses your computer's graphics card)
Cons: Limited easing formula choices, not supported everywhere

The difference in performance is huge on mobile devices.

Minimize reflows.

One approach is to use "position: absolute".

A better approach is to use the transform property.

Animating <canvas> requires different techniques.

Avoid clearing the canvas on every animation frame. This is the number one performance killer.

There are a couple techniques that help with <canvas> animations--pre-rendering and using multiple canvases (one for the background and one for each shape).

CSS transforms can use sub-pixel rendering.

When you scale stuff, CSS doesn't smooth out stuff once you get above 1.0. Hence, always scale up to 1.0. Try not to exceed 1.0.

Here is a list of CSS tools: Ceaser, SASS mixins

Here is a list of tools for JavaScript tweening: jQuery, Zepto, Shifty (which he wrote)

Here is a list of keyframing tools: Rekapi (which he wrote; it's a model and controller for keyframe animations)

Here is a list of graphical tools: Sencha Animator, Adobe Edge, Stylie (which he wrote)

HTML5 is still trying to catch up with Flash when it comes to authoring tools.

Should you use JavaScript or CSS for animations? He recommended using JavaScript to control CSS animations.

HTML5 is still trying to catch up with Flash's authoring environment.

He used Keynote for his slides. Keynote has some cool transitions.

Hype is an HTML5 authoring tool.

You should separate your state logic and your drawing logic.

Mobile Web Debugging and Performance

The speaker works at Oracle. He uses PhoneGap.

He wants to replicate the experience that Developer Tools provide on Chrome, but in a mobile context.

There's a JavaScript debugger on iOS. It shows errors. It's very limited.

The same is true in the debugger for Android. It's a "little better than iOS."

WebKit supports remote debugging. Hence, you can debug an app running on the simulator.

Chrome for Android 4.0 and above has USB debugging. This lets you debug on a separate port. It looks like Developer Tools on Chrome. Search for "Chrome for Android remote debugging".

abd = Android Debug Bridge

See: code.google.com/chrome/mobile

This doesn't currently work for WebView widgets because those don't run in Chrome.

"The native WebKit browser is the default for WebView apps, and it's way behind the curve."

Opera also supports remote debugging. You need Opera on your desktop and Opera mobile on your mobile device.

Firefox Mobile also has remote debugging, but it's brand new. It was developed at a hackathon.

iWebInspector is a web debugging tool for the iOS simulator. It was written by the guy who wrote jQuery Mobile.

Remote JS is a tool from Sencha Touch. It's tool for Android applications that use WebViews. You don't have to use Sencha Touch in order to use it.

Wiener (see the Apache Cordova Project) is another tool for doing remote debugging. However, it's somewhat limited.

jsconsole.com is a JavaScript console that you can add to your app.

Socketbug is a project written using Node.js and Socket.IO. It's another tool for mobile JavaScript debugging. It supports iOS, Android, and Palm.

Adobe Shadow is a commercial tool for mobile JavaScript debugging. It's currently free. It allows you to control multiple remote devices at the same time.

blaze.io has a free mobile web performance testing tool called Mobitest.

MobilePerf is another mobile web performance testing tool.

Friday, May 18, 2012

Code Reuse has Finally Arrived...Obviously!

I got into a discussion about code reuse with my co-worker, Jarek. I remember reading books on object oriented programming from a few decades ago in which OOP promised to increase code reuse. A lot of people doubted that serious code reuse would ever happen. OOP suggested that code reuse would be possible by reusing and subclassing existing classes. I even remember hearing C++ programmers talk about purchasing libraries of useful classes.

To some degree, this did actually happen. For instance, QT is a wonderful library for C++, and it certainly makes coding in C++ a lot easier. It was even proprietary up until a few years ago. Library reuse also happened via language-specific package sites such as CPAN (for Perl), PyPI (for Python), and a range of sites for Ruby. Furthermore, most languages come with a lot more stuff built in these days. For instance, compare Python's standard library ("batteries included") with that of C.

However, compared to decades ago, there is less of an emphasis on class-level reuse these days. For instance, rather than trying to code classes that will be flexible enough to solve tomorrow's problems, agile programmers suggests you write your code in such a way that you will be able to modify it to solve tomorrow's problems (with an emphasis on tests that will prevent regressions when you modify the existing code).

These days, rather than focusing on class-level reuse, we've achieved reuse via RESTful web services (and variations thereof) as well as via open source stacks. RESTful web services are usually closed source and proprietary. However, they have a lot of benefits. For instance, they're language agnostic. It's also a lot easier to control access to a RESTful Web Service than it is to a class library. This permits good authorization controls and various monetization strategies.

As I mentioned before, another place where we see a lot of reuse is in the open source stack. Just as Taco Bell has a million things on the menu that are all made from the same 10 ingredients, there are a million startups out there all based on the same stack of open source software. We even have a word for it--LAMP. Sure, there are variations of this--PostgreSQL instead of MySQL, Nginx instead of Apache, Python and Ruby instead of PHP--however, the fact remains that a lot of startups are only possible because of the tremendous amount of code that is available for reuse.

Hence, it's clear that code reuse arrived during the last 10-20 years. It didn't come exactly as we expected, but it's definitely here.

Documentation Sucks, Video Games Rule!

One thing that's great about a lot of video games is that you don't have to read any documentation in order to start playing them. I tend to play games on the Wii with my wife, and those games tend to teach you what you need as you go along.

I think the same thing applies to user interfaces. Having to read documentation kind of sucks. Being able to learn as you go along is much nicer. A user interface that is easy to get started with is great, and one that teaches you on the fly how to become a power user is even better. For instance, I think that menus are a great way to enable users to discover new features. Menus that also show you the relevant shortcut keys are great because you become a power user by gradually memorizing frequently used commands. A user interface that does useful things automatically (assist and suggest) is the best of all (for instance, code assist).

Another thing I feel strongly about is that a piece of software doesn't have to be hard to use in order to be powerful. Compared to raw SQL, spreadsheets are very easy to use. However, they're also very powerful.

I continue to be a fan of Vim. However, when I first tried TextMate, I was amazed at how well its menus were organized. It even had a system for organizing the actions provided by plugins. This permits you to learn the advanced features (even of plugins) as you go. In fact, I was able to do stuff in TextMate that I still don't know how to do in Vim. I'm sure that Vim doesn't lack such features; it's just that it was a lot easier for me to discover how to use them in TextMate.

These days, I've been using Sublime Text 2. One thing that it can do that I really like is automatically suggest word completions as I type. The visual feedback is very discoverable. The Chrome location bar does the same thing.

Wednesday, May 16, 2012

Dart in Glasgow, Scotland

I gave a talk on Dart in Glasgow, Scotland. A big round of thanks go to my friends at Moviecom.tv and 29studios for not only setting up the talk, but also recording and editing it.

Tuesday, May 15, 2012

Unit Tests Don't Find Bugs: the Death of QA

Unit tests don't find bugs. They find regressions. This is a painful lesson I learned when I first started doing TDD (test-driven development), and it's well known among most TDD circles.

TDD's goal is to prevent programmers from introducing new bugs into working code. However, when you're writing code from scratch, your tests won't help you find all the bugs in your code. That's because you can't possibly write tests for all the ways your software will be used (or abused). When I first started doing TDD, I had really good tests, but I was too tired to do much exploratory QA. However, my boss wasn't, and I was very embarrassed to find that my software had lots of bugs. Simply put, he used my software in ways that I hadn't intended.

I've seen a lot of companies that don't bother writing any tests or doing any QA. They just let their users find all the bugs. Needless to say, I've never had respect for those companies.

However, it's growing more and more popular to destaff the QA department and just require engineers to write lots and lots of tests. Often, these are in the form of unit tests. Even though integration tests can conceivably catch more bugs, they take much longer to run. Hence, even integration tests are often deprioritized.

What I'm discovering is that a lot of projects have both lots of unit tests and lots of bugs. These are bugs that could have been found manually by a QA engineer, but it seems that manual QA testing (i.e. exploratory testing) has gone out of vogue.

I used to think that code that was well-documented, well-styled, well-tested, and code reviewed would rarely have bugs. Sadly, I no longer believe that to be the case. I think we need to go back to the days when we had decently-sized QA departments, perhaps in addition to all the other things we do.

To tweak what Knuth said, "Beware of the above code. I have only tested that it works. I haven't actually tried it."

Friday, April 06, 2012

Books: Agile Web Development with Rails

I finished reading most of Agile Web Development with Rails, fourth edition. This is the third edition that I've bought and read. I still love this book.

This edition is much shorter than the previous editions. I tend to read books cover-to-cover, so I appreciate when they aren't full of reference information that I can find online. I think this book struck the right balance of breadth, depth, and length. My buddy Shailen Tuli thought it was too short. Your mileage may vary.

I think the only thing bad about this book is that it focuses on Rails 3.0. Because Rails 3.1 has something called the "asset pipeline", it's not so easy to make the jump. I decided to stick with Rails 3.0 for the time being because I really like having a book to refer to.

Tuesday, April 03, 2012

Personal: My Laptop is Worried About Me

My laptop and I have a close relationship. After all, we do spend all day with each other! Hence, it's unsurprising that we show little bits of affection for each other. For instance, my shell prompt is "Yes, dear?", and the last command I type before reinstalling my OS is always, "thank you". It always replies "command not found" which is its way of saying, "Oh, it was nothing. Think nothing of it!"

Naturally, we're not always so sentimental. We like to joke with each other too. For instance, whenever I start a new terminal, it always gives me my fortune, which is usually something funny. Today, my fortune read, "You look tired."

Maybe my laptop is right. I've been giving so many talks lately, maybe it's time to take a break ;-)

Friday, March 23, 2012

PyCon: Parsing Sentences with the OTHER Natural Language Tool: Link Grammar

See the website.

When it comes to NLTK (the Natural Language Toolkit), some assembly is definitely required. If you're not a linguist, it's not so easy.

Link Grammar is a theory of parsing sentences. It is also a specific implementation written in C. It's from CMU. It started in 1991. The latest release was in 2004.

Link Grammar does not use "explicit constituents" (e.g. noun phrases).

It puts an emphasis on lexicality.

Sometimes, specific words have a large and important meaning in the language. For instance, consider the word "on" in "I went to work on Friday."

pylinkgrammar is a Python wrapper for Link Grammar. (Make sure you use the version of pylinkgrammar on BitBucket.)

Often, there are multiple valid linkages for a specific sentence.

It can produce a sentence tree. It can even generate Postscript containing the syntax tree. (The demo was impressive.)

A link grammar is a set of rules defining how words can be linked together to form sentences.

Words have connectors to the left and/or the right.

General rules:
  • Connectivity: All words must be connected (with some exceptions).
  • Exclusion: No two links can connect the same pair of words.
  • Planarity: If links are drawn above words, no two links can cross.
  • Ordering: When satisfying multiple connectors on one side of a disjunct, the ordering of the words must match the ordering of the connectors.
The software does some post processing in order to handle a lot of things.

The system has knowledge of 60,000 words and 170 (?) link types.

It's amazing how it can recognize the invalidity of certain sentences that are only very subtly invalid.

It can guess the purpose of unknown words.

It's pretty robust against garbage thrown into the sentence.

It's used as the grammar checker in Abiword.

Link Grammar is a powerful theory for parsing sentences. It has a comprehensive implementation. It's easy to use with Python.

The distinction between syntax and semantics is pretty blurry.

PyCon: Building A Python-Based Search Engine

See the website. Here's the video.

Unfortunately, I showed up late for this talk and didn't take notes. However, it was one of the best talks! If you've ever wanted to build a search engine like Lucene from scratch, I highly recommend this talk!

PyCon: Parsing Horrible Things with Python

See the website.

He's trying to parse MediaWiki text. MediaWiki is based on lots of regex replacements. It doesn't have a proper parser.

He's doing this for the Mozilla wiki.

He tried Pyparsing. (Looking at it, I think I like PLY better, syntactically at least.) He had problems with debugging. Pyparsing is a recursive decent parser.

He tried PLY. He really likes it. It is LALR or LR(1). PLY has stood the test of time, and it has great debugging output.

However, it turns out that MediaWiki's syntax is a bit too sloppy for PLY.

LALR or LR(1) just doesn't work for MediaWiki.

Next, he tried Pijnu. It supports PEG, partial expression grammars. He got it to parse MediaWiki. However, it has no tests, it's not written Pythonicly, it's way too slow, and it eats up a ton of RAM!

He wrote his own parser called Parsimonious. His goals were to make it fast, short, frugal on RAM usage, minimalistic, understandable, idiomatic, well tested, and readable. He wanted to separate parsing from doing something with the parse tree. Hence, Parsimonious's goal is to produce an AST. Parsimonious is very readable.

If you optimize a bit of code, write a test to test that the optimization technique keeps working.

In Parsimonious, you put all the parsing rules in one triple quoted string.

It has nice syntax.

See github.com/erikrose/mediawiki-parser/blob/master/parsers.rst for his notes on a ton of different parsers.

Proper error handling is a master's thesis in itself.

There are Python bindings for Antlr. However, he didn't want a separate code generation step, and it's slow.

You can do interesting things with sys.getframe and dis.dis.