All the Rails security guides say that you should call reset_session after the user logs in or logs out. This clears out the session and forces a new session ID to be created. It seems there have been a few Rails bugs related to reset_session over the years.
In my login action, I call reset_session and then put a nice message in flash. When I actually use the website, I can see (via Firefox) that I'm getting a new session ID, and I can see my flash message. However, when I write tests for those two things, the flash message gets lost, and I don't get a new session ID in my cookies. It's almost as if the new session is being ignored, and the old session is being used.
I submitted a bug to Webrat about this, but it turns out it's an issue in Rails. This issue is present in version 2.3.8. If this is affecting you, there's a workaround here. I implemented the workaround, and it worked like a charm :)