After narrowly escaping Encoding Hell, I fell into Case-sensitivity Hell.
I work with data in big batches. This time around, I was starting with an empty database and importing some data. I noticed that it said that some rows were deleted, which is a funny thing to see when importing data into an empty database. I came up with a plausible explanation which I won't get into and just moved on.
During my batch processing, I pull down all the URLs from a table into a Python dict. My program was getting a KeyError because it couldn't find the URL "http://www.example.com/layout/keywords/Broken%20heart". I did a select statement in the MySQL shell, and I could see it.
Hmm, that's weird. I looked at the code for pulling down all the rows, and it looked fine. I did a count(*) on the table, and it matched the size of my dict. Very weird. I couldn't figure out why my dict didn't have the URL even though it should clearly be there.
After an agonizing, multi-hour debugging session, I finally explained the situation to my wife using an analogy, and she gave me a solution for my analogy.
I finally figured out that since the column was a TEXT column with the database-wide COLLATE set to utf8_unicode_ci, it was returning "http://www.example.com/layout/keywords/broken%20heart" even though I had asked for ""http://www.example.com/layout/keywords/Broken%20heart". See the difference in b vs. B? I didn't ;)
Then it dawned on me, MySQL was "deleting" rows during the MySQL import because it was ignoring URLs that only differed in case. Later, it was giving me one URL in my select statement even though I had searched for a different URL that differed by case. However, my Python dict was definitely not case insensitive, which is why I was getting KeyErrors.
Well, how do you tell MySQL that you want a column to be unique in a case sensitive manner? It turns out there is no COLLATE utf8_unicode_cs (cs stands for case-sensitive). Instead you have to use COLLATE utf8_bin which causes the sort order to behave weirdly as explained here. Apparently, Case-sensitivity Hell and Encoding Hell are next door neighbors.
Ugh, painful. The sad thing is that if you went back in time 30 years ago, you could probably find some other programmer griping about the fact that he just spent a day in Case-sensitivity Hell ;)