Next article: Friday Q&A 2009-08-28: Intro to Grand Central Dispatch, Part I: Basics and Dispatch Queues
Previous article: Reading Between the Lines of Apple's FCC Reply
Tags: meta python unicode
As some of you are aware, my comment system did not support non-ASCII characters. This is not because I am unaware that there exists a world outside of the United States. No, it's simply that I could never force the horrible combination of Python and MySQL to cooperate when it came to non-ASCII characters. I struggled with it for a long time, and finally gave up. Well, I'm happy to say that I have finally solved the problem and my blog now supports Unicode comments!
What was the solution, you ask? Easy: give up on MySQL and switch everything to SQLite. SQLite is fully Unicode-aware and makes everything related to non-ASCII text easy. I had to do some hacking in my Python code to get it to do the conversions at the right place, but it was fairly minor work. Dumping all the old comments over took a fairly short script, and now I'm up and running on the new system. Overall SQLite was really nice to work with, a big contrast from MySQL. I realize they're targeted at different types of work, but the fact is that with the low amount of traffic I get, and especially the small number of comments that are posted, I don't need the performance, multiuser facilities, or other capabilities offered by MySQL.
Anyway, it's done, it appears to work, so enjoy. If you see any problems, let me know. (Or post a comment if you can!)
Does ‮ work?
I don't parse entities. Everything you write gets HTML escaped except for a very small, predefined set of known tags. There's just too much guesswork involved as to whether you want unichar 8238 or whether you want ampersand, hash mark, digit eight, etc.
@astrange: Well utf8 takes the same storage space as latin1 unless you use non-ASCII chars.
Plus, SQLite is easier to work with and easier to back up, so, bonus!
It's one of many reasons why i never used MySQL, but under menace. I'm usually using Postgres when I need a full features, multi-clients database and SQLite for simpler needs :-)
Comments RSS feed for this page
Add your thoughts, post a comment:
Spam and off-topic posts will be deleted without notice. Culprits may be publicly humiliated at my sole discretion.