I don't know if this is something specific for my install but I don't believe it is. If I try to an apostrophe (') in the title of my blog I get the following error:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 's Photoblog' where admin='admin'' at line 1
I don't know a lot about programming, but I assume it has something to do with the way the string is parsed in to the SQL statement. Meaning when it sees my (') in the title it thinks it the end of the text, however since there is more after it, the command then errors.