So 10.5.2 didn't fix the Sync Server issue, but I found out the cause. I've basically kept the same Address Book data now from 10.1 and across multiple machines. I've used 3rd party sync tools with it when I've had multiple machines, and countless apps have integrated and touched the files at some point. And it seems adding .Mac sync in pushed it over the edge. In ~/Library/Application Support/Address Book/, the main database file was ~25 megs. I found that you can delete all files in that folder, except the Images, Metadata, and MailRecents files/folders with no harm. When Address Book was relaunched, it rebuilt the database and was a small 350k file. .Mac syncs then ran quickly, and with little CPU interruption.

Whatever breaking point I crossed started manifesting as crashes after a little bit in any app that touched the database. Skype, Safari, Adium and others were crashing, and when I looked at the crash logs, the addressbook framework was what caused it.

Things like this really make me think Apple needs to add some sort of checker to the system to purge out of control databases. I've had wonky Mail issues over the years due to similar issues. It would make cleaning up problems like this much easier.