Ok, this irritated me enough today that I bumped RAM up on the VPS to 1gb total. Won't have time to look at optimizations for a while, so I'll let money fix the problem for now. This should hopefully help, as there is now enough RAM for the entire DB to sit in, and still leave room for my mail server and other processes on the box.