iDVD (all of iLife) only comes with the OS disc set for new computers, it isn't included on the retail OS. After installing Leopard from scratch (a retail copy since my machine didn't ship with it), there's still a bunch of software I haven't re-installed, including any of the iLife apps.

I'm still very curious why the ffmpegX transcoded VOB files have this issue with the DVD player.