VPC 3, which is scheduled to ship in September, will support USB, and PCI->USB cards can be bought for as little as $30. The VPC FAQ states "extensive USB device support", which may be something different than "complete" support. We'll have to wait and see. The current version of VPC has decent serial port support though (lots of people use VPC to talk to their PDAs), so it should be possible to upload to the empeg that way.
Empeg is supposed to ship with basic Linux downloader software, as well as the specs for the transfer format, so LinuxPPC/MkLinux might be a better option. I'm sure it won't take long for a GUI Linux downloader to be developed.