Did you see anything come over the serial port when it booted properly? If not, then there's something wrong with your serial port setup. It might be that you're using a straight-thru instead of null-modem cable. It might be that you have an app already listening on that serial port. It might be that you didn't set the option properly in hyperterm.

But if you don't see anything and the player works at all (that is, there's not smoke coming out of it), then it should produce something from the serial port. Take my word for it. Without that data it's unlikely that we can do anything to diagnose your problem more than take guesses.