Yup, that's the problem, your player is wired incorrectly. The amp remote wire should not be connected to the ignition switched wire.

I'm surprised the player works at all. There should be a whole host of other strange things happening with it above and beyond the LED problem. I'm amazed you didn't blow a fuse or something.