You really shouldn't be hearing anything when the unit is in standby (even as you're pulling it), as when the display is blanked, the amp remote line is off. It sounds like you have your amp permanently on (which I doubt, as you'd soon notice!). I sometimes get a mild pop if I pull the unit while it's playing, as the amp doesn't shut off quickly enough, but nothing to complain about.

Player state should be saved in either event. I assume it works correctly on AC power?

Perhaps the amp remote line is drawing a lot of current (measure it)? That would reduce the time available for saving state, and possibly account for pops when starting the car. If so, you might need to isolate it with a relay or a little transistor circuit. See if it saves state when the amp line is connected to positive instead of to the player.