You can do the grounding anywhere along the cable - on the connection to the portable player, etc. The shield is the outer of the RCA socket, left and right grounds are connected together.

Hugo