Yep, as Tony says - that's a ground loop. Not having to plug USB in all the way is a tell-tale sign - the shield touching is enough to hook the earth of the player into the PC's earth.

Ethernet is isolated (all ethernet devices are) hence doesn't suffer from this way.

The RCA out audio numbers are better than the ones via the headphone out (no distortion from the headphone amp) - maybe it's just a listening preference?

Hugo