The RCAs are front/rear outputs (left/right pairs) and an AUX input (left/right) - if you don't have it hooked up to an amp (or another music source in the case of the AUX input), then there's no difference signal-wise if you remove the lead at the back of the harness, or keep it.

/Michael