Does it misbehave when you're making a phone call? A three second latency for music is annoying. For phone calls, it's unusable. That might be your angle to fix this.

(Analogy: back in the day if your phone line had crappy audio quality and it rendered modems unusable, you'd complain that your fax machine wouldn't work, otherwise they'd laugh at you.)