My player also makes this noise. I first noticed it when I was listening to it at home through headphones. I can't hear it in the car, nor at home when I listen through normal speakers so it doesn't bother me much.

I'll try what Tony suggests : use the compact top adapter (right now I'm also using the brick type), and see if this solves it.