The short answer, yes. I, personally, can tell the difference between 128, 160, 192 and 256 - each one has a noticible decrease in the amount of artifacting in each sample - that's the effect where it sounds tinny or has a "digital" effect, mostly in the higher ranges.
For example, the sample that ships with the unit is the worst - it was so bad it I couldn't delete it fast enough. Not because it wasn't good music, but because it was horribly distorted with artifacts.
As for file size, the VBR files I generated in my test were about a meg larger than the ones encoded at 128kb, but the difference in sound quality is well worth the extra space.