I have got to figure out a way to get to the stuff I want to listen to without so much scrolling.

You've tried the alphanumeric search features, including search by playlist? And using the knob to scroll the playlists menu (it's faster than L/R buttons) ? I've found that between those two things, I can make short work of even a very flat playlist structure.