Yes.

You can sorta-kinda do this now using the always randomize playlist property, but, since it's always on and it only affects the playlist itself, not sub-playlists, you have to create a new playlist with appropriate albums in it.

It would be nice if there was a way to select playlists-with-no-subplaylists as the item on which to perform shuffles, but even that wouldn't necessarily work all the time.

Maybe it does make the most sense to just set it up beforehand.