Mine is getting too big, so I am doing more than just the artist and then sub-dirs for their albums.

I have a directory are:
1. oldies (60's and earlier)
2. 70's
3. 80's
4. Current tunes (includes artists that transcend decades - aerosmith)
5. Rap
6. Country
7. Vocal
8. Compilation Albums and Soundtracks
9. Test Tracks (treble, bass, pink, white noise, etc.)
10. Random tunes, unattached stuff, etc.
11. Favorite Albums and Playlists (at the end so it's easier to get to, includes stuff from other directories)

If I have a directory get too big, like my current tunes directory, then I'll split it up into alphabetical order (A-M, N-Z). I just don't like having too many subdirectories.