Put backslashes into the filespec in the configuration box. Like this:

{genre}\{artist}\{source}\{artist} - {source} - {year} - {tracknr:2} - {title}{ext}

That's if you're downloading from the soup. If you're downloading from playlists, it should be making folders that equal the playlist names already.

Also, substitute {pos:2} if you want it to go by playlist position instead of the track number field (which is not always filled out).