I assume you have a mechanical drive as well for data storage? I'm not saying that's where your downloads folder is mapped, but recently I've noticed something odd on my own Windows 7 installation. When I do certain things, I can hear one of my disks spinning up, and I can tell that the computer is waiting for it to do so before it completes the action. It does this for weird stuff like emptying the recycle bin or opening Notepad. I have no idea why.

I don't have an SSD for my main drive, but I do have one of those WD Raptor drives, which is pretty fast. Regardless, I'm not sure why my computer feels the need to spin up a dormant drive for such trivial stuff that should all be in the main drive anyway...