Aside from H&H AoE ( which is _nowhere_ near as mathematical as my first year of university electronics was), I'd also highly recommend Forrest Mims' books. He has a couple of introductory books and also some 'engineers notebooks' which are excellent for learning some basic circuits. Radio Shack have carried some of them for many years. I pretty much got started with a Radio Shack '50 in 1' electronics experimenters kit and a couple of Forrest Mims books when I was about 10.

In fact, get yourself down to Radio Shack and buy the Electronic Learning Lab. For $50 you can't go wrong - the (Forrest Mims) documented experiments will be designed around the kit so you won't have to worry about sourcing any components immediately, Once you get beyond those experiments the kit can still serve as a basis for further experimentation or prototyping as it includes a breadboard.