I spent 4 years in college (I guess you call that University in the US). Usually, it takes 3 years to graduate: The first two years are general electronics (from surface mount devices to power electronics) and the last one is specialized in 1 of 8 areas (A/V, telecom, industrial electronics, computers). I got out with 2 specializations: electrodynamics (motor and power electronics) and instrumentation & control (industrial process measurement & control).
So I guess you'll first want to learn the basics: what do each type of "low level" component (resistors, transistors, diodes, cap and such) and master the basic concepts of electronics (voltage, current, power, frequency, capacitance). Then, you can go higher and begin to look for basics circuits and their behaviour and for more complicated components (ICs like TTL, op-amps)
From that point, you'll get the basic idea: the rest is a variation on a theme. Every large circuit is understandable, if you look it by small chunks.
Sorry, I can't really point you to books or reference, as I mainly read French... (unless you need another challenge :-)
I understand what you mean. It's really what I enjoy in electronics: being able to make "mechanical", "physical" devices really do something. There's nothing like starting a fan here or logging wind speed in a computer with stuff you made yourself, at twice the price it would have cost if you had bought it :-) and 100 times the time. :-)
_________________________
Patrick