Also, until very recently, Americans were able to travel to Canada, Mexico, and many Caribbean islands with nothing more than a driver's license.

I got my passport when I was in my teens, but it was years before getting a stamp in it despite frequent travel to Canada and the Caribbean.