Windows grobbles around in the headers of the binary

Footers, technically. The VERSIONINFO block is stashed with the other resources (menus, dialogs, etc.) towards the end of the file (though it doesn't have to be).

The Win32 executable format also has space in the COFF header for a file version, but very few applications bother to fill this in.