If I may contribute, the vast majority of the energy consumed by a heating system, or cooling system, is simply to replace the energy transferred to the outside. The heating system does not 'heat' the house, it simply replaces heat that has been lost through the walls, windows, etc to the outside. Essentially all the money you pay to heat your house is being spent to replace energy loss, not make the house 'warmer'.

Imagine the temperature of the core structure of your house was maintained at a near constant temperature (which if you have a good thermostat and heating system, it is.) All winter long those interior walls and floors are at the same temperature. You never need to 'warm up' those building elements, as they never get cold. The interior surfaces of the exterior walls are of course not quite as warm as the interior structure, so the room air loses heat to those cooler wall surfaces.

The heating system adds heat to the interior air to compensate for the heat loss to the outside walls, and that added 'air heat' slowly but continually transfers to the outer walls, and hence to the outside world.

As the room air becomes cooler (such as when you 'turn down' the thermostat) the heat transfer to the outside walls slows, since the heat loss rate is driven by the temperature difference between the interior air and the outside air. During the time the interior air is cooler than your normal inside temperature, the heat loss happens more slowly. The longer the interior spends at this lower temperature the greater the difference in total heat loss compared to maintaining the thermostat at your normal 'comfort' level.

The slower the rate of heat loss, the less total energy needed to maintain the 'set back' temperature and then return the interior to the desired comfort level when you return.

If you turn the thermostat off, the rate that the interior loses temperature is an indicator of how well the building is able to retain heat, or how quickly it is losing heat. A well insulated building should change temperature very slowly with the thermostat turned off.