The greater the temperature difference between inside and outside the more energy needed to maintain that difference.

By lowering the temperature difference when you are not home you are lowering the average temperature difference. A lower average temperature difference requires less energy.