Half wave or a smaller cap will only increase the ripple on the rectified DC. A larger cap will decrease the ripple, but increase in the initial inrush current when switching on.

The peak voltage of 28VAC RMS = 39.5V peak minus the diode drop in the rectifier diode(s) will still exist either way.

As for the original zener idea, I would put something with some margin over what's normal. Something like 42-43V perhaps. The voltage regulator should be able to run with that. The zener should be thought of as a last resort protection really.

If your current consumption is reasonably constant, you could just use an appropriately rated resistor in series. The voltage drop is proportional to the current. This is not efficient though and the resistor will be just burning that power.

Some numbers:

Say we look for a 5V drop.
You mention 100mA-250mA (at 5V).
Assuming say 70% total efficiency on the buck regulator/rectifier means about 500mW-1.25W output or 700-1.8W input power.
At 40V that's ~20-45mA. 5V at 20mA is 250R. At 45mA it's ~100R.
Sooo.... let's put a 100R resistor in series.
At 20mA that's 2V drop and only 40mW dissipated in the resistor
At 45mA that's 4.5V drop and 200mW dissipated in the resistor

You need to consider your worst case power consumption since that will decide how much is dissipated in the resistor.