This really is a technical area. Defining a speaker as 250W RMS is misleading at best. You could define it as 250W RMS @64-120Hz +-0.5THD with a duty cycle of .5sec, for example. Manufacturers rarely give the full specs out.

These are the important parts of the test. The total power output at a specific frequency range falls inside a tolerance of distorsion for a set period of time.

Physical excusion of the speaker will kill it; that is, too high a voltage going in. Sending a high signal for too long is the same thing. The higher the voltage, the shorter the burnout time.
Reproducing distorsion can be done quite well by most speakers, usually the distorsion is caused by an under powered (or overdriven) amplifier. In this case the amp usually burns before the speaker does. If there aren't obvious signs it was the amp, the speakers are thrown out in error.

Some speakers can handle a high RMS for a short duty cycle but can't win any competitions because they can't deliver over the required time.

Improtantly, having high powered speakers and amps doesn't mean you have to have a stereo only playing loud, it may mean that you play things softly but with less probability of distorsion. But this then brings in a different array of arguments