I am not really an expert in radio or anything, but it sounds like it is more the filter issue than a modulation technique issue. If the signal was understandable, then it was using frequency modulation to 'decode' the FM signal. If you use AM on an FM signal, you will not get anything that makes sense.