You're recording it in medium-quality VBR (-V 5, --vbr-new) with the base bitrate (-b) and ceiling bitrate (-B) both at 96? That seems odd and it might be confusing lame. Try:

-b 96 -m m -h --lowpass 15.4 --cbr -q 0 --cwlimit 10

In fact, you might just want to try a basic:

--cbr -b 96 -m m

to make sure it's doing the right thing at all.