I did some very extensive testing and found the reason for the hick-ups:
All my VBR files are corrupted for some reason. F**k! In a major way. And my f****ng backup is also ruined.

Thanks for the responses, sorry to have bothered you. I am going to drink a lot of beer and wonder whether I should recollect and rerip 20 GB of files.