I saw it the first time and got the count right. I didn't understand the point of the exercise until I read all the spoilers though!

Here's my theory.

It's just your brain filtering out unimportant data. When I'm reading a webpage, my mind filters out useless banner ads on the page. The unimportant data doesn't contribute to the success of the mission (of counting the number of passes) so it never enters into your conscious mind.

However, as I was watching this video for the first time, I was much more perceptive to the entire scene due to jbauer implying the strangeness of it. In addition, the "Visual Cognition Lab" page title caught my eye. So my mind really had two missions: (1) to count the number of passes and (2) to figure out what the "trick" is. Thus, all data in the video was deemed important by my unconscious (or preconscious?) and was passed on into my conscious for analyzation.