It behaves differently if you have favorite visuals enabled (it's a setting in config.ini) If you set that, then holding the visual button switches between Favo(u)rite visuals and All visuals modes.

Maybe the info mode display is just an oversight? I think if you don't have favo(u)rite visuals enabled, holding the visuals should do a "previous visual" which currently only exists for Sony stalk owners, apparently.