I don't have any Chromecasts, but I do have a GoogleTV and a TiVo. Both now support the YouTube-specific casting from the Android YouTube app. What happens, on my phone, when I hit the "cast" button, is that it gives me a pop-up menu with the names of my TiVo and my GoogleTV.
My guess is that this paradigm is how things are going to shake out.
Food for thought: if you've got a Chromecast installed on a shared WiFi segment (I'm thinking our university's WiFi network) then it's going to be visible to anybody. So far as I can tell, there's no way to do any sort of pairing. The Chromecast or GTV box will accept a casting request from anybody. This allows for an exciting new form of spamming: casting videos to randomly visible TV devices.
You heard it here first.