IIRC you need to set it [the thread] as a favorite... There's a checkbox on the Reply To page that let's you set this for the thread. Plus a setting in your profile to receive mails for favorite threads.