I just rub the silicone with my thumb/fingers, and it eventually all rolls off. Rough textured leather gloves also work for me.