Agreed on replacing them. It's usually extremely easy (turn the chair over and yank out the casters with your hand).

I think these are the kind I bought. I got them because my big chair and big self were creating grooves in the hardwood floor. Plastic mats kept falling apart. These casters fixed the problem and made rolling much easier. The only potential tricky part is matching the post size.