The pins themselves will break off from fiddling and flexing before your solder joints fail. That's what you have to be careful of.

Usually an IC like that would be soldered to a circuit board to prevent that problem, and the cat5 would go into an actual RJ45 plug. But as long as you're careful not to break the pins, there's nothing wrong with doing it that way either.