Telnet interprets some byte sequences as internal commands. I don't know if zmodem can be configured to avoid those sequences or not. Also, scp is much, much easier to deal with.

Honestly, I've always had a lot of trouble doing zmodem over the serial port since there's no flow control enabled.