i.e. even the internal battery is drained. Does it remember the coordinates in that case?

If the hardware is like the Earthmate, yes (that or whatever backing store they use uses a very small amount of power and I never left it off long enough to matter, but it ran from AA batteries and powered off when the serial port wasn't open, so it seems like they won't try to charge something from those).

But maybe the hardware isn't.