Can I assume you know what the SSID of the wireless network generated by your WRT54G2 is, and what the WEP or WPA key is? If so, that is what you need. Basically, your WRT54G2 is defining the specs of the network you want to propagate, and where it takes internet connectivity from is not relevant. You want to simply extend the reach of the wireless network signal you already generate.

To this purpose, you need a signal repeater. A signal repeater is easily obtainable by using a DD-WRT firmware (which offers repeater functionality) on compatible hardware (namely, WRT54G). So, I would suggest you buy off eBay a WRT54G (and not G2, which seems to be less compatible with DD-WRT), and install DD-WRT firmware in it.
DD-WRT is in fact an alternative firmware that can be installed on many Linksys (and other brands) routers to obtain more features.

So, get a WRT54G, download DD-WRT, update the router firmware from the proper firmware update page within the router web interface, reboot, and you now have a completely new web interface with more features.

Among those features, locate the signal repeater one, and, if I remember correctly, you should be able to enter parameters of the wireless network generated by your WRT5G2 router (SSID, WEP, probably channel used), so that the WRT54G+DD-WRT can pick it up and propagate it to the apt below you.

Now, while the above may be correct conceptually, I can't remember the menu structure of DD-WRT, and I may be forgetting some steps in configuring the repeater. I'd like to give you a more step-by-step guideline, but I don't have DD-WRT here.

In any case, I hope you now have a better idea of what to do.

I use a WRT54G with Tomato firmware (not DD-WRT) and currently it works great. It also has the repeater feature, so you may use Tomato as well.