Looking at the stock firmware on another WRT54GL in the Singapore company apartment I'm staying in at the moment, shows no evidence of client bridge capability. I'm quite sure you'll need DD-WRT or other 3rd party to do it on at least the "client" side.

Also if you have 4 wired clients or less on each side, you can use the inbuilt switch and save a piece of hardware. In fact with DD-WRT I'm 99% certain you can connect the "WAN" port to the switch fabric internally via software making it a 5 port switch at the expense of it's "router" function which you won't be using from the sounds of things.