I actually took the easy way out and used Carol's Dell laptop to run the Setup Wizard CD.
It was a good step by step tutorial telling me to unplug the existing connections and then guided me through physically connecting the cable modem to the router and then Stygian up a wireless network name, a password and security. I guessed at the security type though since there was no explanation as to what each security method meant.
Let's do a new speed test and see if there's any difference:
Speed Test 3:
Wired connection to Linksys WRT54G. This router supports speeds up to 54Mbps.
Sweet. Twice the download speed.
It's right at the same speed I would get if I was connected directly into the modem.
Speed Test 4:
Wireless connection to Linksys WRT54G.
Hmm. The packets seem to be flowing nicely!