Yes, you can use your multimeter...but firstly;

At least one of your capacitors is an electrolytic capacitor. Electrolytics are polarised, ie it matters which way around they are. I hope (and think from what I can see in the photos) that you noticed the grey stripe with the '-' sign on it. The lead nearest this stripe needs to be at the ground connection. If this is reversed, then it will probably explode - probably wouldn't damage anything besides itself, but it would make a nice bang and stink. One other thing to check on the electrolytic - they are voltage rated too. As long as it's 16V or above you're fine. It's probable that this is the case, but check anyway. Typically the same result if it's wrong. I think that the second cap is unpolarised, although the photo isn't clear. If it is polarised, then it should be clearly marked like the electrolytic is.

After that's been checked;
Verify that the regulator's pins are connected correctly. There should have been a pinout on the package. Note that the tab is not electrically isolated, and is connected to Vout (middle pin), and not ground. That means that you have to ensure that it is eventually isolated from the car chassis somehow. For the time being, just make sure it doesn't touch anything during testing.

Set the multimeter to measure resistance, measure across the pot connections and roughly set the pot to 870ohms.

Look to see if your multimeter has a continuity test. This is usually shown by a buzzer symbol.Test the mode by touching the probe ends together - you should hear a buzz.
1) Move the switch to the 'stock' position. Test that the MAP connection and the 'stock wire' connection are connected together (buzz)
2).. and that neither are connected to Vout (the middle leg of the regulator) (no buzz).
3) Move the switch to the 'tuned' position, and now verify that the MAP connection is electrically connected to Vout,
4) and that neither are connected to the 'stock wire' connection.
5) (Still 'tuned') Now check the LED is connected to the 12V connection.
6) Move to 'stock'. Check the LED is not connected to the 12V connection.
7) Change the multimeter to resistance mode again. Measure the resistance between Vout (middle leg) and Ground. It should be ~490ohms. If it's out by more than 5% then verify the pot resistance again.

Assuming all is good up to now;
You ideally need to find a 12V source somewhere to test this. Ideally it will be protected by a fast acting .5A fuse, but we don't live in a land of ideals.
I'd use an old PSU from an 'AT' type PC. Actually, I lie, I bought a new 'AT' PSU as a 12V supply! PC power supplies are useful as cheap power supplies for us geeks because they can typically supply large currents at 5V and 12V, both of which are very useful. An 'AT' type is more useful than ATX because it doesn't require any hacking to get it to turn on (ATX expect a motherboard to be there).
Actually, as it stands, you should be able to test it with a 9V battery, which should be nice and safe. Note that we are at the limit of the regulator here - it really wants >3V difference between Vin and Vout, but as long as the current draw is low (which it should be) then it should work.
Anyhoo, find a 9-12V test source, and connect the 12V and ground wires appropriately.

8) If there is a sudden burning smell, or you notice anything getting really hot, or melting, or going bang, disconnect immediately. (Goes without saying really )
9) Set the multimeter to measure DC Volts. Measure Vout (with respect the Ground). It should be very close to 6V. Verify that tweaking the pot does indeed change the voltage, and then set it to be whatever the Jeep enthusiasts deem to be ideal. ( ~6V IIRC..)
10) Verify the LED works If it doesn't, but all the previous tests passed, then the LED is probably wired in reverse.
11) Disconnect the 12V wire. Change the multimeter to measure DC Current (mA). You'll possibly need to move the probe to a different hole in the multimeter as well. Now connect the multimeter between the 9/12V source and the 12V wire, ie;
9V/12V -> MM -> Circuit -> Ground.
Measure the current drawn with the switch in 'stock' mode. It should be about 12mA, give or take.
12) Measure with the switch in 'tuned' mode. It should be about 30mA, give or take. (This depends heavily on the LED specs. As long as we're not over 40mA, don't worry. Otherwise post the specs, so we can confirm that it isn't a circuit fault).

If you've got this far, then it looks like it's all good. It's probably a good idea to use an ignition-switched 12V supply to avoid that 10-40mA drain on the battery when the car is off, but not absolutely essential unless you leave your car for weeks at a time. Remember to ensure that the regulator tab remains isolated from the car chassis. Reliability of homebuilt circuits really relies on the final finishing touches - it will last much longer if put in a plastic case and components are suitably affixed to prevent vibration damage.