Using a regexp to tokenize?

Is there a way to write a regular expression such that it will produce a match for each of a series of repeating tokens.

I'm trying to parse a query string into a set of name,value matches. These are my test input cases:

a = b
&a=b&
a=b& c=d
a =b&c=d&
a=b&c= d&e=f

I'd like to produce matches of (a,b) or (a,b,c,d) or (a,b,c,d,e,f). It's obvious for a human to see what the desired matches are.

The following regexp will match against the first token of each example and give the correct matching substrings.

[& ]*([^= ]*)[ ]*=[ ]*([^&= ]*)[& ]*

But is there a way to make the pattern iterate over the entire input and return multiple matches. I could write controlling logic to walk through the input string but it would be easier if I could make the regexp engine do it.

Thanks.

I don't thnk the regexp can do all the work in the regard your thinking- but why not use a replacing regular expression to replace all the matches with commas "," and then enclose it the resulting string in paranthesis.

What language are you using? This is fairly trivial using perl:

        perl -e'while(<>){@m=/(\w+)\s*=\s*(\w+)/g;print "@m\n"}'

It gets a bit trickier (but not a whole lot) if you want to allow more than word characters (alphanumerics and underscore) as your a and b:

        @m=/\s*([^&=]+?)\s*=\s*([^&]+)/g

And, if you're sure you'll always get pairs, then you can assign that to a hash, as well, so you automatically have key => value pairs.

edit: Note... the while() is just looping over stdin, so you can input your test cases, not looping your regex over the input. That part is taken care of by the /g modifier.