For this answer, I will briefly channel my inner asshole (a.k.a., my father).

You don't owe this guy squat. You've already bent over backward to accomodate him and paid for things where the two of you agreed to share the cost! Your real goal seems to be to make this guy get out of your hair, once and for good. Tell him that he can either do what you want (pay your half of the fence, etc.), or you'll help the judge to laugh him out of court. You may find it entertaining to discuss this with the most level-headed tone you can manage while your neighbor presumably blows his top in responding to you. (Score yourself a point for staying calm while he loses his.) Most likely, you reach no agreement. Go to court with photographs, receipts, documentation, and everything else. Tell the court that you've more than held up your side of the bargain, despite his not keeping up his end. However, your primary interest is not to collect but simply to get this guy out of your hair. Most likely, you get a judgement to dismiss the case, and the judge tells off your neighbor for you.

If you want to really channel your inner asshole and teach your neighbor a lesson, you play the counterclaim game and try to get a judgement to that effect. The upside is you get the smug satisfaction of seeing him get slammed. The downside is more effort on your part. I wouldn't worry too much about lingering animosity between you. If anything, hitting back hard will make your neighbor back off on the next perceived transgression, as you've shown yourself to be one who is not easily pushed around. He's picking on you because you're an easier target than the builder.

(Had my dad been your neighbor, he'd have filed suit against the builder and gotten them to not only fix the drainage problem but probably build him a toolshed or something gratis for his aggrivation. It amazes me how my dad can squeeze things like this out of people. Needless to say, he doesn't care too much for leaving hard feelings.)