Originally Posted By: JeffS
I guess the big question is, how does QA validate the binding code? The notion that they may not be testing what they think they are testing due to a bug in the binding code is pretty scary.


There's a couple of things that work here:

1. Coverage -- if you've got enough tests checking enough scenarios, gaps in the tests will tend to be fairly obvious. This is backed up by a certain amount of exploratory testing. It's not perfect: we had a bug in our tests for a couple of iterations before we noticed. Fortunately, the bug was only in the tests; real-world usage wouldn't have triggered it. If it had been a real-world bug, I'm fairly confident that it would have been spotted sooner.

2. Technically proficient QA. Our QA guys are probably marginal at actually writing the bindings, but they're perfectly happy to review them.

3. Code reviews. We have a rule that nothing is allowed to be checked in without being reviewed. Obviously, it's actually more like a guideline, but we're pretty good at following this rule. Pairing supplements this for the trickier bits of code.

Quote:
Other less important questions- when are these BDD scripts created? I could see QA doing this while devs are working on the feature, meaning that there is a lot less strain on QA near the end of the sprint.


This is exactly how we do it. And the lack of strain on QA at the end of the sprint/iteration is awesome to behold.

Quote:
Are you using TDD and this testing is supplemental? I've never really understood if BDD is supposed to replace or supplement TDD.


It's supplemental. Define behavior in BDD. It fails. Write unit tests (TDD) for those missing features, make those pass, refactor. We've tended to slack on the unit tests, which I think is hurting us slightly.

Here's a cool slide deck: http://www.slideshare.net/drumwurzel/getting-comfortable-with-bdd

You will require more dev time, but that's balanced by having the whole automated testing thing, and also frees your QA guys to do more in-depth exploratory testing. It also means that your QA have more time to get involved earlier, which means that everyone's on the same page for what you're building.

Writing the BDD tests properly is a bit of an art, and takes a while to get comfortable. That said, you can be fairly productive with it quite quickly.
_________________________
-- roger