Why do new Agile teams find QA so Hard?

From AgileOpenNorthwest

Why Do New Agile Teams Find QA So Hard? Wednesday, February 10, 9:30am, Shaw A

Geoff Hewson, Software Productivity Center Inc. Email: ghewson@spc.ca Twitter: geoffhewson Web: www.spc.ca

Recap of do’s and don’ts for integrating QA with Agile

Do’s

Rule of three (all discussions include qa, dev and product owner)

Non-exclusive specialization

Use a “verificationeeer”

Acceptance Test Driven Development

CI and test automation

Don’t reach for the sky...

Don’ts

Keep dev and test in separate teams

Use a single person to represent the testers

Focus on local, role-based optimization (i.e. trying to get the most out of developers without considering the output of the whole team)

Too much reliance on manual testing

Hurdles/Barriers QA Teams Experience

• Functional matrix resourcing models create barriers – individuals still identify with their functional role, rather than the whole team (e.g. it’s not my job to do testing, I’m a developer)

• QA resources not being involved in important discussions between dev and customer

• Out of scope work (e.g. bug fixes)

• Trying to test everything is too much work

• Developers continue to work on new features at the end of the iteration, even if there is no way they can be tested

• Not enough testers to do all the testing that is needed

• Management wanting to revert back to old ways when the going gets tough

• Some people are not comfortable with the changes Agile demands

• Lack of automation engineers, or separate from the project team

• Recording manual tests produces brittle automated tests

o systems need to be designed to be easily testable

o There is a project at MIT that is building technology that can recognize UI elements with visual fuzzy logic

• All the great things you need to be able to do to be good at testing in an agile team appears to be a daunting, insurmountable effort

Thoughts and Ideas

QA in agile teams

• You will always need some manual testing for exploration

• Agile QA includes defect prevention too (e.g. design reviews) as well as testing)

• QA is still an essential function because customers don’t usually think of negative test scenarios (only positive)

• Kanban boards can help resolve functional matrix problems by focusing the whole team on what needs doing to finish the iteration

• When QA is sitting next to dev, dev gets better at self-policing because they hear QA’s reaction to sloppy code

• Test case and design reviews involving dev, QA and customer really improves the quality of delivered features (catches more scenarios). In time dev will start pointing out additional things that need testing

• Is pairing 1 QA + 1 dev better than pairing 2 devs? Can depend on culture, but the group thought it IS better. What about BA + test, BA + dev? Getting the pairing mix right is a subtle management skill

Transitioning to Agile

• Get help from someone who knows how to do it when you start out (e.g. setting up and using CI/automation tools)

• Early on you will need some “enablement” stories – things you need to set up (e.g. Fitnesse) so you can deliver features better

• Gelling the team really drives improvements. Coders don’t appreciate the nuances and richness added by QA, a paradigm from the usual dev viewpoint of positive testing

• The culture clicks when the tests are seen by all as a shared resource (collective test ownership?)

• Don’t reach for the sky on day one: means take it incrementally, baby steps (e.g. can you build the system every check-in? Then run a suite of automated unit tests at check-in. Then run a regression suite of automated acceptance tests daily, etc, etc)

Management

• Size and explain the impact of out-of-scope work as part of your retrospectives, especially those that impacted your ability to finish testing

• Need to leverage risk-based testing and rely on targeted automation to constrain the maount of QA work to fit within an iteration

• Use Kanban to optimize flow, limit Work-In-Progress, to prevent developers building more features than can be tested

• Forcing to much change on the team (from the top down) leads to process shock, leads to teams resistance. Management may need to hold off on some process improvements until later

• Management and staff need to know that failure IS an option within an iteration, so long as you learn from it and fix it

Open Questions

• Is testing one iteration behind dev good or bad? Under what circumstances?