July 2009 Monthly Meeting Summary

Topic:
Test Automation Management via ROI and Scorecards - Rick VanKeuren, VeriSign Inc.
* Automation tracking and management via scorecards
* Determining automation Return on Investment
* Other issues in managing and tracking test automation

Bio: Rick has worked in the software industry for over 18 years. He has extensive management and software engineering experience spanning a variety of roles but focused primarily on software development and testing. Rick is currently a QA Director with VeriSign where he manages a team of QA Engineers and Analysts responsible for the software quality assurance of the company's naming registration systems that include all com, net, edu, and other top level domain registrations. Rick holds a B.S. degree in Computer Science from Texas Christian University and an M.S. degree in Technology Management from George Mason University.

Took place on: Wed. July 8 2009 6:30 PM

Attendance: 14

Meeting Notes:
Rick VanKeuren presented on some innovative and interesting approaches to test automation management, and there was extensive discussions at the meeting about challenges and experiences of other attendees and pros and cons of various approaches. A 'scorecard' approach to tracking and managing were presented, including tracking long term and short term targets/goals, quarterly metrics, and semi annual resetting of targets. ROI reports and calculations were discussed; such calculations were noted as providing visibility re QA efforts in a language that senior management is often more familiar with. Some points made/discussions included: that trends and relative values were more useful than specific numbers in metrics; that target automation 'coverage' could vary from project to project or business unit to business unit depending on a variety of factors; that terms like 'coverage' and 'test case' etc can have varying meanings; maintenance of automation suites can be a challenge to track and manage.
Some questions discussed also included:

• How 'successful' are your automation efforts?
• What does 'success' mean?
• What should your priorities be?
• How can management keep track of the progress of automation efforts?
• What kinds of metrics can be used to track/manage automation efforts?
• What kind of information is helpful in justifying and priortiziing automation efforts?

 
 
 
     NoVaTAIG Home Page

Northern Virginia Test Automation Interest Group