June 2010 Monthly Meeting Summary
Topic:
Test Automation Coding Best Practices
Roundtable discussion facilitated by Keith Bennett, Software Developer at CSC, and Rick Hower, QA Manager at CSC
Potential sub-topics were:
* 'Best practices' - or 'Best practices in context'?
* Naming conventions
* Source control
* Module organization
* Coding style
* Cohesion/coupling
* Error handling & recovery
* Commenting and documentation
Took place on: Wed. June 9 2010 6:30 PM
Attendance: 13
Meeting Notes:
- In some projects its appropriate to use existing automation frameworks, in some it may not be or they may not be available,
and in some it may be best to build on an existing framework. Some insisted that every automation project should start
with one of the existing frameworks, while others siad that's not possible in some contexts.
- Best practice suggestion: it's helpful to consider who is the consumer of the automation output - for example dev 'consumers' may only want to know
about failures including details, while management may want to know the number of passed and number failed.
- Best practice suggestion - write automation code in such a way as to make it easy for testers to add tests without knowing details of all the automation code
- Difference between software code in general and automation code - 'customer concerns' won;t be as important for automation code
- Best practice suggestion - keep things simple
- Project size affects best practices considerations - for one-person automation teams, standards may not be so important
- Best practice suggestion - code backups, source control, documentation
- For multi-person automation teams, agree on best practices as a team
- There was a suggestion that automation functions/methods should only return a Boolean value
- It was suggested that in automation projects its often a good idea to have an early/initial demo for marketing' purposes - to build support for the automation effort.
- Best practice suggestion - include or build robust logging/reporting capabilities
- Best practice suggestion - in automation code or even in coming up with 'best practices' consider unintended consequences, assumptions, and the practice of 'least surprise'
- Best practice suggestion - write code with a primary intention of not just functionality but the intention of its being read by other people
- Best practice suggestion - for documenting code consider use of wiki's, n\good naming conventions, code comments, and consider the issue of keeping it up to date
- In automation code consider the question 'does a test do what it should?' and 'how do you know?'
- Best practice suggestion - consider including very extensive logging capabilities in the automation code to enable easyier problem analysis when tests fail or
when there are automation problems; among other things this can help determine if a test failure was real or due to automation code problems.
- Best practice suggestion - as part of QA practices for automation code there should be random checks/verifications of automation code capabilitieson a periodic basis over time.
- Best praqctice suggestion - consider how to handle prioritizing of automation tests to make it easier to run quick small test runs vs long full end-to-end automation runs.
- There was some discussion about the past NoVaTAIG meeting on Test Automation Frameworks
NoVaTAIG Home Page
