July 2010 Monthly Meeting Summary
Topic:
Streamlining Automation - Working With Dev Toward More Efficient Test Automation (roundtable discussion)
Potential sub-topics were:
* Software architecture- planning for test automation and system wide conventions
* Software configuration design and parameterization
* Installation planning and design
* Well-defined API's/interfaces
* Cross-platform considerations (path handling etc)
* Unit testing- better design for testability
* Web page coding considerations
* Reflection
Took place on: Wed. July 7 2010 6:30 PM
Attendance: 14
Meeting Notes:
- There was energetic discussion among the developers and testers present on this topic, with a wide variety of useful ideas and experiences;
some ideas were met with wide agreement while some resulted in some interesting disagreement.
- There was discussion of keeping application layers separate (eg, business layer, presentation layer) then the business logic testing can be
easier to test separately via automation
- It ewas mentioned that for web services test automation, thorough descriptive WSDL's are helpful
- There was discussion of test result reporting and how critical it was to making effective use of test automation - mention was
made of using Google docs and spreadsheets, Jasper Reports open source tool, iReport open source report designer for JasperReports,
and the Spreadsheet::WriteExcel Perl module
- In the SUT code, remember to keep methods written such they are more amenable to unit testing
- There was a suggestion to include test automation code in the build process
- Set up the build process to run every time code is checked in (eg, Cruise Control, Hudson, Maven, etc) with automated test runs each build
- Automation tests have to be solid so that the automation code is not breaking a build
- There was a suggestion that it may be best to keep automation code separate from build process
- There was much discussion re how advantageous it is for the SUT to have a good logging framework with good informative log info, including
such things as queueing info, etc, and solid exception handling and error codes. This enables much more efficient analysis of test automation results analysis
and tracking down issues found.
- Integration of static code analyzers can be helpful
- Include unique ID's along with time stamps in SUT logs; ensure there's a way to synchronize logs on multiple servers
- Enable multiple log levels in the SUT logging, parameterized.
- It may be helpful to have a highly technical person on the QA team to coordinate closely with dev on automation (if there are no 'technical' persons on the
test team already); there were some who disagreed with the necessity of this approach.
- Use of reflection capabilities of the SUT (where available) can be helpful for automation - from outside the app being tested, it
can be queried to determine the methods, parameters, etc that the automation can interact with; this can enable more dynamic test
automation and better logging
NoVaTAIG Home Page
