August 2009 Monthly Meeting Summary
Topic:
Choosing a Test Automation Tool
This meeting was primarily a roundtable discussion covering issues that are
typically encountered in selecting a test automation tool; Bill Mosteller of Eloqua Inc.
provided some initial focus to the discussion with a short presentation on his experience
with the Selenium vs Watir selection process. Discussion included such topics as:
* Open Source or COTS?
* Roll your own?
* Programming/scripting language considerations
* Support considerations
* Differing perspectives – mgmt, engineering, etc.
* Other topics
Took place on: Wed. August 5 2009 6:30 PM
Attendance: 11
Meeting Notes:
Bill Mosteller provided a 20 minute presentation on his experience with evaluating
Selenium vs. Watir for use in web testing projects. Some of their initial considerations
included Multi Language Support (non-ascii characters), Docs , Maturity, Frames,
Learning Curve, Existing team knowledge, Price/License, Community. They spent some time
trying out both tools on early versions of their app or other web apps. They found that
research skills were important for using open source tools since info/documentation can
be dispersed.
Bill's powerpoint file re evaluating Selenium vs Watir
There was much discussion during and following the presentation, and a list of things
to consider in choosing a test automation tool was developed during the meeting, which is as
follows:
- Is it OK for your choice to be open source? COTS? either?
- Is it OK - or preferable - to develope your own custom tool?
- Considerations re which programming/scripting language
- Support issues
- What are the concerns/interest of different automation stakeholders (management, dev, others?)
- Results reporting capabilities
- AUT constraints and OS constraints
- How much time/effort is needed to try out tool candidates.
- Available documentation/info, and how easy is ti to find needed info
- Programming skills of the test automation personnel; training needs
- What is of interest to the test automation personnel? What's cool/new? (Motivation/interest)
- The team's/organization's previous investments in automation tools/approaches
- Learning curve for the tool/language/approach
- The available 'ecosystem' that has built up around the tool
- Available budget/time
- Management prejudices
- Internal organizational politics
- Maintenance issues
- Expected ROI - monetary
- Expected ROI - non-monetary
- Can you 'start small' with the tool/approach?
- Demo availability
- Trial version
- Trial period
NoVaTAIG Home Page
