December 2009 Monthly Meeting Summary

Topic:
Roundtable on Automation Tool Experience Reports
This meeting was a roundtable discussion on past or current personal experiences with particular test automation tools. We discussed:
* Specific tools and their attributes, such as kind of tool (load, web, etc), open source or COTS, cost, etc
* Experience with the tool and context (type of project, etc)
* Tool pros and cons
* Tool tips and tricks
* Etc.

Took place on: Wed. December 2 2009 6:30 PM

Attendance: 9

Meeting Notes:
Many of those present had developed custom tools/programs/utilities for test automation using Unix shell scripting, Perl, Java, Ruby, VBScript, Python, etc. However this meeting was focused on experiences with Open Source or COTS tools, as opposed to custom proprietary tools

Among the tools mentioned by those at the meeting, along with participants' comments, were:

• JTest from Parasoft - creates Java unit tests from source code, and does Java static code analysis; expensive, there were some comments re issues with customer support and tool bugs and effectiveness
• Watir - open source web testing framework in Ruby - Ruby programming knowledge needed; used to work only with Firefox but now with any browser; there were comments that documentation was so-so but there is plenty of info on the web; several attendees had used it extensively and had cgood comments about it.
• WebLoad - web load testing tool from Radview, for Windows; comments included that it was expensive, had good documentation and good support, produced PDF reports, does recording, utilized Javascript scripting.
• SilkTest previously from Segue but now MicroFocus, for UI testing including web testing; for Windows; there were some comments from someone using it several years ago that it was expensive and that it was hard to get fixes from the vendor without getting pressure to buy more licenses; other comments were that this is not the case now.
• QTPro - from Mercury now HP, for UI and web testing, utilizes VBScript for scripting, comments were that it was expensive but very popular and had many capabilities and plugins available
• Selenium - open source web testing tool, often used by agile teams and developers; comments included that documentation was not so great and thta it had a good grid control capability.
• Expect - long-existing UNIX open source tool; comments included that it was good for dealing with test setup security issues
• AutoIT - freeware tool for Windows automation, has a batch processing language
• JMeter - open source web load testing tool from Apache
• ParasoftSOATest - web services testing tool from ParaSoft; comments included that it was a very powerful tool but expensive and had a steep learning curve, and can do both functional web services testing and load testing, and that it ended up being very successful on a project to automate system testing against daily builds.
• Rational Performance Tester - load testing tool from Rational now IBM; for Win and Linux, comments included that it was expensive, scripts can be extended with Java.
• Grinder - open source load testing framework - test anything that has a Java API; comments included that scripting is done with Jython
• SQLCompare - commerecial tool for comparing metadata and data of two SQLServer databases; from Red-Gate; comments were that it was not expensive, and that it worked well for checking that a test environment DB matched up with a production environment DB.
• QAPartner (SilkTest) - Automation tool for Windows from years ago, from Segue (current incarnation is SilkTest) - comments were that it was around 10 years ago and used a proprietary C-like scripting language.
• Packet Sniffers - there are many available to help with testing/automation, including HW and software packet sniffers, and some are expensive. WireShark was mentioned as a free open source packet sniffer.
• LoadRunner - load testing tool from Mercury now HP; comments were that it was expensive aqnd utilized C++ for scripting.
• AB - Apache HTTP server benchmarking tool; *ix tool for simulating many virtual users' http requests to check on capacity of web server
• diff - a file/text comparison utility for *ix; comments were that it can be very useful in Unix-based test automation.
• cURL - command line tool for getting/sending files using URL syntax; works on many platforms; supports wide range of Internet protocols; comments were that it can be useful in test automation.

 
 
 
     NoVaTAIG Home Page

Northern Virginia Test Automation Interest Group