What to Look for in a Test Automation Tool

Given the recent emergence of utilizing automation tools for equipment QA, we thought it would be helpful to provide a framework for evaluating testing and verification automation tools.

Specifically, a test case automation tool needs to:

Separate device-specific information from the test case in order to be maintainable

An ongoing challenge for QA professionals is maintenance of automated test cases in regression. Each time a new software version or product variant is created, existing test scripts may no longer work with that version and need to be debugged or even recreated. A test automation tool needs to preserve device-specific and release-specific behavior separately from the test cases. When something changes in a new product release or version of the software, the tester can then simply update the device information in a single location rather than updating all of the relevant test cases.

For example, a device definition file can be created that contains specific properties such as CLI prompts. These files are defined and maintained for a particular device, with each property assigned a specific name. Test cases should refer to these properties by the given name so even if the value of the field changes, the test case does not need to be updated.

Provide documentation that can be easily shared with developers responsible for fixing the problem

In addition to automating test development and execution, an automation tool should also automatically generate test reports, in the form of easily accessed XML documents, every time a test is executed. These documents will help pinpoint a failure’s root cause and can be easily shared with the developers responsible for fixing the problems. In addition, if developers use the same tool for unit testing as the QA group uses, they will be able to quickly re-run tests to validate fixes, reducing testing cycles even further.

Rapidly kick off test creation by capturing commands as the Device Under Test (DUT) is executed

To accelerate the development of new test cases, the automation tool should allow the tester to capture a test case as it is performed manually, recording all commands and responses in all What are the benefits of an IDE for QA Professionals? multiple sessions. These test steps should then be placed into an interactive environment where the test can be further developed and enhanced. Capturing commands is the absolutely fastest way to start a new test case.

Leverage existing testing infrastructure

Very simply, existing test cases should be leveraged, not replaced.

As the automation builds, the user must retain the ability to insert previously captured steps into the current test case or into new procedures. Users should be able to edit captured steps to remove extraneous commands, correct mistakes, and insert additional commands. This process should always show the actual commands involved. In addition, any existing scripts should be integrated into the new test cases, so work that is already done doesn’t have to be redone.

Summary

With the advent of new automation tool technology for QA, the tester can now perform more robust testing, more frequently without losing valuable time. Automation tools allow users to do the testing they do best by enabling all members of the test team to automate their tedious and mundane tasks. The result? The tester can now play a more significant internal role and provide a greater contribution to over-all product quality and customer satisfaction.