iTest Features

Following is a comprehensive overview of the features and functionality available in iTest.

Feature categories for rapidly developing tests:

• Defining a test case
• Executing a test case

Feature categories for automating test cases:

• Defining automation
• Executing automated tests
• Documentation
• Extensibility and integration

Feature categories for maintaining tests:

• Testbed and device abstraction

Rapidly develop tests
Defining a test case
iTest Assistant	Interactive workflow to reinforce a particular company's quality process and assist new users of iTest with step-by-step processes for key tasks.
Single-click capture	Automatically capture commands from Telnet sessions, SNMP MIB browsers, Web user interfaces, Java Swing, Serial Ports, command shells, and more to instantly generate a test case that can be easily edited, debugged, re-executed, and documented.
Integrated applications	Work in one testing solution to meet all testing requirements via an integrated Web browser, Java Swing, SNMP MIB browser, Telnet/SSH client, Tcl shell, Serial Port, and command shell.
Grid-based editing	Easily copy, paste, and move test cases and test case steps and analysis rules inside the grid without performing manual processes.
Grid-based analysis	Gain intuitive pass/fail analysis directly under each step response. iTest grid-based analysis provides sophisticated actions and/or reports for passing and failing scenarios.
Command completion	Enter test commands faster with command completion prompts and drop-down menus of commands.
Session profile	Start several sessions/connections, spanning multiple applications and devices, with a mouse click. Session profiles can be built for multiple device types and dragged and dropped into numerous tests. Easily leverage pre-defined session profiles to begin testing right away.
iTest Explorer	Use folders to organize and access all iTest files according to projects or test types, and be able to perform operations on single or multiple files simultaneously.
Favorites	Use the Favorites view for quick access to frequently used files, procedures, or session profiles. These items can then be dragged and dropped directly into test cases.
Device control	Exercise a wide variety of interfaces found in complex high-tech equipment, including, Ixia, Spirent, Agilent. Use command line interfaces (CLI), such as Telnet and SSH, embedded Web interfaces, Tcl, and more. The command structure enables device setup, configuration, and commands.
Extensible command sets	Extend iTest built-in command sets (such as CLI, HTTP, Web, etc.) by identifying Tcl packages whose functions will be exposed as commands.
Step nesting	Use nesting and indentation with iTest test cases in the same way you would write a test plan. Groups of related steps can be nested together under another step. This nesting is also used to make procedural structures (if, while, for, etc.) and looping structures (for, foreach, while) easier to write.
Rich parameterization	iTest parameters are contained in a structured data document that is available throughout the lifecycle of a test case. Rather than a flat list of named parameters, users now have a hierarchical structure supporting rich queries to access the data of interest.
Workspaces and perspectives	Work in a workspace that contains all of the related files. Perspectives provide all the tools and functions around specific tasks, such as building the test case, augmenting it with analysis or logic functions, and abstracting for regression and execution. Perspectives are customizable to match user preferences and work styles.
Procedures	Create reusable blocks for setup, configuration, or tear down, as well as create complex test cases from smaller test cases/procedures. iTest enables easy creation of procedures that can be shared among a team.
High Availability (HA)	High Availability functionality which eases the testing of carrier-class devices by managing multiple connections to the same device.
Call Procedure	The ability to use a cleanup (or other) procedure based on a certain event or analysis result. For example, a global rule might check for a failure message in any response, in which case it needs to run a procedure that will clean up the state of the DUT and then exit the test.
Executing a test case
Interactive execution	Interact with the DUTs and/or test equipment during testing or post-completion. Run test cases incrementally as new test steps are added or changed.
Automatic "expect"	Auto-populate commands, login information, and prompts (which normally require human interaction). This information is learned automatically by iTest.
True multi-threaded execution	Perform concurrent execution of steps utilizing true multi-threading to manage complex system and scenario testing. Additionally, test suites can leverage multi-threading for faster completion.
Charting and exporting extracted data	Display charts of test case data in real time.
Execution view	View in real time any issues that occur during execution.

[return to top]

Automate test cases
Defining automation
Conditional execution and iteration	Use iTest execution commands to run steps or procedures conditionally, based on data extracted from other steps.
Configurable parameters	Define any number of name/value pairs that can dynamically change the behavior of the test case, based on the testbed, the types of devices, or arguments passed when the test is executed.
Modular test case creation	Create master tests in iTest that invoke other test cases and procedures that leverage previous work and enable sophisticated analysis, concurrent execution, conditional execution, and reporting features to handle complex situations. Testing can be structured hierarchically, reporting roll-up results at each layer.
Intelligent Response Parser (IRP)	Automatically parses common responses and dynamically draws gray boxes around each field signifying that iTest understands the response. The user can create a rule on that value or string with a single mouse click. It also represents how automatic response mapping will likely be used so the analysis can be easily abstracted for lights-out regression systems.
Powerful analysis rules	Highlight desired data, responses, or fields; then use the analysis rule wizard to add a rule that will process the data. The data may be charted, stored in a variable, or compared to another value to determine pass/fail criteria for a test.
Real-time validation	Experience real-time validation as you develop a test case. iTest constantly monitors test case steps for validity and sends warning notifications, so users can take immediate action when a problem occurs within a test case step. Additionally, iTest offers a results preview, showing the result of analysis arguments constructed with data from previous executions.
Assignable variable names (queries)	Quickly associate names (queries) with fields in a recent response from a recently executed test. This allows easy transfer, manipulation, and comparison of values throughout the test case.
Auto-generated queries	Automatically generate queries based on a key assigned to a particular field.
Virtual Testbeds	Emulate responses from one or more devices in a testbed allowing testers to develop tests months before a feature or device is developed. Provides offline and remote capabilities freeing testers from noisy labs and enabling better utilization of costly lab equipment.
Executing automated tests
Test Suites	Quickly identify and group abstract tests for scheduling, execution, and reporting.
Scheduling Jobs	Schedule testing jobs to run overnight or whenever you choose directly from iTest.
Interactive and batch-mode execution	Execute any test interactively from within iTest or at the command line, or invoke it from a Tcl script. iTest is supported on Windows, Linux, and Solaris.
Load, stress, and performance testing	Automate load, stress, and performance testing for equipment ranging from data, video, and voice, to wireless, telephony, and storage.
Debugging	Pause execution and set breakpoints on any test case step to troubleshoot the test case. Users can also single-step to observe the flow of individual steps and the results of each step.
Real-time execution messages	Identify important execution issues as interactive execution is underway-rather than waiting for execution to reach completion. Execution messages provide an excellent summary of the test case, accelerating root-cause failure analysis.
Scalable iTest Execution	iTest Runtime (iTestRT) enables headless execution that is not dependent on an iTest workspace.
Documentation
Automatic test documentation	Quickly and thoroughly document each test case. Each time iTest runs a test, it automatically creates a test report that captures all setup information, every command or action, and the corresponding response in a logical, chronological progression-and entirely in the language of the device. Java and Web test reports can also capture images of screen shots taken during execution.
Test Audit	Easily browse and search every step in every session that you perform interactively or that iTest executes automatically. All of this information is kept in long-term storage, so users can go back and find something that happened days or weeks ago and look at it, replay it, or use it to build a reusable asset such as a procedure or test case. In addition, once users retrieve a test, Test Audit automatically creates a report documenting that the test was run and whether it passed or a defect was found.
Executable reports	Re-execute test reports to share as part of a bug report, so testers and developers can easily recreate defects and validate fixes.
Dependency View	View a test's dependencies for debug or in preparation for transferring test assets to other teams.
Extensibility and integration
Cross-platform support	Gain multiplatform support on Windows, Linux, and Solaris. Users can capture, document, replay, automate, and execute on any of these platforms, and seamlessly exchange files between platforms. Some limitations apply.
Extensibility via Tcl interpreter	Enable Tcl scripts and commands to be evaluated on the fly with the built-in Tcl interpreter.
Integration with existing regression systems	Execute iTest test cases inside any existing regression infrastructure via a CLI, Tcl API, or a custom integration.
Integration with existing script libraries	Incorporate existing script libraries into iTest test cases and present them via drop-down menus. This can encourage QA engineers with limited programming expertise to adopt script libraries.
SNMP	Use iTest to test (capture, document, and replay) SNMP v1, v2c, or V3 interfaces. Supported actions include Get, Get Next, Set, Get Table, Walk, List Traps, Wait for Trap. iTest has a built-in MIB browser.
Database	Integrate databases into tests for data-driven testing (e.g. NMS testing) and aggregating performance data. Pull records from a database and use the data to drive a test case. Harvest data from a test while it is executing and push it into a database for later post-processing.
Web	Use iTest to test (capture, document, and replay) Web-based interfaces of networking devices, allowing users to compare CLI and Web information in the same test case.
Java Swing	Use the Java Swing session type to capture steps directly from user actions such as mouse clicks and keyboard entries. The built-in Swing control window can also be used for more fine-grained actions (e.g. X-Y information).
Screen-based CLI	Use iTest to test (capture, document, and replay) screen-based CLI applications, such as virtual terminal applications that provide terminal-based menus and commands, or use a Linux application such as "top."
SSH capture	Capture, document, and replay everything going on in CLI sessions that are based on SSH. This feature supports key-based authentication including KBI, public key, host-based, host RSA, and challenge-based approaches.
Telnet localhost capture	Capture and test loopback traffic to "localhost" over tunnels and VPNs.
Windows CMD	Use the CMD application in iTest to test (capture, document, and replay) a command line prompt if you are working on a Windows machine. This makes it very easy to perform most file-based actions.
Tcl shell	Use the integrated Tcl shell application in iTest to easily access and test (capture, document, and replay) these Tcl-based tools.
Test equipment/Traffic generation	Easily control traffic generators (such as Ixia, Spirent, and Agilent) interactively from a simple user interface integrated into iTest. All of these actions and the information extracted are documented and can be replayed and integrated into test cases.
Ixia traffic	Integrated support for Ixia traffic generators.
Agilent traffic	Integrated support for Agilent N2X traffic generators.
Spirent SmartBits	Integrated support for Spirent SmartBits traffic generators.
Spirent TestCenter	Integrated support for Spirent TestCenter.
Spirent Avalanche	Integrated support for Spirent Avalanche.
Ixia IxLoad	Integrated support for Ixia IxLoad.
Ixia IxNetwork	Integrated support for Ixia IxNetwork.
Wireshark	Integrated support for working with pcap files.
Syslog	Integrated support for working with syslog messages.
JSON	iTest includes JavaScript Object Notation (JSON) Mappers that auto-generate queries for JSON format responses.
HP Quality Center Integration	iTest is an officially certified HPQC application. The integration creates the bidirectional flow of information between iTest and HPQC. Publish single or batch results directly into HPQC as well as place copies of the entire test report and test case inside of HPQC. Suites of tests can be targeted for execution within HPQC and then have iTest execute the tests and return the results, reports, and test cases back into HPQC.
Eclipse Plug-in	Build on Eclipse by installing iTest as a plug-in.
Convert to Tcl	Render test cases into Tcl with iTest. Users can control the format of the Tcl file that is rendered by iTest.

[return to top]

Maintain tests
Testbed and device abstraction
Testbed definitions	Define testbeds as a set of named devices in separate testbed documents, so that test cases can port easily between different testbeds-even when the testbeds contain different versions or variants of the devices.
Session profiles	Allow sessions to be started instantly and dragged and dropped into any test case. When devices or their properties change, the powerful abstraction techniques provided by iTest allow users to modify only the one session profile, rather than every test case
Scriptless parsing	Automatically convert raw text, unstructured data, and structured data (IP addresses, MAC addresses, timestamps, integers, real numbers, and so on) into easily manageable variables (tokens).
Scriptless analysis definitions	Declare expected behavior easily by selecting among a set of rules and arguments that are customized to the specific context, and then editing and/or combining the rules to achieve precisely the desired analysis.
Web page mapping	Convert a Web page into a set of easily recognized targets (links, buttons, text fields, tables) for easy test case maintenance and abstraction. When targets change on the page, users need to modify only the one map, rather than every test case.
Named variables, queries, and parameters	Specify and name critical variables/tokens (words, whitespace, numbers, IP addresses, timestamps) to enable test case portability across product variants.
Automatic Response Mapping (ARM)	ARM makes the generation of response maps easier by leveraging the gray boxes created with IRP to quickly create response maps. Query-based analysis rules constructed with IRP can be abstracted with the creation of a corresponding response map.
Response Map Wizard	Be guided through the steps for creating a response map, accelerating response map creation for a broad class of responses. When responses change, users need to modify only the one map, rather than every test case. This allows tests to be easily maintained for fully automated regression testing across releases.

[return to top]