NI TestStand 3.5 – Designing Custom Automated Test Solutions with Industry-Standard Tools and Protocols

The devices being tested today include a wider array of components than ever before. As an example of this convergence of technologies, telephones have been transformed from hardwired audio devices the size of a toaster into less than pocket-sized wireless audio, video, and network communication devices. The rapid growth in technology adoption among consumer and commercial products is resulting in the growing complexity of devices. Increasingly tight business operations and budgets focused on the bottom line are also creating new challenges for today's design and test engineer. Dealing with these challenges is daunting enough. Constantly upgrading and maintaining proprietary and in many cases legacy, home-grown test systems, is an unnecessary burden on both the engineers and the organization trying to maintain their competitive edge. Fortunately, there is a way to transform your test strategy from an unnecessary burden into a competitive advantage by designing your next customized automated test system with industry-standard, commercial-off-the-shelf (COTS) tools and protocols. Building your next-generation test system strategy on this core principal, you can deliver the following benefits to your team and organization:

1. Maximum system flexibility for easy reuse of system software and hardware components;
2. Rapid scalability to quickly turn a single-station, single-unit testing environment into an environment with multiple stations capable of multiproduct, parallel testing; and
3. Minimized total cost of ownership including software, hardware, development, deployment, training, and maintenance costs.

Industry-Standard Tools and Protocols

The desire to design everything from scratch is one of the greatest challenges that we, as engineers, face in developing streamlined, cost-effective solutions. Yielding to this desire is often the source of the largest challenges in today's test environment, including increased time and money spent enhancing, maintaining, and fixing current test systems. Taking advantage of widely available, industry-standard tools and protocols designed specifically for test can save you a lot of time and frustration while still requiring your expertise to ensure the optimum level of integration and modularity in the overall test system architecture.

While it is nearly impossible to design a 100 percent COTS-based test solution due to the complexity of today's units under test (UUTs), there are many common elements in every test system that can benefit from using the following COTS components:

· Mass interconnects
· Signal conditioning
· Instrument buses
· Instrument control protocols
· Instrument drivers
· Application development environments
· Enterprise connectivity protocols
· *NEW* test management software

Mass Interconnects
COTS terminal blocks, industry-standard wiring connectors, and mass interconnect panels provide the lowest-level connectivity to your UUT. These connectors ensure that signals to and from the UUT are transmitted reliably. Often, the UUT is connected to your test instruments via switch hardware, increasing throughput for your automated tests. National Instruments provides several off-the-shelf terminal blocks for connecting your UUT to your instruments and switches. Additionally, companies like Virginia Panel and MAC Panel provide mass interconnect panels for connecting all of your signals to your UUT through one hardware interface.

Signal Conditioning
Sensors and transducers require signal conditioning before a computer-based measurement system can effectively and accurately acquire the signal. The front-end signal conditioning system can include functions such as signal amplification, attenuation, filtering, electrical isolation, simultaneous sampling, and multiplexing. In addition, many transducers require excitation currents or voltages, bridge completion, linearization, or high amplification for proper and accurate operation. Industry-standard COTS technologies and solutions, including USB connectivity and 1451.4 TEDS smart sensor plug-n-play, are available from National Instruments and other vendors to assist with your signal conditioning needs.

Instrument Buses
IEEE 488.2 general-purpose interface bus (GPIB) is the most widely used standard in automated test. There are thousands of IEEE 488.2 compliant instruments for use in automated test. PCI Extensions for Instrumentation (PXI) is the fastest growing new instrument bus in the history of test and measurement with more than 60 PXI Systems Alliance vendors and more than 1,000 hardware devices available in PXI modules. PXI is an industry standard designed to interoperate with GPIB and other existing instrumentation platforms. The PXI standard delivers industrial strength and durability with the speed of a PCI bus as well as built in timing and triggering lines.

Instrument Control Protocols
Communicating with and controlling instruments is accomplished with an instrument programming interface or protocol. Industry-standard instrument control protocols include SCPI, VISA (Virtual Instrumentation Software Architecture), and
VXIplug&play. Additionally, VISA provides a bus independent protocol by providing one set of functions for communicating over Ethernet, serial, GPIB, or USB. Using a common, tried-and-true instrument control protocol establishes a trustworthy platform and removes the guesswork of automating your instruments.

Instrument Drivers
Instrument drivers are highly recommended to maximize your code reusability and instrument bus interchangeability. National Instruments develops and hosts instrument drivers for more than 4,000 instruments from more than 300 instrument vendors. LabVIEW and LabWindows/CVI plug-and-play source code drivers are recommended for the majority of automated measurement applications. Applications requiring maximum flexibility, throughput, and reuse should use IVI (Interchangeable Virtual Instrument) -compliant drivers.

Application Development Environments (ADEs)
It is important to use an ADE that is widely used and designed for developing test and measurement applications. NI LabVIEW and LabWindows/CVI are de facto standard test development environments, with graphical or dialog based programming, and designed for use by engineers. These NI ADEs also remove the guesswork of language syntax details and provide analysis function libraries for common data and signal analysis operations. Additionally, thousands of instrument drivers have been developed in and for these environments.

Enterprise Connectivity Protocols
Connectivity to enterprise software gives your test systems capabilities like result reporting, database logging, version control, and requirements management. Protocols such as XML, COM, and SCC provide standard means for integrating enterprise solutions and functionality into your test systems.

Extensible Markup Language (XML) provides a standard for sharing a variety of data in a text-based, human and machine readable form. With XML you can also customize the language through schemas and the implementation of style sheets for improved appearance of documents.

ActiveX Data Objects (ADO) is a COM based protocol for communicating with databases. Using ADO, yoiu can connect to databases without knowledge of SQL and independent of the particular implementation of the database.

Microsoft SCC is an interface for applications to communicate with source code control providers. Implementing source code control, several engineers can collaborate on test module development by tracking the changes made to the modules.

Requirements management software is used to document system and test requirements. Additionally, requirements management software can be used to guide software development by providing requirements validation, verification, and traceability features. Ideally, your test system software can be linked to your requirements management software to track and trace requirements from documentation to implementation.

*NEW* Test Management Software
Test management software, such as NI TestStand, provides a ready-to-run test environment for automating your test and validation systems. Test management software is used to develop, manage, and execute sequences, which integrate test modules written in any programming language, and specify sequence flow, reporting, database logging and connectivity to other enterprise systems. Test management software is designed to maximize test-code reuse through built-in language adapters and simplify maintenance by using a modular fully customizable test-system framework. NI TestStand is the most commonly used test management software in the industry. It is used by 14 of the top 15 electronics manufacturers and the top 10 electronics manufacturing services companies.

Implementing a customized test system using COTS industry standards and protocols will significantly decrease the bulk of development and software integration efforts required for most applications. In order to fully maximize the total flexibility, scalability, and decreased total cost of ownership, you should apply the aforementioned industry standards and protocols, within a modular test system framework.

NI TestStand

NI TestStand is test management software at its best. Automating thousands of test systems worldwide, NI TestStand is a tried-and-true COTS solution for test. The core of NI TestStand is a robust test sequence creation and execution environment. From this robust core, NI TestStand provides connectivity and integration of enterprise software and industry-standard tools and protocols. NI TestStand provides a reliable framework for your open, modular test system architecture.


[+] Enlarge Image
Figure 1. Modular Test System Architecture

Robust Test Management

NI TestStand provides a test sequence editing environment for the creation and configuration of your test system. The operator interfaces provided for sequence execution are fully customizable. The sequence editor and operator interfaces are user interfaces to the robust test management software that is NI TestStand. NI TestStand provides standard (and in many cases customizable) features for the creation, management, and execution of tests.

TestStand Engine

At the heart of NI TestStand is the TestStand Engine. The TestStand Engine is a set of 32-bit DLLs, optimized for speed, flexibility and portability. This engine is the brain serving test system management and execution. Having this well-designed engine running "under the hood" of your test system removes the guess work of how and when tasks are executed. The TestStand Engine provides an open application programming interface (API) for programmatic configuration and control of NI TestStand. The TestStand API gives engineers all of the power of NI TestStand, with which they can achieve complete automation of test creation, customization, and execution.

Process Models

Tasks that are common to all test sequences are handled by a process model, which is used in combination with many test sequences. Using a process model is not merely a case of code reuse. Using a process model, test developers can focus on developing tests by handling tasks such as serial number entry, report generation, and database logging.

For database logging, NI TestStand provides ADO connectivity to industry-standard databases, like Access, SQL, and Oracle, through OLE DB and ODBC providers. NI TestStand provides default mappings (schemas) that can be customized to fit particular needs. The customization of database logging can be accomplished through dialog-based configuration so that there is no need to be an SQL expert.

In addition to managing common tasks, two process models in NI TestStand are provided for parallel testing. Test throughput can be increased many times over by implementing parallel testing. In the parallel testing scenarios, units can be tested in independent sockets or in batches. Parallel testing takes advantage of the multithreading capabilities of standard processors.

By implementing these tasks in the process models, NI TestStand handles common and potentially difficult programming tasks such as multithreading and database connectivity. Thus, the implementation of a process model reduces development times and costs, relieves test engineers and managers from the task of developing and maintaining common tasks, and frees test engineers and managers to focus on developing sound and reliable test modules.

ADE Integration

ADE integration in NI TestStand is accomplished through an open language interface for test module development and execution. Test sequences in NI TestStand are composed of individual steps. Each step in a sequence can be configured to call LabVIEW, LabWindows/CVI, C++, ActiveX/COM and .NET test modules. Using C++ or ActiveX/COM as an intermediary, it is also possible to use custom step types to call code written in many other languages, such as ATLAS, Java, and VEE. Step configuration can also include rules for branching, remote execution, and flow control.

The native integration with LabVIEW, LabWindows/CVI, and Visual Studio with Measurement Studio can be used to create, debug, and execute test modules. Quick-start code, or templates, can be specified to reduce programming common test code. The quick-start code can include as little as a common set of parameters in a function prototype to as much as is appropriate to maximize reuse.

Standard debugging features for test sequences in NI TestStand include breakpoints, watch expressions, and execution tracing. With NI TestStand you can also move from a step in your test sequence into the associated test module in LabVIEW or LabWindows/CVI for debugging your test modules. Memory corruption detection features for debugging test modules include stack checking, buffer checking, and reporting NI TestStand property object leaks.

You can achieve intermodule communication using the open language interface. Parameters can be passed from NI TestStand to your test modules. Test results can be stored in NI TestStand variables that can be shared between sequences, steps, and test modules.

Using the open language interface, engineers can dramatically reduce development and validation time. Test modules can be easily reused and replaced without corrupting test sequences, the test system, or the test environment. Standard debugging tools provide common and intuitive means for testing and validating tests.

System Management and Deployment

Test system deployment and management tools and features are standard in NI TestStand. These tools and features include a deployment utility, user manager, SCC integration, and password protection for test sequence files.

With the deployment utility in NI TestStand, you can deploy all test system software, including driver software, in one Windows installer-based distribution package. This helps ensure that all of the necessary software is installed on a deployment station to deliver a shorter time to first test and reduced downtime.


[+] Enlarge Image
Figure 2. Streamlined Test System Deployment

In NI TestStand, users are organized into groups according to a set of privileges. Four default groups are provided to handle the most common sets of privileges. Just as new users can be created, new groups can be created. Members of the Administrator group have full access to NI TestStand features while Operators have minimal privileges. When new users are created, they can be included in a group and have their privileges tied to the privileges of the group. Then, when updates are made to the privileges of a group, any users who are members of that group also have their privileges updated.

Integration with SCC providers is accomplished with workspace files. Similar to workspaces in common ADEs, NI TestStand workspaces can contain projects and these projects can contain test sequence files, test modules, and other test related files. From NI TestStand workspace files, test related files can be checked in and out of an SCC provider.

With NI TestStand, developers can limit user ability to edit sequence files; so users cannot edit file contents on disk or in the sequence editor unless they have a password. Additionally, developers can prevent users from seeing internal sequence file and step type information to prevent general viewing.

Enterprise Software and Industry-Standard Tools and Protocols

From this robust test sequence creation and execution environment, NI TestStand provides connectivity and integration of enterprise solutions with industry-standard tools and protocols. In addition to integrating test modules, a complete test system includes functionality provided by enterprise software as well as other industry-standard tools and protocols. In NI TestStand, this added functionality includes switching, requirements management, and implementation of ATML, and database integration beyond test result logging.

Database Connectivity

Beyond logging test results to a database, database connectivity in NI TestStand can include dynamically reading and writing values to and from a database. Test parameters, like limits, can easily be loaded from a database so that test sequences can be reused for multiple product models when the testing scenario differs only by parameter values. Database connectivity is ADO based just as database logging.

NI TestStand provides two tools for importing property values into a test sequence. The Import/Export Property Tool is a dialog-based utility for getting parameter values to and from a database. The Property Loader is a type of step with which values can be imported dynamically from a database. Placed at the beginning of a test sequence, this step creates another layer of reusability – one test sequence for multiple product models whose testing differs only by parameter values. Additionally, the Import/Export Property Tool and Property Loader step type can be used in conjunction with spreadsheet files and tab-delimited text files.

There are database step types available for complete free-form database connectivity. SQL-based statements can be executed as steps in a test sequence for importing and exporting data, and dialog-based SQL statement creation is available to remove the burden of having SQL expertise.

Switch Integration

Automated parallel testing and instrument sharing is made possible with switching. Rather than having switching handled in a test module, switching can be managed at the same level as the test management software, which keeps the test modules independent of a particular hardware setup and thus ensures their reusability.

National Instruments offers switches in both SCXI and PXI modules. As far as architectures, NI switches include basic relays to matrices. Controlling switches could be integrated into test modules, but this would damage the reusability and flexibility of the modules.

The seamless integration of NI Switch Executive into step configuration in NI TestStand maintains reusability and flexibility. NI Switch Executive abstracts low-level switch programming to provide painless programmatic (and static) selection of switch routes. The same step with the same test module can be used on distinct devices/instruments/UUTs by implementing and configuring switching at the level of a test sequence step. In a parallel test with multiple UUTs, each thread can be using a different switch route, by configuring the step to programmatically select the route.

Requirements Management

There is a growing concern in aerospace, automotive, and medical organizations to ensure that test coverage satisfies specific requirements. Requirements management tools, such as Telelogic DOORS and Rational RequisitePro, are used in these industries to create requirements documents. Then, test sequences and test modules are developed to provide coverage for these requirements.

Traceability is a key function of requirements management. Traceability of requirements provides data for impact analysis - the impact of changing a requirements and the impact of test failures. These analyses can be used for cost/benefit analysis for reducing costs associated with testing.

NI TestStand includes built-in fields for notating product and unit requirements at the workspace, project, sequence file, sequence, and step level. Third-party requirements management packages can use the NI TestStand API to retrieve or specify requirement values directly. With additional application software from NI, requirements management software tools can be linked with NI TestStand to track and trace requirements from documentation to implementation.


[+] Enlarge Image
Figure 3. Requirements Management Integration

Automatic Test Markup Language

The Automatic Test Markup Language (ATML) is the result of a cooperative industry effort to define a collection of XML schemas to represent test information. National Instruments actively contributes to the definition of this standard in partnership with industry leaders in automated test and the IEEE. ATML is intended to ensure standard data interfacing between test systems.


[+] Enlarge Image
Figure 4. ATML Test Results Schema Implemented in NI TestStand Reporting

The first schema defined is the Test Results schema. According to the ATML definition, a test is any procedure for evaluating or quantifying the operation of a device or system. The Test Results schema provides an industry-standard format for exchanging and storing measured values, pass/fail results, and accompanying data (including test operator, station information, and environmental conditions). The report options in NI TestStand include an implementation of the Test Results schema for automatic generation of ATML-compliant test reports.

See Also:
About the TestStand Process Models
Custom Step Types
Using the Import/Export Properties Tool to Export Limits to a Database
ATML Home

Benefits of Integrating Standard Tools and Protocols

The connectivity and integration of enterprise software and standard tools benefits test engineers and managers by ensuring standard interfacing between test systems as well as maximum system and test module reusability and flexibility. This connectivity and integration would be meaningless without a reliable core for test creation, management, and execution. Having an optimized execution engine, multithreading capabilities, configuration management, and deployment features gives test systems the robust foundation needed in automated test systems. Using test management software, such as NI TestStand, gives your COTS-based test system a framework for an open, modular architecture.

By building your next generation test systems on industry-standard COTS tools and protocols, your test strategy gains a competitive edge. You get the benefit of having maximum system flexibility for easy reuse of system components, rapid scalability, and minimized total cost of ownership including software, hardware, development, deployment, training, and maintenance costs.
Related Links:
NI TestStand

Reader Comments | Submit a comment »