Increasing Performance with Parallel Test

As time to market continues to be a priority and devices under test increase in complexity, today’s engineers face the challenges of increasing throughput and reducing test time. Also, test departments must find ways of growing test budgets at a much lower rate than production volumes. Parallel test helps engineers meet these challenges by increasing the use of existing instrumentation and running tests on multiple units or subsystems concurrently to increase throughput.

Engineers can implement parallel test in a number of different ways, depending on the application. Developers can use a pipelining architecture to easily create a parallel test system based on a previous sequential test system or use autoscheduling to further decrease test times. In addition, by taking advantage of multicore processors, developers can use test systems to optimize analysis tasks that rely heavily on the processor to execute. As engineers set out their strategies for next-generation test systems, parallel test provides a way of meeting test time challenges with existing instrumentation resources.

What Is Parallel Test?

Developers can increase test throughput by investing in more hardware resources to test more than one unit concurrently. But relying on more hardware as a way of increasing throughput is usually not an option because of the cost. A new set of instruments would be necessary to increase throughput by one device per test cycle.

On the other hand, using parallel test, engineers can optimize instrument idle time and use a single set of instruments to test multiple devices in parallel. In addition, with the increasing popularity of multicore processors, engineers can use parallel test systems to execute analysis tasks on different cores, achieving truly parallel execution.

 

Choosing a Parallel Test Software Architecture

When selecting a software architecture for parallel test, two of the most important elements engineers should consider are flexibility and support for parallel test technologies. Flexible software architectures meet the changing software development requirements of today’s devices under test. The software architecture must work with both legacy and next-generation code written in different programming languages. Modular software architectures, such as the one shown in Figure 1, provide flexibility by relying on test management software to control and execute tests written in multiple languages. With a wide range of programming languages to choose from, engineers can communicate with existing and future instrumentation.

[+] Enlarge Image

Figure 1. Modular software architecture helps engineers increase the flexibility of their test systems.

The software architecture must also support parallel test technologies, such as multithreading and parallel execution of test sequences. For engineers to take advantage of new multicore processors, the whole software architecture, from the drivers to the test management software, must be multithreaded. With a multithreaded application, the operating system can send threads to different processor cores for execution, capitalizing on the whole processor. If the application runs on a single thread, it forfeits the processing power of the whole processor by executing on a single core.

In addition to being flexible, the test software architecture should include test management software, such as NI TestStand, that facilitates the development of parallel test sequences. Features such as a parallel process model that can execute multiple instances of the same test sequence in parallel can reduce the amount of time necessary to develop a test system. High-level tools to share instrumentation across devices reduce the learning curve of developers who are new to parallel test. Finally, execution profiling tools can help optimize test systems by highlighting bottlenecks and instruments with high rates of use.

 

Implementing Parallel Test

Sharing instruments across multiple units is at the core of parallel test implementation. Depending on the application, the test system can rely on a pipelining architecture, which is simple to implement, or autoscheduling, which can further increase test throughput. In addition, processor-intensive tasks that don’t rely on instrumentation can use multicore processors to increase performance.

Pipelining Execution

Parallel test systems usually test multiple units of the same type of device concurrently. Each device requires the same sequence of tests. One of the simplest methods of implementing a parallel test system is to use a pipelining architecture, such as the one shown in Figure 2. Pipelining shares instruments across multiple units by staggering the execution of each unit. In this architecture, as soon as one device finishes using the instrument necessary to run the first test, the following device starts using the instrument. Each subsequent test begins executing once the previous device is done with the necessary instrumentation. Test systems that use pipelining can reduce test time by up to 50 percent.

[+] Enlarge Image

Figure 2. Pipelining helps optimize test systems by running test sequences in parallel and sharing resources.

Implementing a pipelining architecture can be very simple by using test management software that supports parallel process models and locks to guarantee that no two threads use the same instrument concurrently. Test management software can generate four instances of the same sequence and run them in parallel. Each test step in a sequence should acquire a lock for the instrumentation it needs to execute. If the step can’t acquire the lock, it waits until another test sequence releases the lock. Once the lock is available, the step starts executing the test. When this behavior is repeated for multiple devices, the test system automatically implements a pipelining architecture as seen in Figure 2.

Autoschedule Execution

Pipelining helps increase the throughput of test systems, but there is still an opportunity to further optimize instrument use. Instrumentation that could be used to run other tests is still idle during the test system’s execution. Autoscheduling execution further increases throughput by reordering tests to take advantage of the time when instruments are idle. Test systems that implement autoscheduling can increase throughput by up 65 percent without the need for additional instrumentation.

For example, during the first test in Figure 2, two of the three instruments are idle, and during the second test, one instrument is not performing a test. If engineers could change the execution order of the tests, they could run tests on these idle instruments. Figure 3 shows how reordering tests and running them on idle instrumentation can increase the same test system’s throughput.

 
[+] Enlarge Image

Figure 3. Autoscheduling helps increase the throughput of test systems by reordering tests to take advantage of idle instrumentation.

Optimizing Processor-Intensive Tasks

Up to now, it has been assumed that the types of tasks that need optimizing rely on instrumentation to execute. Automated test systems don’t depend solely on instrumentation as a resource. In some cases, the system must perform analysis tasks on the acquired data. The main resource necessary to run an analysis step is processor time. With the increasing popularity of multicore processors, optimizing processor-intensive tasks must focus on taking advantage of idle cores, much like taking advantage of idle instrumentation in autoscheduling and pipelining.

In most off-the-shelf operating systems, scheduling threads on different cores is handled by the operating system. Therefore, the most important consideration for optimizing processor-intensive tasks is to guarantee that the whole software architecture is multithreaded. If any of the pieces of the architecture do not support multithreading, all processing tasks run on a single core and do not take advantage of idle cores. Once the analysis tasks are generated as different threads, the operating system can schedule them on different cores.

 

More Resources on Developing Parallel Test Systems

Implementing Parallel Test in NI TestStand: View a webcast on how to implement parallel test systems using NI test management software.

Benefits of Parallel Testing: Learn why you should implement parallel test systems.

 

The mark LabWindows is used under a license from Microsoft Corporation. Windows is a registered trademark of Microsoft Corporation in the United States and other countries.

Reader Comments | Submit a comment »