Comparing Resource Usage Strategies

TestStand 2019 Help

Edition Date: May 2019

Part Number: 370052AA-01

»View Product Info
Download Help (Windows Only)

Description

This example demonstrates how to use the Execution Profiler to measure and understand how different multithreaded resource usage strategies affect the performance of a simultaneous test of four UUTs when each UUT requires a DMM measurement and a scope measurement.

Example File Location


Open Example

<TestStand Public>\Examples\Parallel Testing\Comparing Resource Usage Strategies\Comparing Resource Usage Strategies.seq

Highlighted Features

Major API

N/A

Prerequisites

None

How to Use This Example

When you open the Comparing Resource Usage Strategies.seq example sequence file, TestStand also launches the Execution Profiler window.

Complete the following steps to walk through the example:

  1. In the TestStand Sequence Editor, select the MainSequence and select Run MainSequence from the Execute menu. In the Execution Profiler window, view the instrument usage in the Item Usage per Thread graph as the example executes five sequences with five different strategies for using the DMM and scope resources. Make the profiler window wider if your screen has room.

    The first row of the graph shows per step locks on the top level sequence call steps. The example has these locks to better highlight the duration of each strategy in the profiler. Below the square for each lock, you can see the usage of the DMM and scope resources for that strategy.
  2. For Sequential, note that each UUT uses the DMM in turn, and then each UUT uses the scope in turn.
  3. For Parallel, note that each UUT uses the DMM as soon as it is available and then uses the scope as soon as it is available.
  4. For Auto-scheduled, note that each UUT first uses an instrument of either kind, as soon as one is available, and then uses the other kind of instrument as soon as it is available.
  5. For Parallel With Additional DMM, note that each UUT first uses either one of two DMMs as soon as one is available and then uses the scope as soon as it is available.
  6. For Auto Scheduled With Additional DMM, note that each UUT first uses any instrument, as soon as one is available. It then uses the remaining type of instrument as soon as one is available.
  7. Note that each strategy completes in a shorter time than the previous strategy.
  8. Open the View menu in the profiler. To focus on the instrument usage only, the example unchecked the following menu items when it launched the profiler in its SequenceFileLoad sequence:
    • Show Blocked Threads
    • Show Code Modules»Step
    • Show Code Modules»Step Type
    • Show Code Modules»Load
    • Show Code Modules»Unload
    • Show Synchronization»Show Auto Scheduling
    • Show Synchronization»Show Batch Synchronization
    • Show Synchronization»Show Wait Operations
    • Show Steps
    • Show UUTs, Batches, and Lots
    Toggle each menu item on and off to observe the additional data the profiler displays when each item is on.

Complete the following steps to walk through the example:

  1. Checkmark the following items in the View menu:
    • Show Blocked Threads
    • Show Synchronization»Show Auto Scheduling
    • Show Synchronization»Show Batch Synchronization
    • Show Synchronization»Show Wait Operations
    • Show UUTs, Batches, and Lots
  2. Click Clear and complete the following steps:

    1. In the TestStand Sequence Editor, right-click the Sequential step in the Main step group of the MainSequence and select Run Selected Steps from the context menu.
    2. In the Execution Profiler window, checkmark Auto Fit below the Item Usage Per Thread graph.

      The Sequential strategy example sequentially runs the DMM test and then the scope test on each UUT. For each test socket thread, the Item Usage per Thread graph shows the use of the DMM resource and the scope resource. The red rectangles in the graph whose tool tips contain Batch Synchronization indicate threads waiting to enter and exit Batch Serial Sections, which is the mechanism this example uses to enforce sequential use of the DMM and scope resources. You can hide the Batch Synchronization operations by unchecking View»Show Batch Synchronized Sections.

      The Sequential strategy example uses Wait steps to simulate instrument usage. In addition, the Sequential step in the MainSequence is configured to wait for the execution it launches to complete. You can hide the Wait operations by unchecking View»Show Wait Operations.

      The graph also shows that none of the resources are used in parallel. The time displayed when you hover over the square labeled <No Serial Number> (Batch) is 30 seconds, which is the time it took to test the batch of UUTs. The proceeding steps attempt to lower the total time span value.
    3. Click the Clear button to remove the graph information.
    4. In the sequence editor, right-click the Parallel step in the Main step group of the MainSequence and select Run Selected Steps from the context menu.

      In the Execution Profiler window, notice that all threads attempt to use the DMM, but only one thread acquires the DMM while the other threads block. As each thread completes the DMM test, another thread can acquire the DMM. You can hide the time that threads block while waiting to acquire resources from the graph in the profiler window display by unchecking View»Show Blocked Threads.

      Because the DMM and scope resources are used in parallel much of the time, the length of the Batch reduces to 21 seconds. However, because each thread first uses the DMM and then uses the scope, no parallelism exists until the first thread releases the DMM.
    5. Click the Clear button to remove the graph information and select View»Show Blocked Threads to show the time that threads block while waiting to acquire resources.
    6. In the sequence editor, right-click the Auto Schedule step in the Main step group of the MainSequence and select Run Selected Steps from the context menu.

      In the profiler, notice that the graph now includes rectangles labeled Auto Schedule ("Resource name"), which correspond to Use Auto Scheduled Resource steps that define resource usage sections in the sequence. The graph shows that the Use Auto Scheduled Resource steps block the execution of their threads until the steps can lock the resources that their section of the sequence requires. You can hide the operations of the Auto Schedule and Use Auto Scheduled Resource stepsby unchecking View»Show Auto Scheduling.

      Notice that for some threads, the Auto Schedule example reorders the use of the DMM and scope resources to reduce the batch time to approximately 18 seconds, because each DMM test takes a minimum of 4.5 seconds. In this example, the test time is now resource limited, and any further improvements will require additional resources.
    7. Click the Items tab to display the Items table.
    8. Uncheck View»Show UUTs, Batches, and Lots.
    9. Click the % Actual Time in Use column heading twice to sort the percent values in descending order, as indicated by an inverted triangle icon.

      The Auto Scheduled item has the highest % Actual Time in Use value. However, this is only the name of the lock the example holds to force the profiler to display the name of the strategy being used. Ignoring that item, the DMM item now has the highest Percent Time in Use value, which indicates that system performance might improve if you add additional DMM resources.
    10. Click the Clear button to remove the graph information and uncheck View»Display Blocked Threads. Checkmark View»Show UUTs, Batches, and Lots
    11. .
    12. In the sequence editor, right-click the Parallel with Additional DMM step in the Main step group of the MainSequence and select Run Selected Steps from the context menu.

      In the profiler, notice that the additional DMM reduces the value of the batch time to approximately 17 seconds. Because the DMM test always precedes the scope test, the scope test cannot execute in parallel until a thread completes its DMM test.
    13. Click the Clear button to remove the graph information.
    14. In the sequence editor, right-click the Auto Scheduled with Additional DMM step in the Main step group of the MainSequence and select Run Select Steps from the context menu.

      In the profiler, notice that the batch time reduces to approximately 12 seconds, which is a substantial improvement over the initial value of 30 seconds for the Sequential strategy example, in which no parallel resource usage existed.
    15. In the sequence editor, open and examine the Sequential, Parallel, Auto Scheduled, Parallel with Additional DMM, and Auto Scheduled with Additional DMM sequences to view how each sequence synchronizes access to the DMM and scope resources.

      Notice that the DMM and scope are represented by Wait steps. The example models tests that are resource-bound instead of CPU-bound, and is an accurate model for tests that must wait on instrument and I/O operations and for multi-core systems in which the number of cores equals or exceeds the number of UUTs.

      You can change the values of the DMMSimulatedTestDuration and ScopeSimulatedTestDuration file global variables to determine the effect of different resource speeds on total throughput. The example contains a ModelOptions callback sequence that sets the number of UUTs to four. You can edit the callback sequence, or you can delete the sequence and select Configure»Model Options to launch the Model Options dialog box and modify the value of the Number of Test Sockets option to experiment with the effect of changing the number of UUTs.

See Also

Model Options dialog box

Resource Usage Profiler

Synchronization Step Types

WAS THIS ARTICLE HELPFUL?

Not Helpful