Enterprise System Connectivity

Welcome to the Test Management Software Developers Guide. This guide is collection of white papers designed to help you develop test systems that lower your cost and increase your test throughput and can scale with future requirements. This paper describes the enterprise systems connectivity.

To view the other topics, visit the Test Management Software Developers Guide.

Introduction

Test management software should include test sequence creation and execution functionality that will be an integral part of your software framework. In addition to these standard test executive abilities, you are likely to want your framework to include connectivity to enterprise systems. The features, functionality, and benefits of enterprise systems and solutions keep your test systems integrated into the larger network of tools and applications used throughout your business. Whether you are integrating source code control tools or data management systems, there can be inherent challenges when integrating multiple software solutions. The connectivity you include in your framework should be based on industry-standard tools and protocols whenever possible to simplify the integration of enterprise systems.

Before going further, what do we mean by “enterprise systems”? Like the examples of source code control or databases, enterprise, in this context, refers to business-wide operations and management-level systems (predominately software-based). As a minimum, enterprise systems are used by more than a single test station or even a single department.

The enterprise systems commonly integrated into a test software framework include tools for configuration management, requirements management, data management, and communicating results. Each of these concepts has associated software tools, which in turn have interfaces for connectivity from your test-specific software. As each of these examples is discussed, the particular interfaces and the benefits of integration will be covered.

Configuration Management

As part of managing your test systems, you are probably keeping track of the specific hardware and software used in the system. This may include the inventory of instruments as well as ensuring the most recent, or most functional, software is installed on the system. These tasks are all part of the range of tasks and responsibilities of configuration management.

Obviously, configuration management involves software developers, hardware technicians, and possibly even purchasing and inventory resources. For software, performing configuration management includes connecting your test software to source code control (SCC) tools as well as managing software deployment and upgrade processes. Given that this discussion is focused on integrating enterprise software in our test framework, we will focus on software configuration management.

Software Configuration Management
Software configuration management is synonymous with source code control, or version control. That is, a key aspect of managing your software configuration is controlling which version of software is installed on the test stations – both the software you have purchased and the software you have developed. In either case, controlling the version controls the features and functionalities available.

Controlling your source code has the added benefit of organizing development efforts, especially when multiple developers are working on a project. There are several providers of source, or version, control software. Microsoft has defined a standard application programming interface (API) for source code control. Source code control systems that use the Microsoft SCC interface make it easier for your test management software to maintain compatibility with several different systems.

NI TestStand uses Microsoft SCC to offer source control integration features. In addition to choosing and configuring the SCC provider you use, files can be checked in and out of source code control by using NI TestStand workspace files. As the name suggests, workspace files can be used to organize your test projects, sequences and measurement code. Chapter 2 of the NI TestStand Reference Manual provides more details on using workspace files and accessing source code control providers from NI TestStand.

Software Deployment
As part of managing your software configuration, you can plan your software deployment strategy. Whether you choose to build an installer or use a fancy network server to distribute your test software, you will want to use care to ensure the integrity of the test stations while at the same time working quickly to keep the upgrade process as seamless as possible.

NI TestStand includes a deployment utility for distributing your NI TestStand workspace files. There is more detailed information on deployment in System Deployment Strategies. Additionally, there are NI Alliance Partners who develop tools that extend the capabilities of NI TestStand. For more on these partners and on tools for configuration management, view the Large Project Development section of the NI TestStand Partner page.

Upgrading Software
There tends to be a natural concern about upgrading software on deployed test stations. Whether you are upgrading the software you have developed or upgrading a commercial off-the-shelf (COTS) tool, your choice to upgrade will likely be based on incorporating new features or fixing bugs. In both cases, you can plan and prepare for these upgrades to minimize downtime during the transition.

Here are a few of recommendations to ease the upgrade process:

1. Use modular components to reduce the impact of an upgrade. With a test system built of modular components, you can replace a particular component to upgrade a feature without replacing the entire system or subsystem. This modularity, applicable to both software and hardware, can protect you from obsolescence.

2. Maintain backward compatibility for easy transitions. To be successful using the upgrade strategy of modular components, backward compatibility is a must. For example, if you replace the framework of your test executive, you want to be sure that the upgraded framework will still work with your existing test sequences. As another example, if you upgrade an instrument, the replacement will be easier if that instrument can be installed in the same chassis or rack that contains the rest of the system.

3. Use software maintenance options when purchasing COTS tools. In the case of software, you can usually purchase maintenance plans that cover the purchase cost of an upgrade. In the case of NI software, the standard service program (SSP) provides the latest NI software technology through automatic upgrades. Additionally, SSP customers have access to technical support from applications engineers through phone and e-mail.

For information on developing modular code, please refer to Guide to Effective Test Software Development with NI TestStand.

Requirements Management

Most engineering projects start with high-level specifications, followed by the definition of more detailed specifications as the project progresses. Specifications contain technical and procedural requirements that guide the product through each engineering phase. In addition, working documents, such as hardware schematics, simulation models, software source code, and test specifications and procedures must adhere to and cover the requirements defined by specifications.

These specifications, or requirements, are not just used by your manufacturing or production teams. Many people and processes in an organization use requirements. They may be used in planning and purchasing decisions, reporting your development progress to your customer, guiding software development, ensuring ultimate product completion or project success.

Traceability
Requirements management, then, hinges around tracking the relationships between requirements at different levels of detail and the relationship of requirements to implementation and test. Requirements traceability is simply this ability to track relationships. The benefits of managing requirements include successfully dealing with complexity, ensuring compliance, progress tracking, streamlining development efforts, and having confidence and certainty in your process and product.

Tracking the relationship from requirements to test, measurement, and control software is crucial for validating implementation, analyzing the full impact of changing requirements, and understanding the impact of test failures. To get this level of traceability, you are likely to want to connect your test software framework to the tools you have used to document or manage your requirements.

NI Requirements Gateway
NI Requirements Gateway is a requirements traceability solution that links your development and verification documents to formal requirements stored in documents and databases. With NI Requirements Gateway you can perform coverage and impact analysis, graphically display coverage relationships, and generate comprehensive reports.

The idea behind using a tool such as NI Requirements Gateway is to connect a specific requirement to the piece of code either where you have implemented the requirements or where you test whether the requirement is being covered. Once the connections are made, you can respond quickly to changes in your requirements or changes in your code. Plus, you can keep track of which requirements still need to be implemented or tested.

For more on linking your requirements to your code, refer to NI Requirements Gateway for Test, Measurement, and Control Applications.

Data Management

Data management for test systems involves more than logging test results. While result collection is a fundamental component, access to parametric data and data analysis are important features to consider as well. If databases are being used to store test-related information, then the technology for connecting to your database should be chosen wisely.

Database Connectivity
Database connectivity for your test framework will give you the means to dynamically read and write values to and from a database. While there are a variety of ways to interact with your database management system (DBMS), communicating with a database follows five basic steps:
(Note: As you may have noticed, these steps describe a session, much like how you use a session when controlling an instrument.)

Structured Query Language (SQL) is used commonly to perform steps 2 through 4. That is, once a connection to the database is created, SQL can select data in tables and read and write from the selected table of data.

NI TestStand has features to relieve you of being an SQL expert. You can configure database logging through a dialog that walks through configuring your database connection and your schema, a set of SQL commands for mapping NI TestStand data to your database tables. In addition to the database logging configuration, there are database step types for communicating with your databases directly from your test sequences. There will be more about database logging and the step types in the following sections.

Now, what about steps 1 and 5, i.e. what about connecting and disconnecting from the database? One solution is to use Microsoft ActiveX Data Objects (ADO) or the next generation ADO .NET, which is built on the .NET technology. ADO can be used to connect to databases through Object Linking and Embedding for Databases (OLE-DB) or Open Database Connectivity (ODBC) drivers. Using ADO, you get a standard programming interface to any DBMS that uses ADO.

Database connectivity and results logging with NI TestStand are discussed at length in Chapter 6 of the NI TestStand Reference Manual. Chapter 6 also gives you introductory material on databases and the technology NI TestStand uses to communicate with databases.

Logging Results
Though we are in the midst of a discussion on database connectivity, data management does not have to be synonymous with a database management system. Results can certainly be logged to a database, but you may also want results logged to a file.

Results logging is rarely done for its own sake. Granted, you want a historical record of tests that have been run, but you probably want to log results because you want to use them to debug your systems, to analyze productivity or quality, or to generate reports.

There are pros and cons to either using a database or using files. Using data in a database requires some experience with SQL or a DBMS. On the other hand, with the speed of database logging and reading you can get near real-time reports and alerts. File I/O has its own set of challenges such as performance, format, and security. An advantage of logging to a file is the inherent chronological nature of log files. If you experience a system crash, log files are a great asset for examining the cause of such failures.

In the last section, we saw that NI TestStand has out-of-the-box database logging along with a dialog-based configuration utility. Once configured, you can log all relevant information including the operator’s name, UUT serial number, test parameters, and results. Whether you are building your own database logging features or using the functionality in NI TestStand, you will build it on top of how you have implemented your database connectivity.

As an aside, the actual database logging functionality is not native to the NI TestStand Engine or the NI TestStand Sequence Editor. The default process model included with NI TestStand contains customizable sequences that implement the logging features. Refer to Appendix A, Process Model Architecture of the Using the Import/Export Properties Tool to Export Limits to a Database will give you a good introduction to using the tool for exporting parameter values.

Appendix D of the NI TestStand Reference Manual covers the database step types that can be used in your test sequences when using the Property Loader with a database.

For third-party product information on data management solutions, please visit “Data Management, Analysis, and Reporting” section of the NI TestStand Product and Solution Partners page.

Communicating Results

Although data management covers the retrieval and storage of test results, it is still important to discuss how we will communicate this information to users of the test system. Communicating results can be performed in different ways. The test system can communicate directly with the user as the test is executing. You might also want to communicate test results after the execution of a sequence of tests using reports. Test reports can be generated using different formats such as ASCII text, HTML, and XML. Which report format you should choose will depend on the features you wish to include in the report and its target audience.

User Interfaces
The user interface for your test executive is the first line of communication with the test operator. It is a very efficient method of communication because it is responsible for running the test and therefore, has access to the test results as they are being generated. There are different ways in which results can be communicated by the operator interface. First, the information can be placed directly on the main windows of the operator interface. Second, test information could be displayed in a separate window depending of the particular test that is being executed. Third, information used to elicit a response from the operator could also be included in a separate window.

The information that is placed on the main window of the operator interface should be pertinent to any test that is run by that station due to the persistence of this information across test executions. The type of information displayed on the main window of the operator interface can include the name and pass or fail status of the current test that is executing, the list of tests that should be run in this sequence, and the status of the whole test sequence.

On the other hand, there might be result information particular to each test that we might want to communicate to the operator. For example, if we are testing the frequency of a waveform, we could open a window and display the waveform as it is being acquired to the operator. This information could provide the operator valuable information if any debugging of the test needs to be performed. Because this window will be dependent on the test being performed, it is best that it be implemented in the test code to facilitate reuse in future test sequences.

Similarly, the results that need to be communicated also seek to elicit a response from the operator. If a test fails the operator might have to check the connections and try to run the test once again. Because the operator will need to be told to perform this check, a window could pop up with a diagram or image explaining how to check the connections; and once this task has been performed, it could give the operator the chance to run the test once again.

Figure 5. Dialogs can be used to communicate instructions or results to operators.

NI TestStand provides two different types of operator interfaces developed in LabVIEW, LabWindows/CVI, C#, Visual Basic .Net, and C++. The code for each of these is available so you can edit and customize them to meet your needs. Furthermore, both operator interfaces take advantage of the NI TestStand User Interface Controls. The NI TestStand User Interface Controls provide many of the functionalities you might need in an operator interface, such as displaying the sequence of tests that will be executed, setting breakpoints for debugging, showing how a sequence of tests is being executed, and displaying reports. Instead of having to write these features yourself, you can use the user interface controls in any application written in any programming language that interfaces with ActiveX. Furthermore, if you want to implement the user interface result communication in a code module, with the NI TestStand engine, you can show the window for the module as its running and then close it once the code module is done.

Reports
Although user interfaces provide a good way of communicating results as they are occurring, there is still a need to access results after a sequence of tests has already been performed. One of the best ways to communicate results after the execution of a test is to use a report. When picking which kind of report to use to communicate your results, you have to determine the format in which the report will be created. One can use ASCII text, a simple, easily readable format; HTML, which is also easy to read but provides more formatting features; or XML which offers more flexibility and is easier for a computer to read.

Using ASCII text to create your reports can be a good solution if you need to see your report in different OSs and environments. ASCII text is easily readable by most word processors or text viewers. Furthermore this format is easily portable to different OSs. Some ASCII text drawbacks include failure to provide any functionality to change the text font or add images to a document.

If you want to keep the portability of ASCII text while increasing its ability to add formatting to your report, HTML can be a good solution. HTML is readable by any Web browser, meaning that it can also be read on most OSs. Furthermore, with HTML markup, you can change the font and color of text as well as add images to your report to increase the readability of the information. Most Web browsers include functionality to print out an HTML document facilitating the generation of a hard copy of the report as well.

Finally, XML (eXtensible Markup Language) is the most flexible of the three formats, providing the portability of ASCII text and the ability to add formatting found in HTML, while adding the ability to be easily read and parsed by a computer. XML is a text-based format that uses markup to describe and organize the information contained in a document. An XML document can be combined with an XSL style sheet to create an HTML document. XML documents usually do not contain much formatting information. The markup contained in the document explains the nature of the information, not its formatting. An XSL style sheet specifies how to render the information in an XML document.

Figure 6. XML gives you reports that you and your software can read.

An example of a standard, flexible XML report format is ATML, the Automatic Test Markup Language. For more information on ATML follow the link to the article “ATML – The Standard for Interfacing Test System Components Using XML”.

NI TestStand includes functionality to generate ASCII, HTML, and XML reports automatically. By default, each of these reports is generated using a standard format. If the format used to generate the different types of reports does not fit your needs, you can customize it fully. Report Generation Explained is a good description of how reports are generated by the NI TestStand process model.

Conclusion

The enterprise systems commonly integrated into a test software framework include tools for configuration management, requirements management, data management, and communicating results. Configuration management provides system developers the tools to determine the correct software and hardware that should be run on a station. It is important, especially on the software side, to guarantee that the most recent and functional version of the software is running on the station. Requirements management and traceability facilitates tracking the implementation of previously specified functionality in a test system. Furthermore, requirements management and traceability helps test engineers determine the effect of a test failure on multiple levels of requirements. Data management includes both the recording of test results and loading of parametric data to specify the properties of tests. Finally, result communication facilitates the sharing of test results to different users at different stages of the testing process.

Relevant NI Products and White Papers

National Instruments, a leader in automated test, is committed to providing the hardware and software products engineers need to create these next generation test systems.

Software:

• NI TestStand Test Management Framework
• LabVIEW Graphical Programming Environment
• SignalExpress Interactive Measurement Software
  

Hardware:

• Modular Instruments (Oscilloscopes, Multimeters, RF, Switching, and more)
• Multifunction Data Acquisition
• PXI System Components (Chassis and Controllers)
• Instrument Control (GPIB, USB, and LAN)
  

White Papers
NI offers a Designing Next-Generation Test Systems Developers Guide. This guide is collection of white papers designed to help you develop test systems that lower your cost, increase your test throughput, and can scale with future requirements. To read the entire developers guide, you can: Download the PDF (90+ page) version or view the Web version of the Designing Next-Generation Test Systems Developers Guide.

0 ratings | 0.00 out of 5

Print

Reader Comments | Submit a comment »

 

Legal
This tutorial (this "tutorial") was developed by National Instruments ("NI"). Although technical support of this tutorial may be made available by National Instruments, the content in this tutorial may not be completely tested and verified, and NI does not guarantee its quality in any way or that NI will continue to support this content with each new revision of related products and drivers. THIS TUTORIAL IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND AND SUBJECT TO CERTAIN RESTRICTIONS AS MORE SPECIFICALLY SET FORTH IN NI.COM'S TERMS OF USE (http://ni.com/legal/termsofuse/unitedstates/us/).