Test Data Management with LabVIEW and DIAdem

If you collect extensive test data, build test rigs, or take measurements, you have a vested interest in your data. Each data point you save to disk is worth money. By having a well organized test data management system, you can get the most out of your data investment. This document discusses how to use NI software to create a test data management system.

Making the Most of Your Data Investment

The common vision of Test Data Management involves saving any acquired measurement data to a unified data storage location. From this location, any end-user is able to find, select, and evaluate datasets, thus providing information to engineers as well as to managers or controllers. The majority of users know what data they are looking for, what kind of evaluation should be executed, and what information this evaluation provides them with, but know little regarding the technical details of the data storage system.

The entire TDM system can be set up using NI LabVIEW, NI DIAdem, and a standard SQL database. Using NI LabVIEW to acquire and analyze the data, you can use the NI LabVIEW Database Connectivity Toolset to port the information to a standard SQL database. NI DIAdem can then access this database to run the desired reports with the specified information. A properly structured database allows for a central data storage location, where modifications in the database will automatically be reflected in both LabVIEW and DIAdem.

During this discussion, for illustrative purposes, we will reference the "DataMine" Test Data Management example program, which is available for a free download here.

Database Model

The key to the entire TDM system is a well structured database. There are many ways to architect a test data management system. In this article, we will use one particular format to illustrate the core concepts. This "DataMine" example TDM database contains the following database tables and view tables (registered queries):

• Setup/INI/Look-up Table : This database table provides the initialization information and lists all other tables (names, columns, etc) that exist in the database. It maps what information is sought by LabVIEW and DIAdem to the actual locations/names in this particular database. In the "DataMine" example program this is called the "GlobalParameters" table.

• Data Table : This table contains the meta-data of all the data LabVIEW has acquired. LabVIEW inserts the individual records of meta-data each time it acquires a new data set. In the "DataMine" example program this table is called "DataTableWrite". It is vital that all the descriptive attributes for searching and navigating be included as separate columns in this data table. Some specific examples you might want to utilize include UUT, operator, test program, cost center, serial number, part number, test date, temperature, etc. By setting up this table at the outset with all the attributes you will care to store and search on later, you will minimize changes needed to be made to the overall database structure.

• Search Parameter Tables
  • This table lists a series of selection parameters, all of which must have a corresponding column in the Data Table. You can have multiple search parameter tables to group together different sets of meta-data parameters. LabVIEW looks at the default search parameter table to see what meta-data parameters to get from the user and write into the Data Table. DIAdem can choose between these different groups of search parameters to provide different customized search parameter lists to build query conditions to find particular data sets of interest.
  • Parameter Enumeration Table : Each enumerated-type meta-data parameter will have its own lookup table listing its allowed enumerated values.
  • Search Parameter List : This is a table that lists all the possible search parameter tables. It is entitled "SearchParLists" in the "DataMine" example program

• View Tables
  • View Tables (Registered Queries) : These are preconfigured queries registered in the database which you can access from LabVIEW or DIAdem as if they were data tables. They provide a view onto the returned data of a particular query (thus the name "view table") of the table containing the meta-data records, "DataTableWrite" in the "DataMine" example program. DIAdem uses these view tables to display a certain number of columns (parameters) in a particular order for all the records which satisfy the query conditions issued. If you change the appearance of the view table in the database, the dialog showing the data records matching the query conditions in DIAdem will automatically change to this new appearance.
  • View Table List : This is a table that lists all the possible view tables (registered queries). It is entitled "ViewTables" in the "DataMine" example program.

Because the database is the centralized location for all information, a change in the database will be reflected in both LabVIEW and DIAdem. For example, if you needed to add a new allowed value to an enumerated-type test parameter, such as "1000 PSI" for the parameter "Pressure", you would only have to add "1000 PSI" to the parameter enumeration table for "Pressure" and the new value "1000 PSI" would automatically appear as an allowed state of the "Pressure" parameter in DIAdem and LabVIEW.

Getting Data In

The database serves as the centralized data storage location. However, it is vital to have a simple tool for porting data to the database. With the NI LabVIEW Database Connectivity Toolset, you can easily insert or update data table records (or create tables) in any SQL database from LabVIEW.

The flow chart, shown below, demonstrates the process by which LabVIEW writes data to the Database. The process begins with LabVIEW reading the initialization table from the database, which tells LabVIEW the names of the columns and tables within the database.

Once LabVIEW reads the Search Parameters Table, the selection parameters appear in the VI front panel (see Figure 3). After LabVIEW acquires the data for this run it writes these selected meta-data vlaues plus any calculated meta-data values to the central database.

At this point, you have successfully stored your test data and the test's meta-data values into the Database Management System using NI LabVIEW.

Getting Data Out

Once the meta-data has been imported into the database, the key to test data management is the ability to easily access the stored test data based on specific search parameters. Using NI DIAdem, you can design customized dialogs and locate the data you want by interactively filtering the descriptive attributes (meta-data) you defined within your Search Parameters Table and stored with LabVIEW. The DIAdem dialog below shows a list of search parameters (stored in the database) that you could use to build query conditions to find the data you are interested in.

Then, based on the selections made within the dialog, DIAdem queries the Data Table and returns all data that fits the selected specifications in the format defined by the user-selected View Table in the Database. The process by which DIAdem accesses the database information is defined in the following flow chart:

From there you can select which data you specifically want to look at, and use standard DIAdem functions to analyze and generate reports. You can either use the built-in features of DIAdem interactively or you can fully automate your analysis and reporting tasks, and you can analyze and report the meta-data values themselves (pictured below) or the stored waveforms they describe.

Using a standard SQL database, NI LabVIEW, and NI DIAdem, you can optimize your data management. No longer is data scattered throughout your company, but now it resides in a single location where it is easy to access, report, and utilize.

For a specific example of Test Data Management using NI LabVIEW, NI DIAdem, the LabVIEW Database Connectivity Toolset, and Microsoft Access, download the DataMine Example Program .

For assistance getting started writing data to a Database with LabVIEW, the examples that ship with the Database Connectivity VIs provide an excellent resource. Within LabVIEW select the "Help >> Database Toolset VIs and Examples" pull-down menu for examples that demonstrate how to write LabVIEW data to a database. For more information, you can also view the Database Connectivity Toolset User Manual.

For an example of querying data from a Database with DIAdem, see the Example Program: Using VBScript to Access Select Values from an MSAccess Database and Use them in DIAdem.

Reader Comments | Submit a comment »

Good start but could have had more details on database and overall design of project. Figures were hard to read also.
- David Stewart, CDS Engineering Solutions, Inc.. cdses@ieee.org - Dec 8, 2003