Considerations for Managing Data Acquisition System Configuration Information

For design engineers who develop computer-based data acquisition systems, configuration files are a method to manage configuration information for the system. Unlike conventional configuration GUIs, configuration files provide a portable, flexible method to manage configuration information.

Background

In computer-based data acquisition (DAQ) systems, managing configuration information can sometimes seem like a daunting task. For smaller systems, the developer can many times get by with simple runtime configuration graphical user interfaces (GUIs) to pass configuration information directly to the portions of the initialization code in the application. The NI Measurement and Automation Explorer (MAX) is an example of an interface that works well to configure simple, smaller systems. For larger systems, though, configuration interfaces like MAX are not adequate on their own because in many cases the system may be made up of a large number of channels distributed over a network of computer systems or nodes.

Additionally, it may be desirable to have a separate application that handles the pre-runtime configuration on a separate remote terminal and having a simple runtime configuration interface will not be adequate. The developer of these larger systems requires a method to generate, modify and track configuration information in a way that is portable and scalable to make handling the configuration more efficient and usable at the various levels in the overall system. Many times this type of configuration system is used in conjunction with MAX, where MAX handles the basic module configuration and then more advanced information is included in a configuration file.

If the composition of the distributed system should change later on, it would be desirable to make a quick modification to the configuration data to support the new system configuration. Configuration files allow this type of flexibility and support the needs of the developer architecting larger distributed DAQ applications. Since modifications can easily be applied to the configuration file in one location, the application code does not need to be changed to reflect the necessary updates.

It is important to put some careful thought into the information required to completely configure the DAQ system before implementing a configuration file scheme. This document is the first in a series to assist the developer in generating a reliable and scalable method to manage configuration information for a distributed system. This series of documents discusses some of the various items important to developing a configuration scheme. It is the goal of this document to bring up many of the general concerns in a large distributed DAQ system and discuss ways to handle the associated configuration information appropriately. The result of the discussion through this series of documents is the introduction of a framework that can be applied and scaled to various types of system architectures on various platforms that may or may not include hardware configuration. Please remember that the methods discussed here are only recommendations and based on your system composition you may find that additional methods work as well or better for your particular application. The basic concepts discussed here, though, will still apply to any type of configuration managed system.

Configuration File Composition

This document is the first in a three-part series that addresses the concepts of system configuration. This document will focus on configuration considerations relative to National Instruments based platforms, specifically PCI or PXI based systems. It should be noted, though, that many of the concepts discussed here can be applied toward many other data acquisition platforms like Vision, Motion, Compact FieldPoint and CompactRIO systems. The two subsequent documents in this series will address actual implementation of a configuration file format and a set of tools to manage the configuration. Links to these papers can be found at the end of this document. To begin, there are several things we need to know about the proposed system architecture before we start formulating a configuration format.

DAQ Hardware Configuration:
One of the first things we need to know is if the system is distributed with multiple controllers making up the system, or if it is just a single controller system. In a distributed system using Ethernet for communication, for example, each node will have its own unique configuration information, plus additional information about the other nodes to which it will communicate, including a host or master controller. If the distributed system uses Ethernet for communication, for example, then each of the distributed nodes would need to know the IP addresses and ports of any of the other nodes to which it needs to communicate. This information should be included in the configuration file. Distributed systems also bring up another concern regarding configuration files; should the system have one large configuration file for the entire system, or smaller individual files that are dedicated to each of the nodes? We’ll discuss this in more detail in the subsequent section titled Configuration File Implementation Considerations.

Regardless if the system is distributed or not, any of the aforementioned NI systems will have specific information about the single system or each system node that makes up the distributed system. This will include information about the number of slots available in the chassis used to house the controller and I/O modules, which I/O modules populate which slots and which channels will be used in each of the I/O modules.

Slot numbers for the I/O modules and the I/O module type are fairly straightforward items to take care of, although it may be beneficial to include additional information in this area, including the total number of slots available for a given chassis (some slots may not be currently utilized) along with the chassis model number for referencing in the system. In a PXI system it is important to know the slot number for a particular device because the device is referenced in DAQmx by its slot number. Obviously we need to know which I/O module is in which slot so that if we have mixed I/O we know what format the data will be for each of the modules.

Some additional care needs to be taken when referencing the channels themselves. Having a naming convention for each channel that makes sense and is easy to associate with the data acquired from that channel is very beneficial to keeping track of where data is coming from and what it means to the end user of the data. For instance, a technician may connect an accelerometer on the low-speed side of a gear box to an analog input on chassis 2, module 4, channel 8 of a distributed DAQ system. The technician has a much easier time understanding what the data means when he views it on the a screen if he or she can associate the data with a name like “Low Speed Accel”. The application, though is going to have to make the association between the name the technician has given this channel, and the chassis, module, channel and physical sensor location to which the signal is connected. This mapping should be part of the configuration file.

In our example, the data we are acquiring is coming from an accelerometer which returns data in units of g-peak. It is usually beneficial to include information about the transducer converting the physical phenomenon in the configuration file. For this particular transducer we might include the sensor type, model number, any sensitivity information (mV/g-peak in our example) and calibration information that will be necessary to convert the raw voltage acquired back to engineering units.

There may be other configuration information that is necessary to set up the channel on the module itself. For instance, some DAQmx devices include range, coupling, excitation and/or terminal configuration information. You should look at all the information required to properly configure a module and all of its associated channels and plan to include entries in the configuration file for this information. Additionally, you may also want to include information about what should be done if a module is missing. If a module is missing, you may want to inform the user this is the case and shut down the application or take it to a standby state. There may be some modes in which the system runs, though, where certain modules are not required. If this is the case, then it would be acceptable for the application to run normally. To handle these different options, you may want to have a field associated with each device as to whether or not its presence is required for the system to run properly in various modes.

Application Software Configuration:
Many larger DAQ systems are event driven. These systems monitor for various events to occur and based on these occurrences, they log or store real-time data to file. It will be desirable, then, to store the conditions that must be met for the data storage to occur. This might include monitoring results from calculations that are occurring real-time on the system or monitoring some external process for certain conditions. It would be tedious and time consuming for the operator to enter this information each time the system is ran, so having this information included in the configuration file makes sense. Since many DAQ systems require storage of the data acquired, it would be valuable to include the number of files, file location and size for the logging operations.

There may be types of files related to the application itself that you may want to monitor. These may include dynamic linked libraries (DLLs), limit files, user interface files, sound and video files or any other file that may be used during runtime. For instance, to build a robust application, the developer may need to ensure that certain versions of DLLs are used. The information stored in the configuration file may include valid versions that can be used with the current software implementation. At run-time, the application can compare these values against the actual values of the installed software to determine if all the necessary pieces are available for the application to work properly.

Network Configuration:
We mentioned previously that in a distributed Ethernet system it will be necessary to know the location of each of the distributed nodes on the network. To accomplish this, the configuration file will need to store both the IP address and port used to communicate with each of the nodes. Along with this information, you may want to include timeout values for specific network operations. Also, like in the hardware module implementation discussion previously, if a node is not present there may or may not be required action. Having this information stored along with other items for the node will allow the application to handle the situation appropriately.

Another thing we may want to do with each node is limit the number of connections that can be made to the respective nodes by others on the network. If a node is strictly a data server that is only streaming data to a single data client, for example, then you may want to allow only a single connection between the data server node and the data client node. Conversely, if you have a host machine that will be communicating to all of the slave nodes on the network, then you would obviously like to increase the number of connections that can be made to this system, although you may still want to have some upper limit. It would be very beneficial to include this information in the configuration file.

User Configuration:
With large DAQ systems there will often be multiple users that will access the application in various ways. It is often desirable to include user information such as user privileges, login levels and user permissions in the configuration file to manage how various users interact with the application. For instance a technician who is running actual tests with the DAQ system may be limited on the kinds of information they can access and modify. Conversely, a design engineer may need to have access to the entire configuration to make sure things are set up appropriately for the various tests that will be ran by the technicians. The privileges for these two individuals are very different, so it would be beneficial to have fields in the configuration file that handle privileges, passwords and other information used to restrict and define access to the application and various parts of the system.

Configuration File Implementation

The focus of this section is on implementation of the configuration file itself. We need to come up with some scheme to organize and keep track of all the configuration information in software. To start, we need to develop a format that will work well on the platform(s) we are utilizing for the system.

Platform independence is always an attractive feature when building larger distributed data acquisition systems. Constructing a solid configuration file format that doesn’t depend on a specific platform means that changing the system components, OS or platform will not affect the configuration file architecture. There are several different formats we can consider for implementing file configuration. The formats we’ll discuss in this section include developing your own configuration file format, system registry files, database files, spreadsheet files, Extensible Markup Language (XML) files and configuration settings files (files ending with the .INI extension). We’ll discuss the pros and cons of each and then make a recommendation on a particular file format. We’ll also need to discuss methods to populate the configuration file. In the previous sections we discussed what kinds of information should go into the configuration file, but we will need to know how to gather this information to insert into the file.

Developing Your Own Configuration File Format:
Sometimes developers may feel that their system is too unique to describe with some conventional file format. LabVIEW, for example, has a cluster datatype which does not map readily to a particular file format, so the developer may write the cluster information directly to a file as binary data so that it is either very difficult or impossible to interpret the information on another platform. This method doesn’t support the platform independence we discussed earlier because it is relatively proprietary (LabVIEW is the only application that can easily interact with the configuration file).

While making up your own format may seem the quickest way to generate a configuration file, in the long run it will generally cost the developer more time because of various limitations. If we want to use the configuration file on a platform that utilizes C, for example, we now have to build a translator to go from the LabVIEW cluster to a structure or some other format that C can understand. For larger DAQ systems, this method is not recommended.

Using System Registry Files:
System registry files are many times a very attractive method for storing configuration information because the file format is very well defined and there are a set of tools that can be used to easily access and modify the information. Registry files, though, are OS dependant. Many operating systems (OS), such as the LabVIEW RT OS, don't use system registry files but use other more standard file formats (like INI files) to store various configuration information. System registry files are not recommended for storing DAQ system configuration information because they don't achieve the goal of maintaining platform independence.

Using Databases:
When considering existing formats to use for a configuration file, one of the first that come to mind is the database. Databases are a very attractive format because they are organized in a manner that works well when defining a large DAQ system. It is easy to develop individual tables for various pieces of configuration information and make relationships between tables and cells within various tables. Also, databases will have an interface that allows the user building the configuration file to easily add and edit configuration information.

The challenge with this method is that many times the tools used to access the configuration file are platform specific and may restrict the user in a couple of ways. The first, most obvious way is that the user configuring and editing the file must have access to the OS or platform on which the database software is designed to run. A secondary issue is that the Application Programming Interface (API) tools provided to access the database must work on the platform on which the DAQ system is running. Because of these limitations, many times a database is not the most desirable way to build, maintain and utilize a configuration file. Many times the platform that is used to build and configure the file is not the same as the platform that utilizes the configuration information. For example, Windows may be used on the machine that sets up, edits and deploys the configuration files, but the DAQ system itself may very well be utilizing a real-time OS, like LabVIEW Real-Time, to perform acquisition.

Using Spreadsheet files:
Spreadsheet files are similar to database files. Again there is an interface provided by the spreadsheet vendor that already exists to interact with the information in a spreadsheet based configuration file. While many spreadsheet applications have proprietary formats that again will be limiting on a platform or OS basis, most of them can also handle more standard generic formats like tab delimited or comma separated value formats. While these formats do not allow access to some of the advanced features the spreadsheet application has to offer, they still allow basic editing of the content. Since the data is an ASCII file with tab or comma delimited information, it would be possible to create your own set of tools to access the data in a customized GUI, but this might not be the best use of your time.

An important thing to note about the interfaces that a database or spreadsheet vendor provides to the developer and/or user is that the interfaces are very generic and may not provide enough information to a user building and editing the file. The developer may very well wish to develop a GUI specifically designed to aid the end user in building and maintaining a configuration file in an effective manner. Therefore, existing database and spreadsheet applications may not be all that attractive or even necessary to someone designing a large DAQ system.

Using XML files:
At first glance, XML files seem like they might be a good option to consider for storing configuration information. The files are similar to HTML documents and use tags to identify pieces of information. The files are human readable. As of the writing of this paper, though, many of the tools available to parse and manipulate XML files are platform specific and the bulk of them are designed to work on the Windows OS. The lack of platform independent tools makes XML and unattractive option for configuration information.

Using INI files:
Configuration settings files or INI files as they are commonly referred then become a very attractive option for building and maintaining configuration file information. You may already be familiar with this type of file as they are often used to hold configuration information for various applications on a computer. One of the things that make these files so attractive is the fact that they are platform independent because they are simply ASCII files with a predetermined format. Many APIs supply tools to access INI files. Both NI LabVIEW and LabWindows/CVI provide tools to easily parse INI files. With these tools, we can build a customized interface that facilitates building and maintaining the configuration file.

INI files use two main objects to organize information. These two objects are sections and keys. Sections themselves don’t have values, but are used as a method to group keys together in a logical manner. Keys usually have a value associated with them so that you can visualize a key as a variable that can have various values associated with it made up of Boolean, floating point, integer or string datatypes. Figure 1 shows an excerpt from an INI file.

[+] Enlarge Image
Figure 1: Excerpt from INI file

Even though the structure of an INI file seems rather simplistic, the nature of them makes them very powerful and they can be used to store complex configuration information for large DAQ systems. We recommend using the INI file structure to build a configuration file because of the platform independence, simple structure and availability of tools to easily access the INI file.

Populating Configuration Files:
Some of the methods used to populate various fields in the configuration file are more obvious than others. Information like channel names, event parameters and user privileges will be entered by the user configuring the file initially. There will need to be some sort of tool used to act as an interface between the user entering the information and the actual configuration file used to store the data. This tool is called an editor. In many cases the editor is an interface built by the developer of the system to assist the user populating the configuration file. This type of editor is full of features that help streamline the configuration process by presenting various parts of the configuration in a way that the user will easily understand. Information that is configured on a regular basis can be placed on certain panels of the editor GUI where they can be accessed quickly and easily. The developer can also include help documentation that will help the user understand what certain fields in the GUI are used for and what kind of information is valid for entry. An editor GUI also abstracts the configuration file to the user, so they don’t have to understand what fields apply to particular configuration parameters and the correct syntax for entering items into the configuration file. Obviously, this type of editor can be quite elaborate and can take a considerable amount of time to implement. But often, the time spent up front in development of the editor GUI is more than offset by the time that is saved every time a user has to go into the configuration and make additions or modifications. In general, developers don’t want users to work with configuration files directly since modifying them manually is prone to error. Developing a comprehensive editor is often a necessary task that needs to be budgeted as part of the development process.

There are other instances where an elaborate editor GUI is not necessary. At development time, the developer may want to make quick changes to certain fields within the configuration file without running a GUI. In this case, the developer will use an intermediate or even a basic editor to modify the contents of the configuration file. The intermediate editor may give some functionality such as the ability to query the configuration file for values associated with specific fields and the ability to automatically populate other fields with default values.

In contrast, the most basic editor may be just a text interface, where the user of the editor is required to understand syntax and formatting within the configuration file and the user will handle these details directly. In the case of an INI file, for example, the intermediate editor may be a basic GUI that queries for sections and keys and gets the associated values and then takes care of updating the INI file by formatting and writing all of this data back into the configuration file. The user does not have to understand the formatting syntax required for an INI configuration file, but still needs to understand what the various sections and keys mean in the context of the system configuration. The most basic editor for an INI file would be some sort of text editor like Windows Notepad. Notepad will open the file and display the raw ASCII characters in the file. Editing the file in this manner requires that the user understand specific formatting and syntax details about the INI file structure along with an understanding about how each of the fields in the configuration file apply to the system.

The underlying tools we’ve discussed previously that allow the user to edit the configuration file can also be used at runtime on the system that actually utilizes the configuration information. Once the configuration file is properly populated, the file will be deployed to the application that will use the file at runtime. While we’ve discussed how the user or developer interfaces with the configuration file to manually enter configuration data, the DAQ system will need to access the same information to properly set up hardware and organize the data in a way that can be used later on in the application. The developer needs to consider this when building the configuration file and any tools that will be used to access the information in the file. Note that the tools that are used at configuration time by the configuration GUI are the same tools that are used by the DAQ application at runtime.

In conjunction with data retrieval from the configuration file, there are usually a set of tools provided to interface with the hardware configuration provided by the maker of the hardware so that configuration information can be applied to the hardware. With the National Instruments platform, devices like DAQ, Modular Instruments, RIO, CompactRIO, Vision, Motion, and Compact FieldPoint all have a programming API to properly configure the hardware. Understanding how these APIs function and the format required for the configuration information is very important when coming up with a scheme to handle configuration data from initial configuration setup to runtime access of the configuration data.

Configuration File Deployment:
Now that we’ve established what type of file we are going to use to store configuration information, how to interact with the configuration file and some methods used to gather the information in the file, we can address how many files we will need and where we should store the files once they have been built and are ready to be accessed by the system. It’s possible that in a distributed system we may have one machine or controller that acts as a host to the other controllers. This host will generally act as the master device, sending commands to the other slave controllers to tell them what to do. In a distributed DAQ system, for example, these slave controllers may each be acquiring data and will need module and channel configuration information. It’s possible that the developer may want to have a unique configuration file for each system in the distributed network. This is advantageous because the basic framework for the configuration file will be the same for each slave controller in the system. Once the basic framework has been developed, the individual files can be customized for the particular distributed slave and deployed to that system.

We can further enhance the functionality of using separate configuration files by having a single file that contains global information that will be used by every node and then individual configuration files for data that will be node specific. Another way to organize the configuration information for each of the distributed slaves is to put all of the configuration data in to one large file. Each of the slave systems would then have access to this single file and would parse configuration information relative to that particular system. The advantage we have with this scenario is that we only have to manage one configuration file. Data that is used by multiple nodes in the distributed system does not have to be duplicated in the configuration files for each of the nodes. The disadvantage is that the file becomes more complex because we have combined multiple system configuration information into one file and we will have to build additional functionality to parse out data relative to each system.

Generally, in a distributed system, the host machine has the interface for configuring and managing the configuration file or files. It may be desirable, then, to keep the file permanently stored on the host. This especially makes sense if we only have one configuration file for the entire distributed system. In the case where we have an individual configuration file for each subsystem, it still may be desirable to keep all of these files in one central location on the host machine. The host then can organize and maintain the files. At runtime, each of the subsystems in the distributed system would then connect to the host and download the appropriate configuration file and utilize the file to properly configure each individual system. By simply using different names for each of the configuration files we can easily distinguish between the various files for maintenance and runtime operations.

The other option is to permanently store each of the configuration files on each controller for each of the subsystems. This makes things easier to program on each of the distributed controllers at runtime, because the file is stored locally in memory and is simply an exercise in file I/O. The drawback is that at configuration time, the host must go out and obtain each of the configuration files from each of the subsystem controllers, edit the file and then store it back on the controller. There may be some cases though, where this method is more desirable.

Conclusion

Developing a comprehensive configuration management method for large distributed DAQ applications can be challenging. With some careful thought before main application development begins, the developer can implement a flexible and expandable configuration format that will reduce configuration time during testing and provide a robust method to ensure the system is set up properly for deployment.

5 ratings | 3.80 out of 5

Print

Reader Comments | Submit a comment »

Some free tools exist for creating INI files directly from LabVIEW Clusters. See http://www.mooregoodideas.com/ReadWriteAnything.htm or openg.org
- shawn_d_walpole@mooregoodideas.com - Mar 19, 2008