Developing Data Logger Applications with LabVIEW

This paper examines the benefits of creating software data loggers as well as the fundamentals of building these systems. Engineers can use the concepts presented in this paper to design reliable, high performance data loggers today that can easily integrate with future technologies. Concepts from basic, distributed, and portable data loggers will be covered. As a case in point, a temperature data logger will be discussed.

Introduction to Common Data Logger Requirements and Tools

A data logger application involves acquiring data, performing some analysis, storing it, and then recalling it at a later point for some type of analysis and presentation. There are a variety of reasons that engineers may need to recall stored data. Engineers can use stored data to analyze a product or process and look for ways to improve it. Engineers can also use it for debugging and localizing problems. Or, the data may be necessary to prove something to a regulatory body at a later date. Though the applications and data vary, it is important to remember that the tools needed to design different data loggers are quite similar.

Every data logger has a handful of core features. First, engineers need to acquire and log the data to a storage location. Second, engineers must view the data both during and after the acquisition. Third, engineers need to record alarms or events triggered by the data. Fourth, device networking should be simple and transparent to engineers. Fifth, implementing different types of security on the data should be easy. To help with these five requirements, the LabVIEW graphical development environment and LabVIEW SignalExpress interactive measurement environment provide an out-of-the-box data-loggin solution. In addition, LabVIEW provides a specific tool for quickly and easily architecting monitoring applications. This software tool is the Datalogging and Supervisory Control (DSC) Module.

This paper will examine the software components of several different types of temperature data loggers. Basic data loggers, distributed data loggers, remote data loggers, and redundant data loggers will all be discussed. High-speed data loggers will also be discussed as a unique use case.

Basic Data Loggers

Use case: Logging data from a handful of thermocouples over a short period of time

A basic data logger system has all of the elements of a data logger, but usually involves only a small number of I/O points and a relatively simple set up. The analysis required during and after the acquisition is also fairly simple. The primary care abouts for a basic data logger include simple configuration and the flexibility to take common measurements, such as temperature or strain. National Instruments has two solutions for basic data logging applications: LabVIEW SignalExpress and LabVIEW.

Configuration-Based Tools for Data Logging
The first solution is a configuration-based logging application software tool called LabVIEW SignalExpress. LabVIEW SignalExpress allows engineers to create data-logging applications without having to program. It provides a user-friendly environment where all acquisition configuration, analysis, and logging is accomplished through interactive steps. It has built-in functionality for acquiring, scaling and storing data as well as basic alarms and events. After the acquisition, data can be recalled and visually inspected or exported to an interactive analysis tool such as NI DIAdem.

For example, a simple temperature data logger is very easy to configure with LabVIEW SignalExpress. By combining LabVIEW SignalExpress with a USB data acquisition solution, such as NI CompactDAQ, you can build a multi-channel logging solution and acquire data within seconds. Connect the the thermocouples to the direct connectivity on the modular measurement modules used in NI CompactDAQ. Connect the USB cable to the device so that the device is automatically configured. Once recognized, LabVIEW SignalExpress automatically begins acquiring live data from your measurement device. Engineers can view the traces while they are being acquired, stream the data to disc, or dynamically execute an event if a temperature goes out of range.

When engineers need to expand the capabilities of this simple temperature data logger, LabVIEW can help. By programmatically controlling the acquisition and logging or incorporating any of the 500+ analysis functions, LabVIEW can easily extend LabVIEW SignalExpress functionality. LabVIEW seamlessly integrates with LabVIEW SignalExpress by generating LabVIEW code directly from your LabVIEW SignalExpress projects. LabVIEW also has the ability to incorporate a wider range of measurements, for example OLE for Process Control (OPC) devices.

Simply wire the signals into a Write LabVIEW Measurement File VI. This will be stored in a file that can be read by NI DIAdem, Microsoft Notepad, or Microsoft Excel. Or, use the Read LabVIEW Measurement File VI to display the data. The Comparison VI can be used to set up simple warnings.

LabVIEW Express VIs work well for simple stand-alone data loggers. It is easy to acquire, log, and present data. It is also easy to incorporate any analysis functions. 

Long-Term Logging

Use case: Logging data from a handful of thermocouples over a long period of time

When a data logger is expected to run continuously for a long period of time, such as days, months or even years, there are some special requirements that must be considered. The system must have a reliable, efficient mechanism for data storage. Operators and engineers must be able to view the data easily from a range of locations. Alarms and events must be configured and stored for troubleshooting, diagnostics, and validation.

The DSC Module is the ideal tool to help engineers with long-term data logging applications.

Data Storage Mechanisms
For more complicated data logging applications, it becomes critical to have specialized tools to assist with data storage and preservation. There are different ways to store data, and each has advantages and disadvantages. A database is the best way to store large amounts of data. Databases allow engineers to store data in a structured manner as well as track any changes made to the data. This helps to ensure the integrity of the data.

Many ways to store large data sets exist and each has its own advantages and disadvantages. The two primary types of databases used are relational and streaming databases. Relational databases are more traditional databases often used in business applications. Although extremely flexible, they are not optimized for disk space and fast throughput. Streaming databases, on the other hand, are designed for quickly storing large amounts of data to disk. However, an engineer cannot define tables or other structures and therefore loses some flexibility and search capabilities. When building a database, some of the most common challenges are designing an easy-to-use, scalable file structure and data structure. Without the right tools, engineers can spend hours designing and modifying their data structures. The DSC Module features a streaming database designed to meet the needs of measurement and control applications. With built-in data hierarchy and data structure definition, the database provides superior space utilization and data throughput. The LabVIEW 7 DSC Module adds relational qualities to the database for faster, more effective searches.

Real-Time and Historical Data Visualization
Logging data to disk is not the only challenge engineers face when designing these applications. Engineers also need to view the data. There are two ways to view data. First, engineers can view data from the server during the acquisition. This is also known as live data. Engineers need to look at live data to monitor the status of a running system. It is important to understand the type of server engineers are connecting to and choose a software package that will help engineers communicate easily. One of the most common servers is OLE for process control (OPC). Finding a software package with a built-in OPC server/client can save valuable development time.

The second way to view data is after it has been stored in a file, also known as historical visualization. Use this data for post-acquisition analysis and presentations. The challenges associated with this are similar to logging the data. Without the right tools, engineers must know the file and data structures before analyzing and viewing the data. Choosing a software development tool that does this can save time.

The DSC Module builds on the LabVIEW analysis and presentation functions to make this easy with built-in OPC connectivity and visualization for real-time and historical data.

Alarms and Events Management
When acquiring large amounts of data or data over long periods of time, engineers are typically less interested in the value of each data point and are more interested in significant changes in the data values. These changes can be monitored using alarms and events. It is important that the history of these alarms and events be preserved, so they can be analyzed at a later date. Key features needed to support alarms include the ability to generate an alarm, store it along with the associated data, and recall the alarm and all of the relevant information after the acquisition. Relevant information may include when it was triggered, who acknowledged it, and at what time it was acknowledged. With the DSC Module, engineers can configure and monitor alarms using the HMI Wizard, allowing them to focus on what causes the alarms rather than programming an alarm infrastructure themselves.

In summary, best practices for long-term data loggers include:

• Reliable, efficient data storage
• Real-time and historical viewing
• The ability to configure and log alarms and events

Distributed Data Loggers

Use case: Logging data from distributed thermocouples over time

When designing a distributed data logger, it is a good idea to look at the entire system first. At the top level is the backbone of the system. The backbone of a distributed system can be simplified to key servers and the network. The software running on the key servers must manage network transfers, data management, data visualization, alarms and events, and security. A key characteristic of the backbone is that it must be able to communicate with the rest of the hardware through a common protocol, such as TCP/IP. The software used at each machine in the network must support the same communication protocols. In addition, the key servers should support a variety of communication protocols so they can interface with legacy and next-generation machines in the system. Choosing a software package, such as LabVIEW, that supports many industry standard protocols is critical to reducing development time for each machine on the network and integrating networked devices with the key servers.

Data Management
The key servers of a distributed data logger should have the ability to log data. The more complicated a logging application is, the more critical it becomes to have tools that do this well. As discussed above, small amounts of data can easily be stored in text or spreadsheet files; however, larger amounts of data benefit from more sophisticated data storage formats. Distributed data loggers have similar storage requirements as mentioned in the long-term data logger section. On top of a streaming database, a networked database is beneficial. Networked databases enable engineers to log data to a central location, build redundancy into the system, and access data from different computers on the network.

The built-in historical database in the DSC Module is networked. Engineers can simply log data to any computer if they know the machine name and database name. The DSC Module has a tool, the Historical Data Viewer, to help engineers create, archive and compact databases locally or remotely. Locally, databases can be renamed, detached or deleted. It is important that these actions can only be performed locally to prevent accidental loss or destruction of the database.


Data Visualization
Besides viewing live and historical data, distributed systems need more flexibility to view data from different locations. For example, engineers may need to be able to view the temperature on a tank in another facility or even from home.

Viewing live data from a single machine is a relatively straightforward operation. To monitor live data, engineers can create a user interface, such as a LabVIEW Front Panel, and view it on a local monitor or embed it in a Web page using the LabVIEW Remote Panels feature. Using remote panels is very convenient if one person needs to log in and view or take control of the process. However, if more than one person needs to interact with the process, then a more sophisticated client/server architecture is necessary.

In order to view live data from multiple machines, each machine must be running a client, whose job is to query for the required data when asked. It is important to understand what type of server the application is going to be connecting to and choose a software package that will help make the communication easy. The DSC Module not only has an OPC client/server built-in, but it is designed to help engineers quickly and easily set up a client/server architecture.

Managing Alarms and Events
Again, the alarms and events requirements for distributed systems are similar to the long term data loggers, except that engineers must be able to view and act on them from any machine. LabVIEW with the DSC Module has tools to easily configure alarms and events, automatically log them to the database with the associated data, and to view them from any location.

Networking
As a data acquisition system grows, data can be stored on multiple computers and monitored centrally. Or, it can be stored on one central server. The most difficult challenge is to communicate with live data. In order to do this easily, the software tools must be integrated with the operating system's native networking technology and do this as transparently as possible. Networking tools should be designed to maximize throughput and should be stable and reliable if there are disruptions in the network. OPC is an industry standard interface through which software and hardware can communicate irrespective of the manufacturer. LabVIEW provides support for OPC through the DSC Module. Yet, OPC may not operate at the performance needed for higher speeds. For this requirement, the DSC Module also includes a built-in networking protocol for optimal throughput.

Security
Handling sensitive data often brings up security questions. Who should have access to the data and to which parts? Should everyone be able to modify the file or database? Probably not. Developing security code can be time consuming and cumbersome. By defining the system needs up front, engineers can choose tools to help them to do this. LabVIEW with the DSC Module enables engineers to define user profiles that limit access to different specific user interface controls on the application, as well as to different and sensitive sections of the data.

Distributed data logging systems have the most extensive lists of best practices:

• Reliable, efficient data storage
• Efficient tools for managing system-wide data
• Real-time and historical viewing of the entire system
• The ability to configure and log alarms and events
• Easy networking for different types of devices
• User interface security

Redundant Data Loggers

Use case: Logging data thermocouples on a critical process

Sometimes engineers need to develop redundancy into their data loggers. For example, a temperature data logger is monitoring a tank with a critical process. There are several things to consider for this application: logging data to a host computer, detecting a PC crash or network failure, logging data to another location, synchronizing the data after the crash or network failure has been resolved.

The basic logging requirements for this system are similar to a distributed data logging system. Operators and engineers should be able to monitor and control the system from multiple locations. The data must be stored to a secure location. Security may also be necessary.

A good hardware platform for this application is National Instruments Compact FieldPoint. Compact FieldPoint is a rugged, distributed I/O platform. Similar to a PLC, Compact FieldPoint has a controller and different I/O modules that plug into a solid back plane. For redundant data logging purposes, a controller with an onboard compact flash memory card can be used. Writing to compact flash memory routinely can be expensive because the drives only support a finite number of writes. However, they are perfect for adding redundancy to a system.

Compact FieldPoint has a built-in network watchdog. When the network is connected and Compact FieldPoint is communicating with a PC, the watchdog is disabled. However, when the network is disconnected or the PC crashes the watchdog will time out and become enabled. The program can be developed to take action when the watchdog is enabled, such as begin logging locally to the compact flash memory.

When the network connection is re-established, the DSC Module application will disable the watchdog. This causes the data to resume logging to the database. The DSC Module will then use FTP to retrieve the data files from the Compact FieldPoint and then write them to the database via Memory Tags.

By synchronizing the time servers on the PC and Compact FieldPoint, the data can be synchronized in the database. There will be two different traces in the same database.

The DSC Module application can also be configured to notify the operator or engineer of a network failure or when data files are being copied from the Compact FieldPoint.

In summary, there are several critical best practices to consider for redundant data loggers:

• Ability to synchronize logged data
• Real-time and historical viewing of the entire system
• The ability to configure and log alarms and events
• Easy networking for different types of devices

Remote/Portable Data Loggers

Use case: Logging data from remote thermocouples over time and checking with a PDA

When implementing a remote data logger, the requirements are similar to distributed data logging systems. NI CompactDAQ, CompactRIO, or Compact FieldPoint are the preferred platforms because of their size and ruggedness. On some platforms, LabVIEW also has the ability to embed a web server on the controller that engineers can access from any web browser to monitor the unit.

With the LabVIEW PDA Module or LabVIEW Touch Panel Module, a data logging application can be written on a desktop computer and then downloaded to a Palm or PocketPC, or an industrial HMI and touch panel interface. NI PCMCIA and CompactFlash solutions can be plugged into PDAs and used with the LabVIEW PDA module for a portable data logger. Engineers can also use common PCI, PXI, USB, or Ethernet measurement and control modules from NI and deploy portable data loggers to NI HMIs.

In summary, although the requirements for remote and portable data loggers are similar to distributed data loggers, there are a couple of unique best practices:

• Ability to monitor over web
• Remote applications need small, rugged platform such as Compact FieldPoint
• Portable applications need to small, flexible platforms, such as a PDA

Conclusion

When designing a data logging application it is important to address the five common requirements so that best practices can be utilized. These data logging best practices include straightforward data acquisition and logging, reliably viewing real-time and historical data, easily configuring alarms and events, instantly networking devices into a bigger system, and quickly implementing security. Once the particular requirements for a system have been identified, the key to implementing best practices is choosing the right software development tools. National Instruments offers LabVIEW tools that can help engineers design scalable monitoring applications easily and quickly. The LabVIEW Datalogging Supervisory Control Module addresses these concerns, so engineers can spend more time building their applications and less time worrying about programming the tools.

Engineers can put these best practices to use today and start realizing reliable, cost-effective data logging systems.

Additional Resources

Papers and Examples
A Review of PC-Based Data Logging and Recording Techniques

Data Logging Software Products
LabVIEW SignalExpress
LabVIEW
LabVIEW DSC Module

Other Mentioned Data Logging Products
LabVIEW Real-Time Module
LabVIEW PDA Module
DIAdem
Compact FieldPoint
PXI

Reader Comments | Submit a comment »