Redundancy in an Embedded FieldPoint Application

An FP-20xx network module can run embedded LabVIEW code developed using the LabVIEW Real-Time Module. A typical application is controlling an industrial process, which often requires a redundant system. A redundant system duplicates the hardware so that if the primary system fails, the secondary system can take over. Also, the secondary system can control the process while the primary system is offline for maintenance, upgrades, or changes. This document describes some of the requirements for creating a redundant system.

Communication Between the Primary and Secondary Systems

The two systems must share certain information in order to work together. For example, the primary system will send a heartbeat and possibly state information to the secondary system. The heartbeat is a Boolean pulse that lets the secondary system know that the primary system is working. State information can be passed to allow the secondary system to take over where the primary system left off.

There are three methods for passing the information: FP Write, Data Publish and Datasocket.

FP Write can be used to write to a channel on the primary system. The secondary system can then read the value of that channel using a Datasocket Read and the Lookout protocol. This can be very useful for the heartbeat. If the heartbeat is written to one of the LEDs on the FP-20xx, the secondary system can read the value, but you can also visually verify that the primary system is working. Similarly, if the state information is the last output, the secondary system can read it as output from the primary system.

Publish Data allows for creating new items on the FP-20xx that can be read by the secondary system. This would be appropriate for passing state information that is not an input or an output.

Datasocket Write can be used to publish data in a manner similar to Publish Data. However, it requires a computer running Datasocket Server. Datasocket Server cannot run on an FP-20xx. If the computer running Datasocket Server fails, so does the communication between the two FP-20xx modules.

See Also:
Introduction to Using DataSocket Commands to Communicate with FieldPoint Ethernet Network Modules

Heartbeat

The heartbeat is a signal that tells the backup system that the primary system is functioning. The primary system cannot send a signal to indicate that it has failed. Instead, the backup system monitors a constantly changing signal from the primary system. As long as the signal changes, the backup system knows that the primary system is working. If communication is lost or the signal doesn't change, that indicates that the primary system is not functioning. The heartbeat can be a Boolean variable that toggles between true and false or a numeric variable that increments.

The heartbeat can be in the main control loop or in a parallel loop. If the heartbeat is in the main control loop, the loop must iterate at a known speed. It is therefore appropriate for a polled loop.

[+] Enlarge Image
If the main control loop is event driven or its execution time varies, it is better to have a second parallel loop for the heartbeat.

[+] Enlarge Image

Passing State Information

When the secondary system takes over, it needs information about the process under control. State information must be sent continually so that the secondary system has up-to-date information when it takes over. How often the state information is sent depends on the type of system being controlled. Sending the state information more often gives the secondary system more recent information but uses more processing power on both systems.

The secondary system can read state information directly from the outputs of the primary system. For example, a digital output that is true might indicate that a motor is running. Because the secondary system can read the values of the channels on the primary system, it is not necessary to have an extra variable to indicate that the motor is on.


Publish Data.vi can publish other state information as Boolean, integer, double, and string data. The data becomes available as items on the FP-20xx.

Programming the Secondary System

In most cases, the secondary system should use the same VI as the primary system so that the two systems behave identically. Some minor modifications may be needed to allow the VI to start at a given state rather than with default values.

The secondary system also needs a VI to monitor the primary system heartbeat and state information. The heartbeat should be monitored with a Wait for Change and a timeout that exceeds the time period of the heartbeat. If the reading of the heartbeat times out, then there was no value change and the secondary process should be started. State information should be monitored without a Wait for Change so that it does not block heartbeat monitoring.

[+] Enlarge Image

Field Wiring

In some cases, completely redundant systems can be created. For example, consider a system that monitors the water level of a tank and activates a pump to refill the tank when the level is too low. A completely redundant system can be created with two FieldPoint banks, two level sensors, and two pumps. The two systems do not depend on each other except for determining which one is active.

In other situations, a completely redundant system is not feasible and only the FieldPoint hardware will be duplicated. In this case, the input and output signals need to be wired to both FieldPoint banks. This can be done using a double-throw relay controlled by the secondary system. Normally the signals are routed to the primary system. When the secondary system is activated, the relay is switched so that the signals are routed to the secondary system instead.

Conclusion

The complexity of the redundant system is limited only by programming. The simplest system will have a heartbeat that is used to launch the secondary system. A slightly more complex system can pass state information between the two systems. The concepts described in this document can be further enhanced to include a handshaking scheme so that the primary system can regain control once it is functioning again.

Reader Comments | Submit a comment »