Multi-Process Engine (MPE) Reference Library

LabVIEW Real-Time (RT) was designed to take care of two specific engineering needs, real-time event response and closed loop control systems. Often there is a need to use a single RT controller for multiple real-time applications in parallel or in sequence. Operations that take place in parallel can be thought of as “processes”, while operation that occur serially which can be called “steps”. The ultimate goal is a simple method to coordinate the execution of code on the real-time target while maintaining real-time properties and provide simple user feedback. The following is an example of an execution mechanism referred to as a multi-process engine (MPE).

Download and unzip mpe_rc.zip to a familiar location.  Close LabVIEW if it is running.  In the MPE folder double-click setup.exe to begin the installation.  The MPE component files will be downloaded to the LabVIEW X.x\user.lib\MPE directory, and the example will be downloaded to LabVIEW X.x\example\MPE directory.

Feedback

This paper was created by the NI Systems Engineering group.  You can give us feedback by posting questions and comments through the Multi-Process Engine Discussion Forums.

Requirements

This example requires the following support software:

• LabVIEW 8.6 or Later
• LabVIEW Real-Time Module
• Simple TCP/IP Messaging (STM Component)

Table of Contents

1. Background
2. Theory of Operation
3. Implementation
4. Multi-Process Engine API
5. Conclusion

Background

Engineering applications often require software with a degree of determinism that standard operating systems cannot provide.   For these types of applications a real-time operating system becomes the solution that must be used to meet the overall goal of accurate tests and control. A predominate difficulty occurs, how to maintain the dynamic properties of standard operating systems without losing the determinism of a real-time application.

Almost all real-time applications use a host-target paradigm to accomplish a control system.  The system is made of a host program (a VI running on a general purpose operating system) and a target program (a VI running on a FieldPoint, cRIO, PXI real-time controller, or real-time desktop).  The user interacts with the host program which has a user friendly interface.  The host program interacts with the target program in such a way as to not affect determinism (set to a normal priority).  The target program takes care of the real-time operations (at a time-critical priority), while remaining in contact with the host program (at a lesser priority).  The host program and target program interact through a communication protocol (i.e. TCP, UDP, shared variables, etc.).  

For the purpose of this article a multi-process engine is defined as a component that coordinates the execution of code on real-time targets.  The execution can either take place in parallel (juggling control operations) or serially (like a sequence).  In this component, there are two parts to a multi-process engine, the server and the client.  The server typically resides on the target.  It is responsible for creating new processes, loading those processes with steps, aborting processes, and routing messages to/from processes.  The client simply receives and sends messages to the server.  It also performs basic operations depending on which messages it receives.  To make interaction with both sides of the Engine easier there is an Application Specific Interface (API).

[+] Enlarge Image

Figure 1.  Overview of the Multi-process Engine Component 

Theory of Operation

An effective MPE should be usable, maintainable, and scalable as well as include the following characteristics:

Characteristics for a MPE (Client)

•         User-Friendly

•         Event-based to reduce CPU use

•         Receive/Transmit data to the real-time sequencer (target program)

Characteristics for an MPE (Server)

•         Operate  at normal priority so that steps can use time-critical priority for determinism

•         Receive/Transmit data from processes in a deterministic manor using Process API

•         Receive/Transmit data from the MPE (Client) through a communication protocol (includes sending available steps from the target to the host and receiving process messages)

Characteristics for MPE API to be used in both Client and Server

•         Pass data to/from the target program in a deterministic manor

•         The MPE (Client) should interact with the host program through user events.

Implementation

Each portion of the code was written to achieve the characteristics described in the Theory of Operation.  Before the example can be opened properly in LabVIEW, you must follow the instructions from here.

TCP was elected as the communication protocol between the host and target because it is lossless and has low overhead.  To help with the maintainability and scalability of the TCP code a simple TCP/IP Messaging Protocol (STM) was used.  For further information please refer to the Developer Zone Article A Simple TCP/IP Messaging Protocol for LabVIEW.

In reference to fig. 1, we will begin by describing the MPE Client (Process Engine (Client).vi).  The MPE Client begins initializing inputs then creates user events that will allow it to communicate back to the host program.   After initialization, two parallel loops run.  The first loop handles incoming messages from the all of the real-time targets running the MPE Server, as well as keeping track of what targets the client is connected to.  The second loop operates on the messages received from the MPE Servers as well as messages from the Host API.  The operations can include but are not limited to reading configuration files, writing to log files, updating process status, and replying to the MPE Servers.

[+] Enlarge Image

Figure 2.  Multi-process Engine Client

In contrast to the MPE Client, the MPE Sever (Process Engine (Server).vi) operates by continually running four parallel loops.  The first loop handles the TCP connection to the MPE Server.  After a client connects to the server, the server replies with the Meta Data needed for the STM, available steps that reside on the target, and a list of the processes running on the target.  The second loop handles routing messages sent from the processes to the MPE Client.  The third loop handles messages sent from the client to the processes.  The final loop handles all the possible actions of the server, such as creating new processes, loading the processes with steps, and aborting processes.

[+] Enlarge Image

Figure 3.  Multi-process Engine Server

Processes are created by the MPE Server.  When a process is being created VI Server launches MPE_process.vi, and sets it to run asynchronously.  By doing that we are able to have multiple processes running in parallel.  Each process has four inputs, a process name, a steps queue for loading steps into the processes, and two RT FIFOs for deterministically pulling messages in and out of processes. 

Figure 4. A process

Steps are simply VIs that are loaded into the steps queue of a process.  When a process is idle, the process pulls the first step in its steps queue out of the queue, and dynamically runs it.  When that step is finished, the next step in the step queue is run.  In that way, we now can run VIs in parallel via added more processes, and serially by loading steps into a single process.  Steps are detected by the directory specified on the Process Engine (Server).vi, and must meet two requirements.  (1)  Located inside a .rtexe (a built real-time executable).  This ensures that all dependencies for the step are located on the target.  (2)  The step is set to a top-level startup VI (done inside the build specification).  This makes it so that it is easy to identify the appropriate steps that are valid. 

Multi-process Engine API

The MPE server (Process Engine (Server).vi) directs messages sent from the Process Engine (Client).  You must specify the port that the Process Engine (Client) will try to connect to the server, as well as the directory where valid steps are located.  If the server is located on a real-time target, valid steps must be located in a rtexe and set to the top-level VI.  This is to help prevent missing files from occurring on the real-time target.  If the server is located on a Windows machine, valid steps are any files with the .vi extension.  This is to help with debugging the Multi-process engine.

Starts the Process Engine (Client).  The Process Engine (Client) runs in the background.  Typically messages are sent to the Engine through the provided Client API.  Messages are received through the user events found in the Process Engine Events.vi. 

Engine INI File:  The file location the Process Engine uses to reply to INI requests from the Server.  Typically this is where the user would consolidate all of their various process settings.

Engine Log File:  The file location where processes can write text messages.  Typically this would be used as a location to consolidate a log of various events occurring inside of processes.

Restart?:  If the engine is currently running and the user chooses to restart it, it will abort the engine, restart the engine, and then reinitialize the engine.

Provides an Event Registration Refnum to all of the events that the Process Engine (Client) could generate.  The currently implemented events are:

Status:  A message sent from a process using the "Process Update Status.vi".  When this event triggers it returns the name of the process sending the status update as well as the status text.

Process Queue:  A message sent from a process automatically after it starts a new step.  When this event triggers it returns the name of the process and a list of the steps still inside of the process's queue.

Engine Errors:  A message sent from the Process Engine (Client).  When this event triggers it returns an error cluster from the Process Engine itself.  Both warnings and errors are captured.

Opens a new process on a controller at the valid IP address.  The port number must match with port number of the controllers Process Engine (Server).  If the process name already exists, the Process Engine (Client) generates a error event, and disregards the operation.

Loads a process with available steps.  If you specify a step that does not exist on the target, the MPE Client generates an error, and the operation is disregarded.

Aborts a process and all of the steps running inside that process.  If the user attempts to abort a process that the MPE Client cannot find, the MPE Client generates an error, and the operation is ignored.

Stops the Process Engine (Client) from running.

Queries the Process Engine (Client) for an INI key.  The key is determined by the Key Info (similar to the Configuration File VIs).  If the Process Engine (Client) does not reply in a given time the Default is used.  If information is returned that is NOT related to the request INI key, this VI returns "invalid". 

Sends a Status message to the Processes Engine (Client).  Nothing occurs is the Process Engine Server and Client are disconnected.

Sends text to the Process Engine (Client) to be written to a log file.  If the Process Engine (Client) does not reply in a given time the log text is written to a local file.  If information is returned that is NOT related to the request log text, this VI writes the text to a local file.

Reads the raw string coming from the incoming fifo.  Typically this VI is used to expand the Process API.

Writes a string to Outgoing fifo.  Typically this is used to expand the Process API.

Conclusion

A multi-process engine can increase the productivity of the real-time targets in most deterministic applications.  The demonstrated component provides for a scalable and maintainable solution without affecting the determinism of the system significantly.  Communication between the user and the real-time target is handled entirely by the MPE Client and Server, allowing developers to focus on individual operations.

Running the Example on localhost

The following step-by-step instructions demonstrate how to run the MPE host and server locally on your development machine.  Although this is not the intended use of the MPE, it helps in understanding what the MPE is used for, as well as how to expand the MPE functionality.  

1. Open the example project by going to LabVIEW X.x\example\MPE directory, and double-clicking "MPE example.lvproj".  A project like fig. 1 should should open.  There are two folders under My Computer, Target and Host.  The target folder contains all the VIs that will be running under your remote real-time target, while the host folder contains all the files that the user will interact with.  For demonstration purposes both folders are under My Computer making all VIs will run locally on My Computer.  
2. Expand the Target folder and double click "MPE_server_ex.vi".  
3. Run "MPE_server_ex.vi".  
4. Expand the Host folder, and double-click MPE_client_ex.vi.  
5. Run MPE_client_ex.vi.  You should see the Process Engine (Client).vi (the MPE client) appear automatically and already be running. 
6. On the front-panel of "MPE_client_ex.vi", click the "Batch 1" button.  This will open a process called batch_1 and load steps into that process.  Each step should open, run, and after being completed close.
7. Experiment with "Batch 1", "Batch 2", and "Batch 3".

Figure 1.  MPE example project.

To disable front panels from dynamically opening and closing, change the conditional disable symbol DEBUG from TRUE to FALSE.  That will mimic more of what an operation with an actual real-time target will look like.

Running the Example on a Real-Time Target

The following step-by-step instructions demonstrate how to run the MPE host on the local machine, and the MPE server on the real-time target.

1. Open the example project located in the LabVIEW X.x\example\MPE directory, and double-clicking "MPE example.lvproj".
2. With all VIs closed, take the Target folder and drag it under RT Target in the project.
3. Change the IP address of RT Target from 0.0.0.0 to the IP address of your actual real-time target located on the network.
4. Under RT Target, there is a build specification called "All Steps".  Right-click "All Steps" and choose properties.
5. Click the Source Files Category.
6. Browse to the Steps subfolder inside Target folder.
7. Add all the VIs in the Steps folder to the Startup VIs category.  We are not setting these VIs to run at startup, but instead this sets an internal flag (top-level) to each VI that we can exploit to find the VIs easier.
8. Click the Build button.
9. Right-Click "All Steps" and choose Deploy.  You may receive a warning about not setting the executable to start-up, this is expected.  All the Steps have now been deployed with their dependencies.
10. Run "MPE_server_ex.vi" and "MPE_client_ex.vi".  
11. Experiment by clicking on the "Batch 1", "Batch 2", and "Batch 3" buttons.

 

Requirements

Filename: mpe_rc.zip

Software Requirements

Application Software: LabVIEW Professional Development System 8.6
Toolkits and Add-Ons: LabVIEW Real-Time Module 8.6
Language(s): LabVIEW

Hardware Requirements

Hardware Group: Compact FieldPoint, CompactRIO, PXI/CompactPCI

Reader Comments | Submit a comment »