Implementing Neural Networks with LabVIEW - An Introduction

Why use Neural Networks?

Only a small number of the problems we encounter on a daily basis can be solved by an objective set of steps. The vast majority of solutions to problems are not only subjective and based on individual experience, but also on circumstances surrounding a problem. A good example of this kind of problem is a Route Optimization Problem.

In a Route Optimization Problem, software-programming techniques, such as Neural Networks, can be used to enable a computer system to determine the fastest and most accurate solution when finding the optimal route from point A to Point B when many paths are offered. For example, a driver in route to a destination may access a GPS. With the help of Neural Networks, the system would not only be able to give directions, but also be able to determine the fastest route based on such mitigating factors as weather and traffic.

What is a Neural Network?

A Neural Network is a way of allowing a system to learn based on previous experience. It is a new and different paradigm for computing. Rather than basing applications on the traditional algorithmic approach to solving a problem through a state machine, the system will choose a set of steps to reach the solution based previous experience and learning. In a Neural Network, instead of trying to simulate a thought process, applications will simulate the architecture of the human brain, a model that runs on a network of neurons that interact with each other to produce optimum results.

How are they implemented in LabVIEW?

Neural Networks is based on the actions of a neuron within the brain. An artificial neuron is a piece of software able to simulate the 4 main tasks that a biological neuron performs:

Each individual neuron within a network will take care of receiving messages from other neurons through its Synapses and in turn process those messages within its Soma so that an output can be sent to one or more neurons afterwards. Through these interactions a final solution to a problem can be generated.

For the purpose of creating a Neural Network in LabVIEW, a model for the neuron first needs to be defined. The typical model for implementing a neuron is shown in figure 2:

LabVIEW weights each of the inputs Xn is by multiplying each by its respective weight value Wn. A sum of all these results is then calculated and to this a transfer function is applied that generates the neuron’s outputs.

The LabVIEW diagram for implementing a neuron would look something like this:


Depending on the type of application being developed, different transfer functions might be chosen to achieve a specific result.

Once all individual neurons have been developed you can implement the network itself. For the purpose of this document we are going to take a look at a basic network example, also known as a perceptron:


This network consists of 1 neuron that has 2 inputs and one output. All are binary. The transfer function is:
Y= 1 if W0 *I0 + W1 * I1 + Wb > 0
Y= 0 if W0 *I0 + W1 * I1 + Wb <= 0

In this case, we want the Network to learn simple OR: output a 1 if either I0 or I1 is 1.
The equation for this kind of network is called the perceptron learning rule or delta rule:
&Delta Wi = &eta * (D-Y).Ii
where &eta is the learning rate, D is the desired output, and Y is the actual output.


It is important to note that a Neural Network system is meant to be used with problems where an algorithmic solution can’t be formulated, but where examples of the behavior it will perform to have are available. Since this is the case for the OR function where there is a table of results for all its possible inputs, the network can go into the training or learning phase.


In the training phase the network is run several times, and will adapt itself by changing the weight of the inputs on each run in an amount proportionate to the difference between the desired output and the output of the current run. This process is repeated until the difference is zero for all patterns entered, meaning that the optimum weight for all inputs of the neurons in the net has been achieved.

Since this simple example only has two inputs, the decision surface that the network outputs is a line defined by the equation:

I1 = (W0/W1).I0 + (Wb/W1)

This example shows that the number of possible decisions the simple perceptron network can make is very limited.

However, based on combinations of more of this simple perceptrons, a network can implement more complex architectures when a greater number of decisions need to be made. Some examples of common architectures are the following:


Back-Propagated Delta Rule Networks (BP):
An extension of the simple perceptron, this network adds extra hidden layers between inputs and outputs. Each layer will built using a certain number of neurons depending the application and the number of inputs and outputs needed in the net like in the example below:

This kind of network is particularly useful when we need to use machine learning for more complex pattern detection. It has the constraint that it has to work in a feed-forward fashion. This means that all inputs are received in the input layer, and the neurons in this layer will send their outputs into the first hidden layer or any consecutive layer and so forth. Once all the information has been processed in the intermediate levels, the output layer cells will yield the final result. This is a very powerful architecture since any mapping can be learned given two hidden layers.
Training this kind of networks requires a small variation in the weight change rule. For a BP network, weights are changed by an amount proportionate to the error in that neuron times the output of the neuron feeding into the weight. This means that the error will be tracked not only on each iteration, but also on each individual neuron within the network, making the decision much more accurate.

An example of this type of network in LabVIEW can be found at the end of this section

A specialization of these type of networks is the Radial Basis Function Networks, which can determine the number of neurons within a hidden layer based on a Gaussian probability function.

Unsupervised Nets: While simple perceptrons and BP networks require a teacher to tell the network what the desired output should be, networks can be implemented in such a way that they adapt solely in response to the inputs. These are very useful with applications where it is necessary to pick out structure from the input and generate outputs based on them. Data clustering and compression algorithms are typical uses of this type of architecture.

Neural Network Application Examples:

See Also:
Neural Network Learner

Neural network applications and future work.

The use of machine learning techniques and of neural networks has become quite widespread in the last few years. Some of these applications include process control (e.g. LabVIEW PID Toolset), Fuzzy Logic, Process Monitoring (Aircraft Engine State), Data Compression, Speech, Acoustics and Vision Recognition Algorithms. For more information on these, check the related links section.

See Also:
PID Control Toolset for LabVIEW

Related Links:
The IEEE Neural Networks Council
A document about the use of fuzzy logic systems and neural networks
An excellent tutorial on neural nets.

Reader Comments | Submit a comment »