Parsing Instrument Data in Visual Basic

Instruments are notorious for sending back cryptic binary messages or strings containing a combination of header and data information. It is often frustrating trying to make your application work with these strings, and it can be a long and painful experience trying to parse them into a usable data format. Parsing divides data into small, usable components that you can visualize, analyze, and save. The Measurement Studio Serial, GPIB, and VISA ActiveX controls provide a built-in data parsing tool to make working with instrument data easier. You can use this interactive string parsing tool to define rules for parsing information out of instrument strings so you keep only the data you need.

Note: Measurement Studio includes tools to build measurement applications in ANSI C, Visual C++, and Visual Basic. The Measurement Studio Visual Basic tools were formerly known as ComponentWorks.

Measurement Studio Parsing Overview

Use the Task, Pattern, and Token objects to define syntax rules for parsing your instrument data. Figure 1 shows the relationship between these three objects. A Task object holds information about the format of the data through a collection of patterns. You might define several tasks per control to deal with different data formats or more complex instrument strings. A Pattern object contains a collection of tokens that describe the format of the data. You can create custom patterns and choose to repeat those patterns more than once to parse a repetitive instrumentstring or you can use the built-in Number Parser pattern to extract numbers. Patterns are made up of one or more tokens, which are the basic building blocks for parsing. A Token is a single string, character, number, or binary format. You combine tokens to define a pattern in the data string.
Consider the following string returned from an instrument:

WFM 3.246,3.295,4.653,3.334

This string contains header information (information about the data that follows) and data. The header information WMF indicates that the data following the header is waveform data. The waveform data consists of comma-delimited float values. You can define this data format in terms of patterns and tokens so that no matter what data this instrument returns, you can always extract an array of float values. The following list identifies the tokens in the example string:

• "WMF" token -- Header information about the data that follows it
• <number> token -- A single data point expressed with ACSII characters
• "," token -- A comma to separate number tokens

This example has two patterns. The first is the header information, and the pattern consists of one token:

"WMF"

The second pattern is repeated several times and consists of the following tokens:

<number> + ","

Tip: Notice that the last value deviates from the pattern because it is not followed by the "," token. As you develop parsing tasks, consider how you are going to deal with header information and the last value in a data string.

You might use the following algorithm to create a parsing task for this type of string:

1. Find the header information, but don’t add it to the returned data.

2. Find the first <number> + "," pattern.

3. Extract the number and ignore the comma.

4. Repeat the <number> + "," pattern until the last value.

5. Add the last value to the array of numbers returned to your program.

Note: The ActiveX controls return parsed data as an array of tokens. The array can contain a combination of data types (for example, strings and numbers). For this example, an array of doubles is returned to your program.

Parsing Patterns

You can use two types of patterns -- a Number Parser pattern and a user-defined pattern. The built-in Number Parser pattern converts a string -- comma-delimited or other-delimited -- into an array of numbers, stripping out header information and other nonnumeric information. Data strings like the example you just examined can take advantage of the Number Parser pattern.

With user-defined patterns, you can specify the format of the data through a set of tokens. Each token defines a particular element in the pattern. Tokens can represent string or binary data. You can use string tokens to parse data that is represented by a string, and binary tokens to parse binary data. You can even combine string and binary tokens in the same pattern.

You can use one or more of these patterns in a single parsing task. When you run a task, you can execute each pattern once, a fixed number of times, or an unspecified number of times. If you run the task an unspecified number of times, the pattern is executed until it cannot find a match, at which time it continues to the next pattern or token.

Parsing Tokens

There are two types of parsing tokens -- string and binary. String tokens parse any string or number that you can express with ASCII characters. For example, "WMF" and "1.23" are examples of string tokens. Binary tokens parse data stored as bytes. During a parsing task, the ActiveX controls use string tokens to identify numbers, special strings (such as the "WMF" token), or a string of a certain length. Binary tokens interpret the data in the buffer not as a string but as raw binary data. Binary tokens specify the number of bytes to read from the buffer and return in the format of the specified token.

After the task executes, each parsed token is returned unless you specifically mark it to be ignored. If the specified token is not found, the parsing operation fails. If the parsing operation fails, the data that was found is returned and the OnError event is fired.

Tip: When you request that a token be ignored, the control does not return any occurrence of it in the array. For example, you might not want the commas from the <number> + "," pattern returned in the array. If you specify commas as ignored, only the numbers are returned in the array. If only one number is present, the control returns the number as a scalar (not an array).

The following tables list the string and binary tokens that the ActiveX controls define:


¹ Do not use unsigned integer data types in Visual Basic because it does not recognize them.

Example -- Defining a Parsing Task

Open Visual Basic and load a Measurement Studio Instrument (CWSerial or CWGPIB) or VISA (CWVisa) control, depending on the type of instrument with which you want to communicate. Place one of the controls on the form, right click on it, and select Properties. Configure instrument communication on the General page, and then select the Parsing property page to interactively define the parsing task.

When you open the Parsing page, you’ll notice that there is a Number Parser parsing task already defined. Number Parser is the built-in parsing pattern that parses delimited strings. But what if you want to parse the following string and return both numeric and string data?

@@@002ACK;

This particular instrument returns strings with three @ signs, a number, a three-character string, and a single termination character. Imagine that you want to parse a string with this format and return only the number and three-character string. In this example, you’ll define a new pattern named ACK through a series of user-defined tokens to parse this string and display the number and three-character string in text boxes on the user interface.

1. Create a new task. Click Edit and change its name to ACK.

2. Add a new pattern to the ACK task.

3. Now add the first token that describes the string. Click Token to add a new token, and type @@@ for Token value. Because you don’t want this string returned, select Ignore token.

Note: To define your own string or character token, such as the "@@@" token, select any of the pattern types from the token value list and then enter the string that you want to define as a token. You can specify nonprintable ASCII characters as tokens too -- just use the equivalent decimal number, which you can find in an ASCII chart. Notice that the property pages include the decimal numbers for two frequently used string tokens -- the carriage return (\13) and the line feed (\10).

4. Add the second token to the pattern. Click Token, and select <number> from the Token value list.

5. Add the third token. Click Token, and select <string3>. The <string3> token returns any string with three characters. If the instrument always returns ACK as the three-character string, then you can create an "ACK" token rather than using the <string3> token.

6. Add the last token. You can use the <string1> token to parse the termination character. You don’t want this token returned, so check Ignore token.

Your Parsing property page should look like the one in Figure 2.


7. Finally, you need to call the parsing task from your code. The following event procedure is executed when you click on a Parse String button on the user interface. The procedure sets the instrument string in data, calls the ACK parsing task with the Parse method, and then displays the returned data in two text boxes on the user interface, as shown in Figure 3.

Option Explicit
Dim Response As Variant
Dim data As String
Private Sub btnParseString_Click()
data = "@@@002ACK;"
Response = Comm.Tasks("ACK").Parse(data)
Text1.Text = Response(0)
Text2.Text = Response(1)
End Sub

Note: You’ll need to add two text boxes and a command button named btnParseString to your user interface to complete this example.


Open the Measurement Studio for Visual Basic Advanced Parsing example and run it.

Note If you installed Measurement Studio on your computer, you can open this example from MeasurementStudio\Vb\Samples\Instr\Advanced Parsing. Otherwise, you can download this example, Parsing Instrument Data in Visual Basic, at ni.com.

Press Generate Simulated Data (Ascii). Notice that the text box is populated with a string of data. The string begins with #, a number, and a colon. The first number indicates the number of data points included in the string. The simulated ASCII string follows the following format:

#n: val1, val2, val3, ..., valn

where n is the number of values included in the string. Press the Parse and Plot Data button and watch the program parse the instrument string and plot the parsed data on the graph.

Now try pressing the Generate Simulated Data (2 byte Binary) button. Notice that the format of the string is a little different than that of the ASCII string. This 2-byte binary data can be expressed by the following format:

n val1 val2 val3 ... valn

where n is the number of data points in the string. Both strings return header information (although in a slightly different format) that indicate how many data points are contained in the string. You can use this header information to determine the number of times you should repeat a pattern.

Note Check your instrument manual for information about the way your instruments returns data.

Open the Parsing property page and examine it to determine how the CWSerial control uses the parsing tasks, patterns, and tokens. There are three tasks: AsciiNumber, AsciiList, and BinaryData, as shown in Figure 4. The first two tasks -- AsciiNumber and AsciiList -- parse the data that is generated with the Generate Simulated Data (Ascii) button. The BinaryData task parses the binary data that is generated with the Generate Simulated Data (2 byte Binary) button.


Parsing ASCII Data

The AsciiNumber task parses the header information from the ASCII data. AsciiNumber consists of one pattern with three tokens, as shown in Figure 4. The "#" and ":" tokens are marked as ignored, and the <number> token specifies the number of values that follow. You can use this value to determine the number of times you should repeat the AsciiList task.

The AsciiList task reads the elements in the buffer. As shown in Figure 5, it consists of two patterns: List and LastNumber. The List pattern is defined by two tokens (<number> and ","), and it reads all elements in the list except the last element because the last element does not follow the pattern -- it doesn’t contain a comma. The "," token is ignored so that it is not returned in the array. The LastNumber pattern is defined by one token (<number>), and it reads the last element in the buffer.


The following code from the example reads the data, parses it, and plots it on the graph.

Tip: The trick to parsing strings with a varying number of data points is determining the number of times to repeat the pattern. You have to specify the number of times to repeat the pattern programmatically during run time with the RepetitionFactor property.

'Parse the ASCII data and graph it.
Private Sub ParseAsciiData()

'NumberOfElements is the first number in the string.
'Repeat the AsciiList task NumberOfElements-1 to parse all but the last value.
Dim NumberOfElements

'parsedData is the array of parsed values.
Dim parsedData

'Read the first number to find out how many times to repeat AsciiList.
NumberOfElements = CWSerial1.Tasks("AsciiNumber").Parse(AsciiText.Text)

'Set RepetitionFactor to read all but the last value.
CWSerial1.Tasks("AsciiList").Patterns("List").RepetitionFactor = NumberOfElements - 1

'Read values into the parsedData array.
parsedData = CWSerial1.Tasks("AsciiList").Parse(AsciiText.Text)

'Plot parsedData.
CWGraph1.PlotY parsedData

End Sub

Notice that the string is actually parsed twice. The first time you use the AsciiNumber task to look for header information to determine the number of times to repeat the AsciiList task. The second time you run the AsciiList task to find the List and LastNumber patterns and extract the numbers and ignore the commas. The AsciiList task ignores the header information because it is looking for the first occurrence of the <number> + "," pattern.

Parsing Binary Data

The BinaryData task parses data in binary format. As shown in Figure 6, it consists of two patterns -- Number and Data. Each pattern consists of a single token, <int2>. The Number pattern reads the header information that indicates the number of items in the binary data. Use this value just like it was used to parse ASCII data -- to determine at run time the number of times the Data pattern needs to be executed.

The following code from the example shows you how to read the data, call the BinaryData task, and plot the parsed data on the graph. The subroutine parses the data twice. The first time, the task parses the data and returns only the first value, which it uses to set the repetition factor. The second time, the task ignores the first value and returns just the data.

'Parses the binary data and graphs it.
Private Sub ParseBinaryData()

'NumberOfElements is the first number in the string.
Dim NumberOfElements

'parsedData will be the array of parsed values.
Dim parsedData

'Create a reference to the BinaryData task created in the property pages.
Dim Task As CWTask
Set Task = CWSerial1.Tasks("BinaryData")

'Read the first two-byte number using the Number pattern so that you can
'determine the number of times to repeat the Data pattern.
Task.Patterns("Number").Tokens(1).Ignore = False

'Don’t execute the Data pattern yet.
Task.Patterns("Data").RepetitionFactor = 0

'Save the data from the BinaryData parsing task, which is just one value.
'Now you know the number of times to repeat the Data pattern.
NumberOfElements = Task.Parse(BinaryData)

'Now ignore the first two-byte number and specify the number of times to
'repeat the Data pattern to extract all 2-byte numbers.
Task.Patterns("Number").Tokens(1).Ignore = True
Task.Patterns("Data").RepetitionFactor = NumberOfElements

'Return the values from the BinaryData parsing task in the parsedData array.
parsedData = Task.Parse(BinaryData)

'Plot parsedData.
CWGraph1.PlotY parsedData

End Sub

Conclusion

The Measurement Studio interactive parsing tool makes it easier to use the data returned from your instrument. The parsing tool is flexible enough to let you define your own parsing tasks, patterns, and tokens. The Measurement Studio parsing API is simple enough for you to use so that you can enable your program to make parsing decisions during run time, such as determining the number of times to repeat a certain task. Remember to consult your instrument manual for information about the data format that your instrument uses.

Reader Comments | Submit a comment »