Streaming Data to Disk from the NI 6534

The NI 6534 is a high-speed digital I/O device with two independent timing controllers for high-speed data transfer. Each timing controller group consists of four control lines which you can use to time the data I/O with hardware precision. In addition, each group has access to 32 MB of onboard memory. The NI 6534 uses the DAQ-DIO ASIC designed by National Instruments. An onboard 20 MHz oscillator generates the timebase on the NI 6534, and data can be transferred at a maximum rate of 20 MS/s, with samples of one, two, or four bytes. For more information about the NI 6534, refer to the 653x User Manual.

National Instruments has conducted benchmark tests to measure the maximum rate at which data can be streamed to disk (Continuous Buffered Pattern Input) or output from the disk to the ports of the NI 6534 (Continuous Buffered Pattern Output). These benchmark results are system dependent and are not be considered specifications. The LabVIEW VIs used for these tests are attached in the Benchmark Examples section of this document. For a description of the system used for these tests, refer to the Benchmark System Specifications section below.

The following sections explain some of the system dependencies that affect your total streaming rate.

Hard Drives and Transfer Rates

The data transfer rate to and from the hard drive affects the overall transfer rate of your DIO operations, so it is important to understand how data is transferred to and from hard disks and what some common hard drive specifications mean. The hard drive interface has a buffer where data coming from the PC bus is stored. The rate of data transfer from the PC memory to the drive interface buffer is the external transfer rate. From the drive interface buffer, data is transferred through the drive interface to read/write heads and written to the disk cylinders. The rate at which data is transferred from the drive interface buffer to the disk cylinders is the internal transfer rate. The external transfer is entirely electronic, and although it is system dependent, it is always much higher than the internal transfer rate, which depends not only on the speed of the hard disk interface and controller, but on the time it takes to switch between read/write heads and the time needed to seek to the correct cylinder. The internal transfer rate is the bottleneck for overall data transfer rates.

A common misconception is that a hard drive with a higher rotational speed -- the speed the disk spins in revolutions per minute (RPM) -- can stream data faster than a drive with a lower rotational speed. The maximum data transfer rate depends not only on rotational speed, but on other factors like platter size and track density. A disk with larger platters can stream data faster for the same rotational speed and track density. Also, the number of sectors in a track will affect the number of bytes written/read per cylinder revolution. Another factor to consider is that data passes under the disk head at a higher rate on the outer diameter track than on the inner diameter track, resulting in higher transfer rates from the outer track. In fact, the larger outer cylinders have higher sustained transfer rates (STR) than the smaller inner cylinders. Drive specs usually specify the outer and inner transfer rates. The STR spec for a drive is usually an average of the faster outer cylinder rate and the slower inner cylinder rate.

Two commonly used hard drive types are Integrated Drive Electronics (IDE) and Small Computer System Interface (SCSI). These names refer to the type of interface used to transfer data to and from PC memory. While the IDE interface is limited to two hard drives (master and slave), several drives can be connected to the SCSI bus. It is difficult to give a clear-cut comparison between IDE and SCSI drives; you can find many papers comparing IDE and SCSI performance on the Web. In single-drive systems, performance is very similar between IDE and SCSI drives. For multiple drive systems such as servers, SCSI drives are more flexible and have more bandwidth than IDE drives.

For these benchmarks, we used Seagate ST318404LW and ST39103LW SCSI drives, with average transfer rates to disk media of 33 MB/s and 29.5 MB/s, respectively. In both cases the data transfer rate depends on the interface speed. A faster processor is able to read and write data to a hard drive buffer faster than a slower processor, but it is the internal transfer rate from the hard drive buffer to the disk surface that affects the transfer rate the most. Manufacturers of hard drives usually specify burst rates of 100 MB/s or higher. Note that this is the data transfer rate from PC memory to the drive interface, not the rate of saving and retrieving data from the disk surface using the drive bus.

Acquiring Digital Patterns and Streaming to Disk

Pattern I/O applications acquire or generate data patterns on every rising or falling edge of a clock signal. With an NI 6534, you can acquire or generate patterns that are 8, 16, or 32 bits wide. Refer to the 653x User Manual for more information about pattern I/O.

Let’s take a look at the following simplified diagram of the flow of data in a pattern acquisition application using the NI 6534.

First, data is acquired into the NI 6534 onboard memory where it is held until it is transferred over the PCI bus to PC memory (RAM). The NI 6534 hardware is capable of transferring data at 20 MS/s. For 32-bit (4 byte) samples, 20 MS/s is 80 MB/s. The theoretical maximum burst transfer rate for the PCI bus is 132 MB/s, with most systems capable of sustained transfer rates up to 120 MB/s. Even though the 80 MB/s maximum rate for the NI 6534 is only 2/3 of the sustainable PCI bus transfer rate, the NI 6534 may not be able to sustain the maximum rate due to the use of PCI bus bandwidth by other devices.

As data is acquired by the NI 6534, the PCI bus bursts data from the onboard memory to the PC memory while the NI 6534 continues to acquire data to the onboard memory. Data is then placed in the PC buffer, then copied to the application buffer.

Now the data is transferred from the application buffer to the IDE or SCSI interface buffer, again over the PCI bus. From the interface buffer, data is written to the disk over the IDE or SCSI bus. At this point the transfer rate is completely dependent on the drive used and how fast its controller can write data to the disk.

All of the factors discussed so far affect the maximum transfer rate that the system can maintain, with the system bus transfer rate and the drive interface rate the main factors involved. Your data transfer rate cannot be higher than the lower of the bus transfer and drive interface rates. Because there are no mechanical operations involved, the transfer rate from PC memory to the hard drive is usually much faster than the internal transfer rate of the drive, so the overall data transfer rate is limited by the internal transfer rate of the drive you use.

Errors
If the average burst rate of the PCI bus is less than the input transfer rate of the NI 6534, NI-DAQ stops the input operation and returns an onboard FIFO overflow error (-10845). Refer to Maximizing the Performance of the NI 6534 Digital I/O Device for more information about possible pattern I/O errors and how to avoid them.

Pattern Acquisition Benchmark Results

The following tables contain the pattern acquisition benchmark test results.

The IDE drive gave the lowest data transfer clock rate in our tests (1.05 MHz). The Seagate ST39103LW SCSI drive was able to sustain a clock rate of 2 MHz and the ST318404LW sustained a 2.5 MHz rate. The results, as expected, reflected a better transfer rate for a larger buffer (10 MB) and a faster drive; a larger buffer can hold greater amounts of data before they are transferred to the hard drive, and a faster connection allows more data to be transferred in a given time. The highest rate attained was 3.33 MHz using a Seagate ST318404 SCSI drive. For more information about these SCSI drives, please visit www.seagate.com.

Streaming Digital Patterns from Disk to the External Device

The following simplified diagram illustrates the data flow for a pattern output operation using the NI 6534.


Data for output by the NI 6534 takes a path that is the reverse of that for data acquired by the NI 6534. First, data is extracted from the disk through the IDE or SCSI bus, then transferred over the PCI bus to the application buffer where it is copied to the PC memory. Data is then transferred from PC memory to the onboard memory, again over the PCI bus. Once data is in the NI 6534 buffer, it is output to an external device. The output rate is controlled by a hardware clock. While data is output from the NI 6534 onboard memory, new data is burst from the PCI bus to the onboard memory.

Errors
If data is not transferred from the PC memory to the NI 6534 onboard memory quickly enough, the data on the PC buffer can be overwritten before it is transferred to the NI 6534. If this happens, NI-DAQ generates a transferStoppedError (-10803) and stops the operation. If the average burst rate of the PCI bus is less than the output transfer rate of the NI 6534 device, NI-DAQ stops the output operation and return an onboard FIFO underflow (-10843) error to prevent data from being regenerated. Refer to Maximizing the Performance of the NI 6534 Digital I/O Device for more information about possible pattern I/O errors and how to avoid them.

Pattern Generation Benchmark Results

The following table contains the Continuous Pattern Output benchmark results.

Similar to the pattern input benchmarks, for pattern output you obtain faster clock rates by using faster hard drives. Testing was repeated to find a good compromise between the buffer size and the number of updates (the number of data samples transferred to the PC memory from the application buffer every iteration). The fastest output clock rate, 5 MHz, was obtained using the ST318404LW SCSI drive, a buffer size of 100 KB, and 50 KB updates. These results are completely system dependent and will vary from system to system. It is clear, however, that data transfer rates are governed by the speed of the system and how quickly data can be moved to the PC buffer from the hard drive.

Benchmark Examples

The LabVIEW VIs below are benchmark examples; one for data input streaming to disk (Pattern Input to Binary File (cont) Benchmark.vi) and another for data output streaming from disk (Pattern Output from Binary File (cont) Benchmark.vi). Both VIs start at the maximum clock rate. If error -10845 (-10843 for output) or -10803 is generated, then VI stops data I/O, increments the clock divisor to decrease the clock rate, and restarts data I/O. The process is continued until a specified amount of data is transferred without transfer errors. This clock rate is taken as the maximum transfer rate for your system.

 

Benchmark System Specifications

The system used for these benchmarks has the following characteristics:

• NI PXI-8170 controller (X86 family 6 model 8 stepping)
  • 256 MB RAM
  • 700 MHz processor
• NI PXI-6534
• NI PXI-8210, a Fast Ethernet/Ultra Wide SCSI interface for the PXI bus
• IDE drive
• SCSI drives: Seagate ST39103 and ST318404

  Conclusion
  

From the benchmark test results, we can make the following suggestions for increasing the data I/O rate with your NI 6534 pattern I/O application.:

1. Use a faster hard drive to help move data to and from PC memory fast enough to avoid overwrite errors. The sustained clock rate was higher for the SCSI drives than the IDE drive, and the faster SCSI drive sustained the highest clock rate, both in pattern acquisition and pattern generation operations.
2. Experiment with different combinations of buffer size and the size of updates to optimize the transfer rate. It is usually better to transfer larger amounts of data per iteration, but results vary.
3. Use an efficient program. For example, read or write binary data files. It takes longer to read or write other file types.
4. Use the fastest available system processor to move data in and out of PC memory as quickly as possible.

Related Links:
653X User Manual
Maximizing the Performance of the NI 6534 Digital I/O Device

Reader Comments | Submit a comment »