Introduction to PCI Express

This paper gives a brief introduction to the PCI Express bus. In addition, it explains how National Instruments leads the way in the adoption of PCI Express for PC-based measurement and automation hardware and software. For a more informative paper on the PCI Express standard, please refer to PCI Express - An Overview of the PCI Express Standard.

Introduction

The rate of innovation in desktop computers is mind-boggling. Following Moore’s Law, processing speeds have doubled every eighteen months since the invention of the integrated circuit. Software makers create new products and versions to support the latest advances in processing speeds, memory size and hard disk capacity, while hardware vendors release new devices and technologies to keep up with the demands of the latest software. This rapid innovation is also evident with PC-based measurement hardware and software, with plug-in devices now providing 16-bit measurements at up to 15 MHz.

As data acquisition rates increase with advances in silicon technologies, larger amounts of data must be transferred to the PC for processing. These transfers are handled by the data bus connecting the device to PC memory. The bus is analogous to the transmission in a car – without it there is no way to get horsepower from the engine to the road. Like the transmission, the importance of the data bus is often overshadowed by the horsepower of the engine (processing and A/D rates). However, the rate at which data transfers occur is often the bottleneck in measurements, and is the primary reason that many instruments have incorporated expensive onboard memory.

To address the growing appetite for bandwidth, a new bus technology called PCI Express was recently introduced. Originally designed to enable high speed audio and video streaming, PCI Express is also being used to improve the data rate from measurement devices to PC memory by up to 30 times versus the traditional PCI bus used on desktops for the past ten years.

The PCI Express Standard

PCI Express was introduced to overcome the limitations of the original PCI bus. Developed and released by Intel over a decade ago, the original PCI bus operated at 33MHz and 32 bits with a peak theoretical bandwidth of 132MB per second. It used a shared bus topology – bus bandwidth is shared among multiple devices - to enable communication among the different devices on the bus. As devices evolved, new bandwidth hungry devices began starving other devices on the same shared bus. Gigabit LAN cards, for example, can monopolize up to 95% of available PCI bus bandwidth.

To provide the bandwidth required by these modern devices, PCI Express was developed by an industry consortium of PC and peripheral vendors and began shipping in standard desktop PCs in 2004. Already, most desktop machines from the leading suppliers include at least one PCI Express slot. The most notable PCI Express advancement over PCI is its point-to-point bus topology. The shared bus used for PCI is replaced with a shared switch, which provides each device its own direct access to the bus. And unlike PCI which divides bandwidth between all devices on the bus, PCI Express provides each device with its own dedicated data pipeline. Data is sent serially in packets through pairs of transmit and receive signals called lanes, which enable 250 MBytes/s bandwidth per direction, per lane. Multiple lanes can be grouped together into x1 (“by-one”), x2, x4, x8, x12, x16, and x32 lane widths to increase bandwidth to the slot.

Applications such as data acquisition and waveform generation require sufficient bandwidth to ensure that data can be transferred to memory fast enough without being lost or overwritten. PCI Express dramatically improves data bandwidth compared to legacy buses, minimizing the need for onboard memory and enabling faster data streaming. The initial signaling frequency provided by the specification of 2.5 Gbits/s provides 30 times (with a x16 slot) the usable bandwidth of 32 bit, 33 MHz PCI, and this signaling frequency is expected to increase with advances in silicon technology to 10 Gbits/s – the practical limit for signals in copper. And because of the scalable lane topology of PCI Express, data acquisition vendors can implement a PCI Express connector with the number of lanes suitable to the requirements of the device.

Hardware and Software Compatibilities

PCI Express maintains software compatibility with traditional PCI, but replaces the physical bus with a high-speed (2.5 Gb/s) serial bus. Because of this architecture change, the connectors themselves are not compatible. However, during the transition from PCI to PCI Express, most computer motherboards will provide a combination of PCI and PCI Express connectors. Devices with smaller connectors can be “up-plugged” into larger host connectors on the motherboard, improving hardware compatibility and flexibility. However, “down-plugging” into smaller sized connectors is not supported.
Software compatibility is also ensured by the PCI Express specification. The configuration space and programmability of PCI Express devices are unchanged from the traditional PCI methodology. In fact, all operating systems are able to boot without modification on a PCI Express architecture. At boot time, the operating system can discover all of the PCI Express devices present and then allocate system resources such as memory, I/O space, and interrupts to create an optimal system environment. And because the PCI Express physical layer is transparent to application software, programs originally written for PCI devices can run unchanged on PCI Express devices that have the same functionality, and PCI and PCI Express devices can be used together in the same system. This backward compatibility of PCI Express software with traditional PCI is critical in preserving the software investments of both vendors and users.

Choosing the Right PC to Host PCI Express Devices

Most PCs from the leading vendors now include at least one PCI Express slot. The most common slot sizes are x1 and x16. The x1 slot is a general-purpose slot that can host devices such as NI PCIe M Series data acquisition and NI PCIe GPIB devices. Currently, server-class machines are required for x4 and x8 slots which are used by devices including the NI PCIe Camera Link image acquisition device. “Server” does not imply high price, though, as tower servers are available for prices comparable to desktop machines. For example, as of May 2005, the Dell SC420 server has one x1 and one x8 PCI Express slot in addition to three PCI slots starting at $299 (USD).
It is important when choosing a computer to make sure that the PCI Express connectors are wired to the physical connection size. For instance, some vendors use motherboards with x8 connectors that are wired as x4. Devices in these slots will only operate at x4 data rates. In the case that you are up-plugging a PCI Express device, make sure that the computer you are using supports up-plugging at the maximum data rate supported by your device. The PCI Express specification only requires up-plugging to operate at the x1 data rate. This could result in a x4 device that is plugged into a x8 connector to operate at x1 data rates (250 MB/s).

National Instruments PCI Express Devices

National Instruments was the first to introduce multifunction data acquisition, digital multimeter, image acquisition, and GPIB devices for PCI Express. National Instruments also introduced the industry’s first PCI Express digital devices up to 200 MB/s dedicated throughput. The NI PCIe-GPIB is a GPIB controller with a x1 PCI Express connector. The NI PCIe-6537 and NI PCIe-6536 are 50 MHz and 25 MHz Digital I/O boards capable of continuously streaming data up to 200 MB/s and 100 MB/s, respectively, over the PCI Express bus. The NI PCIe-4065 is a 6½ Digit Digital Multimeter with ±300 VDC/Vrms of isolation, current measurements up to 3 A, and 2 or 4-wire resistance measurements. The NI PCIe-1429 is a x4 device for acquiring images with Camera Link cameras, and is the industry’s fastest plug-in image acquisition device with image streaming up to 680 MB/s (a CD-ROM worth of data every second!). The NI PCIe-6251 and NI PCIe-6259 are M Series data acquisition devices that provide up to 32 channels of 16-bit, 1.25 MS/s analog inputs in addition to 32 channels of 10 MHz digital I/O and four channels of 16-bit, 2.8 MS/s analog outputs. All of this I/O can run simultaneously while data is transferred to and from PC memory across a x1 connector.

Figure 5. PCI Express 6536/7 digital I/O boards provide high-speed data streaming up to 200 MB/s

All NI PCI Express devices are backwards-compatible with software designed for equivalent PCI devices. For example, software written in NI-DAQmx and LabVIEW for the NI PCI-6251 can run without modification on the NI PCIe-6251, ensuring that your software investment is preserved while your hardware investment is future-ready.

Related Links:
PCI Express – An Overview of the PCI Express Standard
High-Speed Stream-to-Disk Technology and Applications
Modular Instruments for PCI Express and PXI Express
PXI-Express: Providing New Benefits to the PXI Mixed Signal Test Platform

Reader Comments | Submit a comment »

does it fit small form factor?
ATI website "advisor" recommends a PCIe card for Dell Optiplex 620 SFF. How can I ensure such a card really will fit?
- Roger Noss. roger.noss@sbcglobal.net - Apr 23, 2006

Add OS trade-offs
Add the differences between OS implementations and though-put. (There's point have new HW if the OS does not support it.) Also, how dose this work a realtime OS?
- Aug 17, 2005