Using Edge Detection in Machine Vision Gauging Applications

This document introduces common edge-detection software strategies for applications such as inspection for missing parts, measurement of critical part dimensions using gauging, and identification and verification of electronic user interface displays.

What You Will Learn

• Nine steps for machine vision success
• About common inspection applications
• How to use gauging techniques to measure tolerances
• How to inspect for missing components
• How to create an alignment application
• About identification systems for LCDs, meters, and bar codes

  Nine Steps for Machine Vision Success
  

Clearly understanding what is and what is not a defect in your application as well as integrating a calibration phase into your machine vision system are critical steps for the success of your system. To help you in your machine vision system design, the step-by-step guide below reviews the important considerations when creating a machine vision system. The steps are designed to give you an overview of the issues to consider when developing a system.

1. Identify all defects – Clearly understand what is a good and bad part. Rank defects based on the frequency of their occurrence to quantify what is a bad part.
2. Calculate the FOV – Select the camera and lens to inspect the smallest defect that may occur. Can a human inspector see the defect? If so, typically 8-bit analog cameras will work. If not, digital cameras may be necessary. For applications where the part is moving select the appropriate camera so that the image is not blurred.
3. Lighting – Select a lighting technique that gives the maximum contrast for the defects and features of interest. Experiment with directional light, back lighting, ring lights, and polarizing lenses. Which lighting technique highlights the defect the most?
4. Calibrate – Calibrate the lighting and camera system. Quantify the state of the lighting and camera as a system before testing the inspection system. Determine if the lighting in the field of view is homogeneous. Ensure that the background does not change with time. Calculate the average pixel gray scale value and standard deviation for the image. Inspect the calibration image with line profiles to determine if there are lighting gradients in the image. Guarantee lighting, background, and camera consistency.
5. Compensate and correct – If needed, correct for poor lighting with software. Condition the image scene so that it is easier to process with software. If you cannot create a consistent homogeneously illuminated scene, use software to correct for poor lighting.
6. Identify a fiducial element – Select a unique feature that is not a defect but is always present in the image. This unique feature is used as a point of reference or fiducial. Offset from the fiducial to inspect. If the fiducial is not present then the part being inspected is bad.
7. Locate features – Select a feature-locating technique based on the features and speed requirements for your application. If the feature is of a known size and orientation, use grayscale pattern matching. In general, if the feature is of a known shape and unknown size, use binary shape matching. If the feature is of a known area and perimeter but with varying orientation, use blob analysis.
8. Test inspection – Test the inspection strategy with ideal images and defects. Then test the inspection strategy with images that show atypical defects.
9. Automate – Include lighting and camera calibration in the automated inspection system.

   Common Inspection Applications
   

Driven by advancements in converging technologies such as faster CPUs, robust operating systems, PCI local bus, and user friendly image processing and image acquisition hardware and software, PC-based vision systems are becoming commonplace in industry. In industrial applications, which make up the largest segment of the vision market, vision is used to test incoming parts for quality and to detect missing components. Computer vision systems are more reliable and cost-effective than humans in the high-speed, detailed repetitive manufacturing processed required in making semiconductor, electronic, medical, pharmaceutical, and computer products. Industrial inspection of electronic components, such as connectors, switches, LCD/LED displays, relays, and watch parts, is one of the key applications for PC-based vision systems. Because these components are manufactured in high quantity and are small, human inspection is tedious and time-consuming. Vision systems, on the other hand, can be used robustly to perform such tasks. Another growing application area for vision systems in industry is telecommunication and computer peripherals. Consumer products, such as pagers, printers, monitors, cellular phones, disk drives and components, must pass quality standards required by ISO 9000. Vision systems can assist in the ISO 9000 certification process by recording fault incidents. They can also be used to verify the quality of the vendor’s product. Overall, many machine vision applications can be classified into four areas:

1. Measuring tolerance measurements or “gauging” a component
2. Part present/not present
3. Alignment – determining the orientation and position of a part
4. Identification – identifying a bar code, seven-segment display, meter, or written words

   Parameters That Describe an Edge
   

The IMAQ Vision Edge Tool VI uses three parameters – contrast, width, and steepness – to calculate the location of edges along a path within the image defined by pixel coordinates. Edges can occur on lines or arbitrary regions of interest.

The contrast parameter specifies the threshold for the contrast of the edge. Only edges with a contrast greater than the specified value are used in the detection process. Contrast is defined as the difference between the average pixel intensity before the edge and the average pixel intensity after the edge. The filter width specifies the number of pixels that are averaged to find the contrast at each side of the edge. The steepness specifies the slope of the edge. This value represents the number of pixels that correspond to the transition area of the edge.

For an edge to be located in the line profile, using the filter width and steepness settings, the edge contrast between foreground and background must be greater than the contrast setting.

Edge locations can be calculated to subpixel accuracy using quadratic or cubic spline interpolation.

The subpixel accuracy specifies the number of samples that are obtained from a pixel. For example, a subpixel accuracy of one fourth specifies that each pixel is split into four subpixels.

How to Use Gauging Techniques to Measure Tolerances

In-Line Gauging Applications
Gauging refers to making critical measurements such as lengths, diameters, angles, and counts to determine if the product is manufactured correctly. If the gauged parameter does not fall within tolerance limits, the component or part is rejected. Gauging is often used both in line and off line in production. In in-line processes, each component is inspected as it is manufactured. In-line gauging inspection is often used in mechanical assembly verification, electronic packaging inspection, container inspection, glass vile inspection, and electronic connector inspection.

Off-line Gauging Applications
Often gauging applications measure the quality of products off line. A sample of products is extracted from the production line. Then measured distances between features on the object are used to determine if the sample falls within a tolerance range. Using gauging techniques you can measure the distance between blobs and edges in binary images and easily quantify image measurements.

How to Inspect for Missing Components

Part present/not present applications are typical in electronic connector assembly and mechanical assembly applications. The objective of the application is to use line profiles and edge detection to determine if a part is present or not present. An edge along the line profile is defined by the level of contrast between background and foreground and the slope of the transition. Using this technique, you can count the number of edges along the line profile and compare the result to an expected number of edges. The method of limiting processing to lines, known as line profiling, offers a less numerically intensive alternative to other image processing methods such as image correlation and pattern matching.

How to Create an Alignment Application

In many inspection applications, the object or part that is being inspected can occupy different parts of an image and can be in different orientations.

Consider for example a floppy disk inspection application. The objective of this application is to determine if the label that specifies the density of the disk “HD” is printed correctly. Because these disks usually come down a conveyer belt in production, in each acquired image the disk can be translated and rotated. To be able to track the correct location of the “HD” symbol on the disk, a coordinate system with respect to the disk boundaries must be used. In IMAQ Vision, this is done by using a Coordinate Reference.VI, which requires an input of two points along the top boundary of the disk (the x-axis of the disk) and one point along the left boundary (the y-axis of the disk). Using these three points the function computes a coordinate system for the disk, in a sense, a “local” coordinate system for the disk. Using a local coordinate system resolves the orientation issue. The floppy disk can be at a wide range of orientations for inspection. These three points are obtained by finding the location of
the boundary (or edge) using the Edge Tool.VI at the top and left boundaries of the disk. The location of the HD label on the disk can then be determined as an offset to this coordinate system. For each acquired image of the disk, three points are required to establish the disk current coordinate system. The translation and rotation of the disk are then computed. These values are then used to calculate the position of the HD label in the most recent image. The current HD label is then matched to a template image using the Shape Matching.VI to determine its quality. This process of establishing a local coordinate system for the object in order to make measurements insensitive to the orientation of the object in the image is called alignment.

Identification
In many applications, you simply want to identify or make a reading. For example, you might want to read a bar code, inspect a speedometer to make sure it is calibrated, or to read the seven segment display of a microwave oven during production to ensure the readings work correctly. IMAQ Vision has many built-in functions for reading LCDs, bar codes, and gauges.


A straightforward way to inspect an LCD is first to calibrate the size of the digits when all segments are active. This calibration will determine the location of the boundaries of each seven-segment group. By knowing the location of the boundaries, the inspection algorithm can quickly analyze subsequent images to determine which of the segments is on. Once you know which segments are on, you can map the inspected value to a corresponding display value. The final step involves comparison of the inspected value and the known value to determine if the part is displayed correctly or flawed.

Moving to the details of the algorithm, we first calibrate the size of the seven-segment number by inspecting when all segments are on (giving a display of 8). For this calibration, an operator captures an image of the display and draws a calibration box around all of the displays shown on the acquired image. The operator need not draw the region of interest (ROI) exactly, but it must at least surround all of the displays.

After drawing the ROI, the operator next activates an automatic calibration algorithm. The algorithm first finds all of the vertical bars of the displays by drawing a horizontal line at levels that are 1/3 and 2/3 the height of the ROI. The line slices through the activated segments and returns the position of each vertical segment by using an image processing function called edge detection. An edge along the line profile is defined by the level of contrast between background and foreground and the slope of the transition. Using this technique you can determine if any of the seven segments are defective – active at the wrong time or inactive at the wrong time. This method of limiting processing to lines, known as line profiling, offers a less numerically intensive alternative than other image processing methods such as image correlation and pattern matching. Once the position of the vertical segments is known, the algorithm next locates horizontal segments by inspecting the pixels along a vertical line profile drawn between the location of the vertical segments.

The calibration algorithm, which is designed for LCD and electroluminescent indicators, is insensitive to light drift because it uses contrast values along a line profile. In other words, as long as there are 30 grayscale levels between the foreground and the background, then an edge (or segment) will be detected. (We quantify light drift as the difference between the average pixel values at the top left and the bottom right of the background of the LCD screen.) After locating the vertical and horizontal segments, the function returns an array that is the area of interest and contains the digits. Overall, a local coordinate system defining the digits is returned. A local coordinate system based on this calibration image simplifies and improves the performance of inspection on subsequent images.

Upon completion of calibration, we assume that the size and boundaries of the segment groups will not shift with each new devict under test (DUT), and we run an inspection algorithm many times without recalibration. The inspection function draws a line profile through each of the segments and uses edge detection to determine the on or off state of each segment. The corresponding numeric value (0-9) is then assigned to the display inspected. In the final step, we compare this number with an expected value that is fed to the DUT through a serial line. A mismatch indicates a defective part.

Unfortunately, our algorithm is not perfect. Four factors can cause a bad detection: 1) horizontal or vertical light drift (greater than 90 in an 8-bit image); 2) insufficient contrast between the background and the segments; 3) noise; and 4) insufficient resolution. To quantify several of these factors and determine representative minima and maxima that ensure accuracy for 8-bit (256 gray level) images, we make several definitions. For one, light drift greater than 90, as defined above, will cause problems. Contrast must exceed 30, when defined as the difference between the average pixel values in rectangular regions in the background and foreground. Noise must not exceed 15, when we define it as the standard deviation of the pixel values contained in a rectangular region in the background. Finally, in terms of resolution, the digit must be larger than 12 to 18 pixels to obtain accurate results.

Despite such imperfections, the algorithm is useful for other inspection tasks. For instance, with some modifications, it can test analog gauges and speedometers. The strategy for inspecting gauges is straightforward; you calibrate the full range of the gauge by drawing a line profile along the needle at the minimum reading and at the maximum reading. In so doing, we determine the center point about which the needle swings and the perimeter of the area of swing. After the calibration, we can detect the needle position with line profiles. If the needle is black with a white background, the line profile with the lowest value is the location of the needle. Overall, edge detection and line profiling are very common and simple to understand. They are imaging techniques that can be used in a wide range of inspection applications from mechanical assembly, electronic packaging inspection, quality of markings, and electronic connector inspection.

Example -- Using Gauging to Inspect an Aerosol Can

Objective: To understand the concept of gauging in inspection. The objective of this gauging application is to determine if the spray can tip has been correctly assembled. The spray tip must be in the vertical 90° position or within a tolerance of ±5 degrees. If the spray tip, as installed, is out of tolerance, the part is rejected. Because aerosol cans are coming down a conveyer belt, the position of the can in the image is not fixed. The cans may shift in both horizontal and vertical directions. To correctly determine the position of the nozzle, a local coordinate system must be associated with the can. We do this by finding three points, two along the x-axis and one along the y-axis. Using three points, you can calculate the shift and rotation of the can and move the ROI in the image accordingly.

Description: This aerosol can inspection system calculates the spray tip angle to determine if it has been properly placed.

Objective: To inspect for missing parts to determine if the item has been properly assembled.


The number of edges under the line profile is used to determine if this connector has been properly assembled. Detection of eight edges means that there are four wires. Any other edge count means that the part has not been assembled correctly.

IMAQ Vision Functions
	(IMAQ Vision»Inspection Tools»Caliper Tools»IMAQ ROIProfile.vi) Calculates the profile of the pixels along the boundary of an ROI descriptor. This VI returns a data type (cluster) that is compatible with a LabVIEW or BridgeVIEW graph. This VI also returns other information such as pixel statistics and the true coordinates of the ROI boundary.
	(IMAQ Vision»Inspection Tools»Caliper Tools»IMAQ Edge Tool.vi) Finds edges along a path defined in the image. Edges are determined based on their contrast, width, and steepness.
	(Get Line.vi) This is a subVI that is not shipped with IMAQ Vision. Its function is to wait for a line to be drawn on the image and then pass out the ROI descriptor.

Summary

Driven by converging technologies, advanced PC-based vision and image processing for test, measurement, and industrial automation are a reality. For machine vision developers who need to quickly develop gauging applications, National Instruments LabVIEW and IMAQ Vision software contains high-level gauging and caliper tools that speed up application development. These functions, which deliver a high level of accuracy, are reliable tools for missing part inspection, guidance, and gauging applications. Thanks to the graphical language of LabVIEW and IMAQ Vision, you are empowered to develop sophisticated machine vision applications. Moreover, because of the diverse tools within LabVIEW, other types of I/O such as motion control, instrument control, and data acquisition are easily integrated into your application.

Reader Comments | Submit a comment »