Principles, characteristics and application fields of machine vision technology
“The most basic feature of the machine vision system is to improve the flexibility and automation of production. In some dangerous working environments that are not suitable for manual operation or occasions where artificial vision is difficult to meet the requirements, machine vision is often used to replace artificial vision.
Machine vision is a comprehensive technology, including image processing, mechanical engineering technology, control, electric light source lighting, optical imaging, sensors, analog and digital video technology, computer software and hardware technology (image enhancement and analysis algorithms, image cards, I/O card, etc.).
A typical machine vision application system includes image capture, light source system, image digitization module, digital image processing module, intelligent judgment decision module and mechanical control execution module.
The most basic feature of the machine vision system is to improve the flexibility and automation of production. In some dangerous working environments that are not suitable for manual operation or occasions where artificial vision is difficult to meet the requirements, machine vision is often used to replace artificial vision.
At the same time, in the process of large-scale repetitive industrial production, the use of machine vision inspection methods can greatly improve the efficiency and automation of production.
The feature of machine vision system is to improve the flexibility and automation of production.
A typical industrial machine vision system includes: light source, lens (fixed focus lens, zoom lens, telecentric lens, microscope lens), camera (including CCD camera and CMOS camera), image processing unit (or image capture card), image Processing software, monitors, communication/input/output units, etc.
The system can be subdivided into
1. Separate systems for collection and analysis.
Frame Grabber and Image Processor
fixed focal length lens
Halogen light source LED light source
High frequency fluorescent light source
Other special light sources
Mechanism and Control System
PLC, PC-Base controller
Servo motion machine
2. Integrated system of acquisition and analysis
Smart camera (integrated image acquisition and analysis)
Other supporting peripheral equipment: light source, display, PLC control system, etc.
The machine vision inspection system uses a CCD camera to convert the detected target into an image signal and transmit it to a dedicated image processing system.
According to the pixel distribution, brightness, color and other information, it is converted into a digital signal, and the image processing system performs various operations on these signals to extract the characteristics of the target, such as area, number, position, and length.
Then output the results according to the preset allowable degree and other conditions, including size, angle, number, qualified/unqualified, yes/no, etc., to realize the automatic identification function.
A typical machine vision system includes the following five blocks:
Lighting is an important factor affecting the input of machine vision system, which directly affects the quality and application effect of input data. Since there is no universal machine vision lighting equipment, for each specific application instance, the corresponding lighting device should be selected to achieve the best effect.
Light sources can be divided into visible light and invisible light. Several commonly used visible light sources are incandescent, fluorescent, mercury and sodium lamps. The disadvantage of visible light is that the light energy cannot remain stable. How to keep the light energy stable to a certain extent is an urgent problem to be solved in the practical process.
On the other hand, ambient light may affect the quality of the image, so the method of adding a protective screen can be used to reduce the influence of ambient light. Lighting systems can be divided into: back lighting, forward lighting, structured light and stroboscopic lighting according to their illumination methods.
Among them, back lighting is that the object to be measured is placed between the light source and the camera, and its advantage is that a high-contrast image can be obtained. For forward lighting, the light source and the camera are located on the same side of the object under test, which is convenient for installation.
Structured light illumination is to project gratings or line light sources onto the measured object, and demodulate the three-dimensional information of the measured object according to the distortion they produce.
Strobe lighting is to irradiate high-frequency light pulses onto objects, and the camera shooting requires synchronization with the light source.
FOV (Field of Vision) = desired resolution * sub-pixel * camera size / PRTM (part measurement tolerance ratio)
Lens selection should pay attention to:
⑤ The distance from the image to the target
⑥ center point / node
How to determine the focal length of the lens in visual inspection
The following factors must be considered when selecting the right industrial lens for a specific application:
• Field of View – The size of the area being imaged.
• Working Distance (WD) – The distance between the camera lens and the object or area being viewed.
• CCD – The size of the camera’s imaging sensor unit.
• These factors must be treated in a consistent manner.
If you are measuring the width of an object, you need to use a horizontal CCD specification.
If you measure in inches, do the calculation in feet and then convert to millimeters.
C. High-speed camera
According to different standards, it can be divided into: standard resolution digital camera and analog camera, etc.
Divided by resolution, the ordinary type with the number of pixels below 380,000, and the high-resolution type with the number of pixels above 380,000;
According to the size of the photosensitive surface, it can be divided into 1/4, 1/3, 1/2, 1 inch cameras;
According to the scanning method, it can be divided into line scan camera (line scan camera) and area scan camera (area scan camera); (area scan camera can be divided into interlace scan camera and progressive scan camera);
According to the synchronization method, it can be divided into ordinary cameras (internal synchronization) and cameras with external synchronization functions.
D. Frame grabber
The frame grabber is only one part of a complete machine vision system, but it plays a very important role.
The frame grabber directly determines the interface of the camera: black and white, color, analog, digital, etc.
Typically a PCI or AGP compatible capture card can quickly transfer images to computer memory for processing. Some capture cards have built-in multiplexers.
For example, you can connect 8 different cameras, and then tell the capture card which camera captures the information to use.
Some capture cards have a built-in digital input to trigger the capture card to capture, and the digital output triggers the gate when the capture card captures an image.
E. Vision processor
The vision processor integrates the capture card and the processor.
When computers were slow in the past, vision processors were used to speed up visual processing tasks. The capture card transfers the image to memory for computational analysis.
The current mainstream configuration of PLC, and the configuration is high, the vision processor has almost withdrawn from the market.
In machine vision systems, obtaining a high-quality, processable image is critical.
The reason for the success of the system is to ensure that the image quality is good and the features are obvious. The failure of a machine vision project is mostly due to poor image quality and indistinct features. To ensure a good image, you must choose a suitable light source.
The basic elements of light source selection:
Contrast: Contrast is very important for machine vision. The most important task of lighting for machine vision applications is to maximize the contrast between the features that need to be observed and the image features that need to be ignored, so that the features can be easily distinguished. Contrast is defined as a sufficient amount of grayscale difference between a feature and its surrounding area. Good lighting should ensure that the features to be detected stand out from other backgrounds.
Brightness: When choosing between two light sources, the best option is to choose the brighter one. When the light source is not bright enough, three bad situations may arise. First, the signal-to-noise ratio of the camera is not enough; because the brightness of the light source is not enough, the contrast of the image must be insufficient, and the possibility of noise on the image will increase immediately. Secondly, the brightness of the light source is not enough, and the aperture must be increased, thereby reducing the depth of field. In addition, when the brightness of the light source is not enough, random light such as natural light will have the greatest impact on the system.
Robustness: Another way to test a good light source is to see if the light source is the least sensitive to the position of the part. The resulting image should not change when light sources are placed in different areas of the camera’s field of view or at different angles. A highly directional light source increases the possibility of specular reflection on the highlighted area, which is not conducive to the subsequent feature extraction.
A good light source needs to be able to make the features you are looking for very obvious, in addition to the camera being able to capture the part, a good light source should be able to produce maximum contrast, be bright enough, and be insensitive to changes in the position of the part.
Once the light source is selected, the rest of the work is much easier.
In the production process of cloth, such highly repetitive and intelligent work as cloth quality inspection can only be done by manual inspection.
Behind the modern assembly line, many inspection workers are often seen to perform this process, but still cannot guarantee a 100% inspection pass rate (ie “zero defect”). The automated transformation of the assembly line makes the cloth production line a fast, real-time, accurate and efficient assembly line.
On the assembly line, the color and quantity of all fabrics must be automatically confirmed (hereinafter referred to as “fabric detection”).
Feature extraction and identification
Generally, cloth detection (automatic identification) first uses a high-definition, high-speed camera lens to shoot a standard image, and then sets a certain standard on this basis; then shoots the detected image, and then compares the two. But it is more complicated in the cloth quality inspection project:
1. The content of the image is not a single image, and the quantity, size, color and position of impurities in each measured area may not be consistent.
2. The shape of the impurities is difficult to determine in advance.
3. There may be a lot of noise in the image due to the light reflected by the fast movement of the cloth.
4. On the assembly line, the detection of cloth has real-time requirements.
For the above reasons, corresponding algorithms should be adopted in image recognition processing to extract the characteristics of impurities, perform pattern recognition, and realize intelligent analysis.
In general, images acquired from color CCD cameras are RGB images.
That is to say, each pixel is composed of three components of red (R) green (G) blue (B) to represent a point in the RGB color space.
The problem is that these chromatic aberrations are different from what the human eye perceives. Even a small amount of noise can change the position in the color space. So no matter how similar our human eyes feel, they are not the same in color space.
For the above reasons, we need to convert RGB pixels into another color space, CIELAB. The purpose is to make our human eye feel as close as possible to the color difference in the color space.
According to the processed image obtained above, according to the requirements, detect the impurity color spot on a solid color background, and calculate the area of the color spot to determine whether it is within the detection range.
Therefore, the image processing software should have the functions of separating the target, detecting the target, and calculating its area.
Blob Analysis (Blob Analysis) is to analyze the connected domain of the same pixels in the image, which is called Blob. The color spots in the image processed by Binary Thresholding can be considered as blobs.
Blob analysis tools can separate objects from the background and can calculate the number, location, shape, orientation and size of objects, and can also provide topology between related blobs.
Instead of analyzing individual pixels one by one, the processing is performed on the lines of the graphic. Each row of the image is run-length encoded (RLE) to represent adjacent target ranges.
Compared with the pixel-based algorithm, this algorithm greatly improves the processing speed.
Results Processing and Control
The application program stores the returned result in the database or the position specified by the user, and controls the mechanical part to do the corresponding movement according to the result.
According to the identification result, it is stored in the database for information management. In the future, the information can be retrieved and inquired at any time. The manager can know the busyness of the assembly line within a certain period of time, and make arrangements for the next step; can know the quality of the inner cloth and so on.
The application of machine vision mainly includes two aspects: detection and robot vision:
1. Detection: It can be divided into high-precision quantitative detection (such as cell classification of photomicrographs, size and position measurement of mechanical parts) and qualitative or semi-quantitative detection without measuring instruments (such as product appearance inspection, assembly line inspection, etc.). Part identification and positioning, defect detection and assembly completeness detection).
2. Robot vision: It is used to guide the operation and actions of the robot in a large range, such as picking workpieces from a messy pile of workpieces sent from a hopper and placing them on a conveyor belt or other equipment in a certain orientation (ie, hopper picking problems) . As for small-scale operations and actions, tactile sensing technology is also required.
Beside this there is:
Automated Optical Inspection
Product quality grade classification
Automatic detection of printed matter quality
and other applications of machine vision image recognition.
【Machine Vision Features】
⒈ The speed of the camera automatically matches the speed of the object to be measured, and the ideal image is captured;
⒉ The size of the parts ranges from 2.4mm to 12mm, and the thickness can be different;
3. The system selects workpieces of different sizes according to the operator, calls the corresponding vision program for size detection, and outputs the results;
⒋For parts of different sizes, the sorting device and the conveying device can precisely adjust the width of the material channel, so that the parts can move on a fixed path and perform visual inspection;
⒌The resolution of the machine vision system reaches 2448×2048, and the dynamic detection accuracy can reach 0.02mm;
⒍ The missed detection rate of waste products is 0;
⒎The system can monitor the detection process by displaying images, and can also dynamically view the detection results through the detection data displayed on the interface;
⒏ It has the function of timely and accurately sending out the rejection control signal to the wrong workpiece and rejecting the waste;
⒐The system can self-check whether the status of its main equipment is normal, and it is equipped with a status indicator; at the same time, it can set different operation permissions for system maintenance personnel and users;
⒑ Real-time display of the inspection screen, Chinese interface, you can browse the images of unqualified products several times, and have the function of being able to store and view the images of faulty workpieces in real time;
⒒ Can generate error result information files, including corresponding error images, and can print out.
Example 1. Intelligent integrated test system for instrument panel assembly based on machine vision
The EQ140-Ⅱ automobile instrument panel assembly is an instrument product produced by a Chinese automobile company. The instrument panel is equipped with a speedometer, a water temperature meter, a gasoline meter, an ammeter, and a signal warning light. Final quality inspection.
The inspection items include: detecting the indication error of the five instrument pointers such as the speedometer; detecting whether the 24 signal alarm lights and several lighting 9 lights are damaged or missing, generally by manual visual inspection.
The intelligent integrated test system based on machine vision realizes intelligent, automatic, high-precision and fast quality inspection of the instrument panel assembly.
The whole system is divided into four parts: an integrated multi-channel standard signal source that provides analog signal sources for the instrument panel, a dual-coordinate CNC system with image information feedback positioning, a camera image acquisition system, and a master-slave parallel processing system.
Example 2. Automatic damage control system on metal plate surface
The surface quality of metal plates, such as large power transformer coils, flat wires, radio hazy skins, etc., has high requirements, but the original detection method using manual visual inspection or dial indicator plus needle control is not only susceptible to subjective factors, but also May paint the surface to be tested and bring new scratches.
The automatic flaw detection system for metal plate surface uses machine vision technology to automatically inspect metal surface defects, and performs high-speed and accurate detection during the production process. At the same time, due to the non-contact measurement, the possibility of new scratches is avoided.
In this system, a laser is used as the light source, the stray light around the laser beam is filtered out by a pinhole filter, and the beam expander and collimator mirror make the laser beam into parallel light and uniformly illuminate the inspected object with an incident angle of 45 degrees. sheet metal surface.
The metal plate is placed on the inspection table. The inspection table can move in three directions of X, Y and Z. The camera adopts TCD142D 2048 line Chen CCD, and the lens adopts ordinary camera lens. The CCD interface circuit adopts the single-chip microcomputer system.
The host PC mainly completes image preprocessing, defect classification or scratch depth calculation, etc., and can display the detected defects or scratch images on the monitor. The two-way communication between the CCD interface circuit and the PC is carried out through the RS-232 port, and combined with the asynchronous A/D conversion method, it constitutes a man-machine interactive data acquisition and processing.
The system mainly uses the self-scanning characteristics of the linear CCD and the movement of the inspected steel plate in the X direction to obtain the three-dimensional image information of the surface of the metal plate.
Example 3. Vehicle Body Inspection System
The 100% on-line inspection of the dimensional accuracy of the 800 series car body of the British ROVER Automobile Company is a typical example of the machine vision system used in industrial inspection.
The system consists of 62 measuring cells, each including a laser and a CCD camera, to detect 288 measuring points on the body shell. The car body is placed under the measuring frame, and the precise position of the body is calibrated by software.
The calibration of the measuring unit will affect the detection accuracy, so it is paid special attention. Each laser/camera unit is calibrated offline. At the same time, there is a calibration device calibrated with a coordinate measuring machine in an offline state, which can perform online calibration of the camera top.
The inspection system detects three types of car bodies at a rate of one car body every 40 seconds. The system compares the test results with the qualified size of the person and the CAD model, and the measurement accuracy is ±0.1mm.
ROVER’s quality inspectors use the system to determine the dimensional consistency of key parts, such as the overall body shape, doors, and glass windows. Practice has proved that the system is successful and will be used for vehicle body inspection of other ROVER systems.
Example 4. Banknote Printing Quality Inspection System
The system uses image processing technology to compare and analyze more than 20 features (number, Braille, color, pattern, etc.) of the banknotes on the banknote production line to detect the quality of the banknotes, replacing the traditional human eye identification method.
Example 5. Intelligent Traffic Management System
By placing cameras on traffic arteries, when there are illegal vehicles (such as running a red light), the camera will take pictures of the license plates of the vehicles and transmit them to the central management system. The system uses image processing technology to analyze the captured pictures and extract the license plate number. Stored in the database, it can be retrieved by managers.
Example 6. Metallographic Analysis
The metallographic image analysis system can accurately and objectively analyze the matrix structure, impurity content and tissue composition of metals or other materials, and provide a reliable basis for product quality.
Example 7. Medical Image Analysis
Automatic classification and counting of blood cells, chromosome analysis, cancer cell identification, etc.
Example 8. Bottled beer production line inspection system
It can detect whether the beer reaches the standard capacity and whether the beer label is complete
Example 9. Large-scale workpiece parallelism and perpendicularity measuring instrument
A large-scale workpiece parallelism and perpendicularity measuring instrument using laser scanning and CCD detection system. It uses a stable collimated laser beam as the measurement baseline, is equipped with a rotating shaft system, and rotates a pentagonal prism to sweep out parallel or perpendicular reference planes. Compare it to the sides of the large workpiece being tested. When machining or installing large workpieces, the error detector can be used to measure the parallelism and perpendicularity between surfaces.
Example 10. Detecting device for outer profile dimensions of rebar
Using the stroboscopic light as the illumination light source, and using the area array and linear array CCD as the detection device for the outline and dimension of the rebar, a dynamic detection system for online measurement of the geometric parameters of the hot-rolled rebar is realized.
Example 11. Real-time monitoring of bearings
Vision technology monitors bearing load and temperature changes in real time, eliminating the danger of overloading and overheating.
The traditional passive measurement of machining quality and safe operation by measuring the ball surface is transformed into active monitoring.
Example 12. Crack Measurements on Metal Surfaces
Using microwave as the signal source, according to the square wave of different wave rates emitted by the microwave generator, the cracks on the metal surface are measured. The higher the frequency of the microwave wave, the narrower the measurable cracks.
Because the machine vision system can quickly acquire a large amount of information, it is easy to process automatically, and it is also easy to integrate with design information and processing control information. Therefore, in the modern automated production process, people use the machine vision system widely for working condition monitoring and finished product inspection. and quality control.
However, machine vision technology is more complex, and the biggest difficulty is that the human visual mechanism is still unclear. People can use introspection to describe the process of solving a problem and simulate it with a computer. But although every normal person is a “vision expert”, it is impossible to describe his own visual process by introspection.
Therefore, building a machine vision system is a very difficult task.
It can be expected that with the maturity and development of machine vision technology itself, it will be more and more widely used in modern and future manufacturing enterprises.