赵会斌 程俊
摘 要 随着微电子技术的发展,印刷电路板上的表面贴装设备(SMD)的尺寸越来越小,密度也越来越大。传统的依靠人眼去检测SMD的质量已不太可能,相反,自动光学检测(AOI)具有独特的优势,越来越受到人们的重视和欢迎。美国国家仪器(NationalInstrument,NI)的视觉开发模块以其适应性强、配置灵活、开发简单快速等优势在机器视觉和图像处理已经处于领先的地位。本文以LabWindows/CVI为软件开发环境,利用NI的视觉开发模块开发了一套印刷电路板表面贴装设备的自动光学检测系统。实验证明,该系统配置灵活,检测精度高,完全能满足工业生产中的实际需求。关键词 自动光学检测;PCB;机器视觉;NIVision
ABSTRACT With the development of the micro-electronic industry, surface mount devices (SMD) on the printed circuit board (PCB) become more and more micro size and its mounted density is increasing. It is out of date to depend on manual inspection to assure the joints quality. Instead, automated optical inspection (AOI) for solder joints based on the machine vision has become more and more important. NI Vision with the advance of flexibility and multipurpose has been a leader in machine vision and image processing. This paper proposed the devices and method to develop the AOI system for surface mount devices (SMD) with NI Vision and the inspection result is given and discussed.KEYWORDS AOI; NI vision; PCB; machine vision
1 IntroductionR
ecently, with electronic products become more and more portable, small, thin and multi-function, components in printed circuit board (PCB) tend to be ultra-thin, micro-size and high-density. Accordingly, it has an increasing quality demand to solder joints (which connect components to PCB) in surface mounted technology (SMT) production line. It is obviously out of date to depend on manual inspection to assure solder joints quality. Fortunately, automated optical inspection (AOI) for solder joints based on machine vision has been applied to SMT line widely. AOI system can inspect PCB with density components repeatedly, reliably and unanimously, it has less misjudgment rate and quite fast inspection velocity, which makes AOI to be more and more important in SMT [1-3].
Vision systems can be used to precisely measure number
The work described in this paper is supported by the National Nature Science Foundation of China (No.0000).Lab for Semiconductor Equipment Shenzhen Institute of Advanced Integration Technology , Chinese Academy of Science.hb.zhao@sub.siat.ac.cn, jun.cheng@siat.ac.cn
of variables such as length, angle, position, orientation, color and so on. The main advantage of a vision measuring system in these applications is its non-contact feature, which is important in cases when it is difficult to implement contact measurement [4].
National Instruments has been a leader in machine vision and image processing for nearly a decade and currently supports thousands of different cameras. To acquire, display, save, and monitor images from cameras, use NI Vision Acquisition software, which is included with all NI frame grabbers and sold separately for FireWire and Gigabit Ethernet image acquisition. To process images, NI Vision software comes in two different packages: the Vision Development Module and NI Vision Builder for Automated Inspection (AI). The Vision Development Module is a collection of hundreds of vision functions for programmers using NI LabVIEW, NI LabWindows/CVI, C/C++ or Visual Basic. Vision Builder AI is an interactive software environment for configuring, benchmarking and deploying machine vision applications without programming. Both software packages work
39
Vol. 3 No.12/ Dec. 2009with all NI Vision frame grabbers and the NI Compact Vision System.
This paper proposed the devices and method to develop the AOI system for SMD. The system consists of IEEE1394 camera, IEEE1394 interface card, 2-axies motion platform, LabWindow/CVI software development environment and NI Vision Development Module.
vision camera designed for a wide range of demanding applications in automated inspection, computer vision, semiconductor and component inspection, manufacturing quality control, food and beverage inspection, and many other vision applications. Its resolution is 1.4 mega pixels (13×1024).
2.2 SoftwareWe use NI Vision Development Module (VDM) for LabWindows/CVI to inspect the SMD on PCB.VDM is machine vision application development software for LabVIEW, C/C++, Visual Basic, and .NET. It includes hundreds of image processing and machine vision functions. It also includes vision acquisition software with support for hundreds of cameras including IEEE1394 and Gigabit Ethernet cameras.
We also use NI Vision Assistant which is another vision development module software product to prototype application strategy quickly without having to do any programming.
For motion control, the 8134.dll of PCI8134 which contain a variety of standard applications can be used as a starting point for applications development.
Figure .1 Hardware structure of the AOI system
2 System Overview2.1 HardwareT
he hardware structure of the AOI system is shown in Fig. 1.It consists of motion subsystem, control subsystem and vision subsystem. The motion system consists of 2 axes rotary servo motors which can be transferred to plane motion through screw. The inspected PCB on the motion table can move freely in X-direction and Y-direction under the control of the PC and servo drive. The IEEE1394 digital camera installed above the motion table acquires the image of PCB and sends it to PC for vision analysis and measurement.
3 Vision DevelopmentFigure . 2 Servo control diagram
Fig. 2 shows the servo control diagram for single axis motor. PCI8134 which is a 4 axes motion control card with PCI interface can generate high frequency pulses 0.6 0.5 0.4 0.35 0.3 0.2 Scale
to drive stepping motors and servo motors. Incremental 853.33 850.00 842.50837.14833.33815.00X (pixel)
635.00 638.00 2.502.866.67660.00encoder interface on all four axes provide the ability to Y (pixel)
21.2 16.9 16.2 12.1 11.7 9.3 Time(ms)correct for positioning errors generated by inaccurate
mechanical transmissions. The main feature of PCI8134: 32-bit PCI bus, plug & play; pulse output rate up to 2.4 MHz; pulse output options: OUT/DIR, CW/CCW; 2 axes linear interpolation; programmable acceleration and deceleration time; trapezoidal and S-curve velocity profiles; 28-bit up/down counter for incremental encoder feedback. The resolution of our servo motor is 131072 pulses/rev, the ballscrew lead of X axis is 5mm and the ballscrew lead of Y axis is 10mm, so the maximum resolution of X axis is 0.038μm and the maximum resolution of Y axis is 0.076 μm/pulse.
The camera which is Prosilica’s EC1350C CCD camera with firewire interface is a high performance machine
Scale
0.6
0.5
0.4
0.35
0.3
0.2
Fig. 3 illustrates the basic steps involved in performing machine vision for AOI tasks in LabWindows/CVI.
Figure . 3 Steps to perform machine vision
3.1 Mark RecognitionM
X (pixel) Y (pixel)
853.33 635.00 21.2
850.00 638.00 16.9
842.502.5016.2
837.142.8612.1
833.336.6711.7
815.00660.009.3
Time(ms)
ark recognition is to find the mark points in the PCB and then to locate the object.
Mark point can be circle, rectangle and so on. Our system uses circle as the mark point as shown in Fig. 4. The red circle and its outer black circle is the mark point we find.
In Fig. 4, there are many other small holes in PCB which will disturb the inspection result. So we must process the origin image and exclude the effect of the disturbed holes before we make inspection and recognition. The
0
基于NI视觉开发模块的表面贴装设备的自动光学检测系统:设备和方法processing step is shown in Fig. 5.
algorithm organizes and stores these features and the spatial relationships between these features in a manner that facilitates faster searching in the inspection image.During the matching stage, the geometric matching algorithm extracts geometric feature from the inspection image that correspond to the features in the template image. Then, the algorithm finds matches by locating regions in the inspection image where features align themselves in spatial patterns similar to the spatial patterns of the features in the templates.
In LabWindows/CVI, the function imaqDetectCircles () based geometric matching is used to perform the task. In the function, the threshold which specifies the minimum contrast a seed point must have in order to begin a curve must be set carefully to exclude the disturbing circle. We set the parameter as 220 and get a good result. The minimum radius of the circle also can be set to exclude the small holes. Here we set it as 10. The inspection result is listed in Table I. The coordinate origin is in top left corner of the inspected image by default. There exits a little difference corresponding to different resize scale. The reason lies in the interpolation method when resize the origin image using Resample () function in LabWindows. But it doesn’t affect the inspection result if we convert the coordinate to the benchmark.
Table 1 MARK INSPECTION RESULT
ScaleX (pixel) Y (pixel) Time(ms)
0.6
0.5
0.4
0.35
0.3
0.2
Figure . Mark Point
Figure . Processing step
The origin image’s size is 1360 × 1024, it is so big that it will waste long time to process. So it is necessary to resize the origin image by scale of 0.2. But the result must multiply the same scale of 5 correspondingly to recover its real position in origin image.
To eliminate the overlay white circle, we must increase the contrast of mark but decrease the contrast of the others. So the equalize arithmetic is used. The equalize arithmetic alters the gray-level values of pixels so that they become evenly distributed in the defined grayscale range, which is 0 to 255 for an 8-bit image. The function associates an equal amount of pixels per constant gray-level interval and takes full advantage of the available shades of gray. The equalization can be limited to a gray-level interval, also c.alled the equalization range. The other pixels are set to 0. The image produced reveals details in the regions that have intensity in the equalization range; other areas are cleared. Fig.6 shows the result after equalizing.
853.33 635.00 21.2
850.00 638.00 16.9
842.502.5016.2
837.142.8612.1
833.336.6711.7
815.00660.009.3
3.2 Coordinate ConvertionBefore inspection, the system must be able to relate regions of particular images to individual solder pasts on a PCB or else it will be incapable of determining which solder paste on a given PCB is defective. Usually, the solder paste size and location information in board coordinates will be available from a PCB design/ manufacturing database for each type of board to be inspected [5].
However, in an inline inspection system, when the PCB moving into the inspection position, there are some minor positional or orientation errors because of the mechanical factor and the interpolation factor presented in section A. To eliminate the deviation, it is necessary to align the PCB prior to inspection. This paper presents an algorithm to rectify the error from the above factor.
As shown in Fig. 7, M and N are marks in database with coordinate (Xm, Ym) and (Xn, Yn), M’ and N’ are marks inspected from PCB with coordinate (Xm’, Ym’) and (Xn’, Yn’).The midpoint of M and N is O (Xo, Yo), and the midpoint of M’ and N’ is O’ (Xo’, Yo’). Their coordinate
(1) Pre-Equalized (2) Equalized
Figure . 6 The image pre-equalized and equalized
The geometric matching algorithm is used to detect the mark of circle. Geometric matching is an important tool for machine vision applications; it must work reliably under various, sometimes harsh, conditions. In automated machine vision applications – especially those incorporated into manufacturing process – the visual appearance of materials or components under inspection can change because of factors such as varying part orientation, scale, and lighting. The geometric matching tool must maintain its ability to locate the template patterns despite these changes.
The geometric matching process consists of two stages: learning and matching. During the learning stage, the geometric features from the template image. The
1
Xo
m
n
Yo
Vol. 3 No.12/ Dec. 2009nxy'xyYmYn x'Y PyY' x'Y PyY'mnmn2 X Y X Y oo 'Y PY' xx'L1Y LP'1oocan be calculated as the following equationsBasic correlation is very sensitive to amplitude changes y1YKyK1
YmYnK 1w(x,y)iL,1j)wxx,y)i,fy(xin the image, such as intensity, and in the template. For C(i,j)C (¦¦f((ji),yj)L1K1¦¦XmXnYnY' mo,mnYYC(i,j) w(x,y)f(xi,yj)x 0y 0 YoXo' mmnnx 0y 0L1K11mnexample, if the intensity of the image f is doubled. So L1K¦¦C(i,j) ¦¦w(x,X 2(1)o' 2Xo 22 Yo x 0y 0C(i,j) w(x,y)f(xi,yj)2x 0y 0are the values of CC(i,j) ¦¦w(. We can overcome sensitivity by x,y)f(xi,yj)2¦¦x 0y 0VijVijXmnx 0y 0o'mncomputing the normalized correlation coefficient, which Y'YYYmnX'X' P(,Rij) Vij,X' R(i,j) mnmnxY' oVijY' oX' is defined asn'[f(i,j)]ooR(i,j) D(w)D2D([wf)(i,Dj)]om(,)Rij 2Vij[f(i,j)]22 (2)(i,j)D(w)D[R(i,j) D(w)VDYmYn
R(i,j) YYXnXYmYw)LDYo' XK[f(i,j)] (6)LDK(XYmmmnnmnonYmYnY' Yomn()[(,)]DwDfij XY Y X Y Po2omnLKV(i,j) (w(x,y)w)(f(xi,yj)fooox,P Pxo V(i,j) ¦¦(¦¦w(x,y)w)(f(xi,yj)f(i,j))(i,j))LK '2yX(3)V(i,j) xw(x,0y)w)(f(xi,yj)f(i,Vj)) 0y Xo'Xm'Xn' y oY'mY'nY'L01K01(x¦¦(i,j) ¦¦(w(x,yomnXmx 0y 0 ommnnV(i,j) ¦¦(w(x,y)w)(f(xi,yj)f(i,j)) (7)x 0y 0LKXo nmPnXo' X' xLKox 0y 0LK 2Px X2V(i,j) ¦¦(w(x,y)w)(f2(xi,yj)f(i,j))'YYY2omnYYYomnL1)K1(w(x,y(D(w w(x,y)w)X'D(w) )w)omnX 'Y PY'LK¦¦ Py x 0y 0o mnxy2 D(w) ¦¦Py (w(x,x 0y 0y)w)YmYYYx y 00(8)LK¦¦D(w) ¦¦(w(x,y)YoYmYn'mnYnY'2omnYoY'o ' Yx 0y 0D(w) (w(x,y)w)x 0y 0Yo22YmYn¦¦LKL1K1YYYPy o mnx 0y L01K1L1K12
Py Y YY'X C(i,j2D(w) ((¦¦x,(xy)w)) w(x,y)f(xi,yj)Lj1)]Kw1(¦¦D[f(i, (f(xi,yf(ij,)jf (i,j))2(9)D[f(i,j)] fi,yj)))'Y PY'¦¦L1K1omn¦¦X Y'Y PY'xy2Yy'nYx 0y0moxoommnD[f(i,j)](f(xi,yj)f(i,j))x y 00x 0y 0x 0y 0
Xo'm YYmnn'mmmnn'
omYn'YYYoYoYnm Y Ymn yYmYnPy YPmnY'YYYn'nnPy Yooo YommmYYYYYYomny o mn'PyYYYoYmm'n PY o' Y 'PnY'Px
ooo
x
x
om
n
YYY
P YYY'Py YoYY
mnYoYmYn'
the current location of w. The value of R (i, j) lies in the
frange -1 to 1 and is independent of scale changes in the x 0y 0VijLKas following equationR(i,j) R(i,j) x 0y 0intensity values of V(i,j)D (w)D(w(x,,yj))]w)(f(xi,yj)f(i,f and j))w.[f(iDY(wP)yYD xPxX'',Y 'YX Py ['f(i,j)](4)x 0y 0Normalized cross-correlation is a good technique VijVij(,)Rij for finding patterns in an image when the patterns in 3.3 Matching to Locate and ) LDR(i,jPattern LKK(w)D[f(i,j)]LKL1K11LK1the image are not scaled or rotated. Typically, cross-D(w)D[f(i,j)]Inspect the Wrong Components2xi,yj)f(i,j))V(i,j) (w(x,y)w)(f(V(i,j) (w(x,y)w)(f(xi,yj)f(i,j))D( C(i,(ij,)j w(x(,xy,)yf)(fxi,)xyj)j)(w(x,y)w)C) w(xwi ,0ycorrelation can detect patterns of the same size up to a y 0x 0y 0Pattern matching quickly locates regions of a grayscale x 0y 0K 0x L0y0 0x yrotation of 5° to 10°. Scale-invariant matching adds a LK
image that match a known reference pattern, V(i,j) (w(x,y)w)(f(xi,yj)f(also i,j))significant amount of computation to matching process. V(i,j) L(w(x,y)w)(Lf(xi,yj)f(i,j))KKreferred to as a model or template. When using pattern x 0y 0L1K12To resolve the problem, NI improves the computation V2i(xj,yijDx 0y 0V2(w) (w(x,y)w)D(w) (w)w)matching, first create a template that represents the R(Ri,(ij,)j) D[f(i,j)] (f(xi,yj)f(i,j))time of pattern matching by reducing the size of the x 0y [6]0xyw 0(Df[(fi,Yj,)]D()w)D[D(ij)]objects for which you are searching . Pattern matching x 0y 0L K0Yimage and the template by pyramidal matching. In mnm2nLK ( xX( ) Yapplication then searches for instance of the template in owD( w,oy )w)2
pyramidal matching, both the image and the template D(w) x 0y L(w(x,y)w)fL1K1
1K1each acquired image, calculating a score for each match. 0LLKK
are sampled to smaller spatial resolutions. Because the x 0y 0D[f(ii,,yj)] (jf))(2xi,yj)f(i,j))2D[f(i,j)] (f(xj)f(i,mnThis score relates how closely the template resembles V(i,(oij,)(wx(,xy,)yf(fx(,yj)jf(fi,(ij,))j))Vj ) ((w)w)(w)(xii,y)X' image is smaller, matching is faster. When matching is x 0y 0x y 002the located matches. Pattern matching finds template 1K1x 0x y0 0yL 0
complete, only areas with high match scores need to be L1K1
D[f(iY,mj)](f(xif,yj)f(i,j))2matches regardless of lighting variation, blur, noise, and Yn2
considered as matching areas in the original image.D[f(i(f(xi,yj)f(i,j))f,j)] Yo' LLKx 0y 0geometric transformation such as shifting, rotation, or K
2x 0y 022
scaling of the template.D(w (wx(,xy,)yD()w) ((w)w)w)
fx 0x
0y 0m0ntechniques include normalized foyNI pattern matching Px
'cross-correlation, pyramidal matching, scale-and omn
L1LK11K1
rotation-invariant matching, and image understanding. 22D[f(i,j)] (f(xi,yj)f(i,j))D[f(i,j)] (f(xi,yj)f(i,j))common Normalized cross- correlation is the most YoxYYmy0 0y nx 0 0building Py method for a template in an image. The
YYY'omnfollowing is the basic concept of correlation: consider a f
subimage w(x, y) of size K × L within an image f(x, y) x M × N'Y, where PyY'K ≤ M and L ≤ N. The correlation of size
between w(x, y) and f(x, y) at point (i, j) is given by
L1K1¦¦D[f(i,j)] ¦¦(f( PxPx 2x y 00Figure . Marks in PCB and databaseD[f(i,j)] ¦¦(f(xi,yj)f(i,j))x 0y 0n'n'mL1K1fx 0where X oxo'L1mY PyY'fw (calculated only once) is the average intensity K1Ly1 0K1
X x'to Yabove PyYC'equation, According we can convert Vi)fj(xthe f2(i,j) w(x,yi,yj)f value of the pixels in the template w. The variable C(i,j) w(x,y)f(xi,yj)D[f(i,j)] (f(xi,yj)f(i,j))(,)Rij ¦¦coordinate system in the real PCB to the database fx 0y 0nYnYoYoxYY 0y D(w)D[f(i,j)]is the average value of f in the region coincident with x 0y 0L1YK01mm
¦¦¦¦ Pycoordinate system. A pixel point in PCB with coordinate Py C(iY,ojY) ¦¦w(nx',y)f(xi,yj)YYmY'((X(’, X, Y) mnCi,Yj’) can convert to database coordinate point ()o Ywj)ij¦¦x,y)f(xi,yVL1K1
¦¦¦¦¦¦¦¦¦¦¦¦
¦¦¦¦¦¦¦¦¦¦¦¦
¦¦¦¦
¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦
C(i,j) ¦¦w(x,y)f(xi,yj)
x 0y 0
L1K1
(5)
Figure . User interface experiment result of our AOI
where I = 0, 1…M-1, j = 0, 1,…N-1, and the summation
Vijis taken over the region in the image where w and f
R(i,j)
overlap.D(w)D[f(i,j)]4 Experiment2
V(i,j) ¦¦(w(x,y)w)(f(xi,yj)f(i,j))
x 0y 0LK
2
LK
基于NI视觉开发模块的表面贴装设备的自动光学检测系统:设备和方法W
e performed the proposed algorithm and method with LabWindows/CVI to detect the quality of PCB. Fig. 8 is the software user interface which is designed to detect the SMDs position and rotation. If the rotation is out of the limited range then the component will be labeled as “0”. The minimum matching score, start/end angle can be set. It also allows the user to set the passing rotation range. The inspected image is shown in the image indicator above. The images in white rectangles are SMDs which get high correlation scores to the model image shown at the below of the inspected image and the inspection result is listed in the table.
5 ConclusionT
o detect poor quality of SMD, the application of resample, equalization, geometric matching and pattern matching algorithm were done. And the experimental results show that the proposed devices and methods can complete the AOI task efficaciously. It makes easy of the development of the AOI and improves the accuracy and speed of the AOI.
REFERENCES[1]
H. H. Loh and M. S. Lu, “Printed circuit board inspection using image analysis,” IEEE Trans. on Industry Applications, vol. 35, no. 2, pp. 426-432, 1999.
D. W. Raymond and D. F. Haigh, “Why automatic optical inspection,” Proceeding of the 1997 IEEE International Test Conference, Washingto DC, USA, pp. 1033, Nov. 1-6, 1997.F. Wu, X. Zhang, Y. Kuan, and Z. Z. He, “An AOI algorithm for PCB based on feature extraction,” Proceeding of the 7th Wold Congress on Intelligent Control and Automation, Chongqing, China, pp. 240-247, June 25-27, 2008.
J. Xue, “Machine vision based inspection technology research of surface mounted devices,” Master Dissertation of Southwest Jiaotong University, pp. 11, Dec. 2008.
H. H. Wu, X. M. Zhang, Y. C. Kuang, and S. L. Lu, “A real-time machie vision system for solder paste inspection,” Proceeding of the 2008 IEEE/ASME International Conference on Advanced Instelligent Mechatronics, Xi’an, China, pp. 205-210, July 2-5, 2008.
S. M. Kim, Y. C. Lee, and S. C. Lee, “Vision based automatic inspection system for nuts welded on the support hinge,” SICE-ICASE International Joint Conference 2006, Bexco, Korea, pp. 1508-1512, Oct. 18-21, 2006.
[2]
[3]
[4]
[5]
[6]
作者简介
赵会斌 男,硕士,中国科学院深圳先进技术研究院半导体设备研究室客座学生。2007年于河北大学物理科学与技术学院获学士学位,现为电子科技大学自动化工程学院检测技术与自动化装置专业的硕士研究生。主要从事永磁同步直线电机伺服控制系统和视觉检测系统的研究。程 俊作者简介见本期封2页。3
因篇幅问题不能全部显示,请点此查看更多更全内容
Copyright © 2019- 91gzw.com 版权所有 湘ICP备2023023988号-2
违法及侵权请联系:TEL:199 18 7713 E-MAIL:2724546146@qq.com
本站由北京市万商天勤律师事务所王兴未律师提供法律服务