Humans have degrees-of-freedom (DOF) of varying bandwidths and one casually observes that we coordinate these DOF while visually tracking. This suggests that joint interplay aids tracking performance. In a control scheme we call partitioning, both image and kinematic data are used to visually-servo a 5-DOF robot by defining a joint-coupling among the rotational and translational DOF. Analysis of simulations and experiments reveal that a robot's fast bandwidth joints physically serve as lead compensators when coupled to slower joints thus reducing tracking lag

Allen, Peter K.

Oh, Paul Y.

Columbia University Academic Commons

Proceedings of the 2000 IEEE International Conference on Robotics 8 Automation San Francisco, CA April 2000 Joint-coupled Compensation Effects in Visually Servoed Tracking Paul Y .  Oh and Peter K. Allen Dept. of Computer Science, Columbia University, New York, NY 10027 * Abstract Humans have degrees-of- freedom (DOF) of varying bandwidths and one casually observes that we coor- dinate these DOF while visually tracking. This sug- gests that joint interplay aids tracking performance. I n  a control scheme we call partitioning, both image and kinematic data are used to visually-servo a 5-DOF robot by  defining a joint-coupling among the rotational and translational DOF. Analysis of simulations and experiments reveal that a robot’s fast bandwidth joints physically serve as lead compensators when coupled to  slower joints thus reducing tracking lag. 1 Introduction Compensators are often added to the control law to makeup for performance deficiencies. For example, a common objective in eye-in-hand tracking is to  robot- ically servo the camera such that its image plane re- mains centered over the moving target. But, joint bandwidth and saturation limits not permitting, the camera will lag behind the target. One can then mathcmatically design feedforward 141 or lead com- pensators to add phase and cancel out camera lag. Skeet shooting is a good example of lead compensa- tion. Often a marksmen will aim ahead of the target, taking its flight path, speed and distance into account. He thereby physically adds phase to the target’s bear- ing angle. People display interesting behaviors which also sug- gest that we employ compensation. For example the eyes lead (i.e. move before) the neck, when we track a moving target. One possible explanation is that the eyes, which have fast response times, are physically compensating for the neck’s slower motions. Beyond the mathematical abstractions of compensator design, joints with fast bandwidths can physically act as lead compensators when their kinematic encoder data is ‘This work was supported in part by an ONR/DARPA MURI award ONR N00014-95-1-0601 and NSF grants CDA- 9625374 and EIA-97-29844. Figure 1: PTU and camera mounted on gantry end- effector tracks targets like grippers in workcell added to the visual-servoing loop. This will be illus- trated in this paper. Our particular vision interests are in visually moni- toring a large 3.6 x 6.4 x 1 m3 assembly workcell with a 5-dof robot (Fig. 1). This robot is a custom-built 3-dof Cartesian gantry mounted with a Directed Per- ceptions 2-dof pan-tilt-unit (PTU) and a camcra on its end-effector. For our initial tracking efforts, we designed a pose regulator [6], [9], 121. The control law in such designs rely exclusively on image data to update camera motions using an image Jacobian to maintain a desired camera-to-target pose. This ini- tial effort met with several limitations associated with joint bandwidth and saturation. Target translations forced gantry motions which are marked by largc in- ertial loads and slow responses. Oftentimes the tar- get would accelerate past the camera’s field-of-view (FOV) before the gantry got up to speed thus lcisillg visual contact. By contrast, the small, lightweight PTU responds quickly and easily. Another problcm was that abrupt target starts and stops forced large gantry accelerations that generated camera end-point vibrations. Kalman filtering [IO], [l] was implemented to improve robustness but it requires a priori knowl- edge of the target’s motion dynamics, which in gen- eral, isn’t always available. To overcome these limitations we designed a control 0-7803-5886-4/00/$10.00@ 2000 IEEE 2094 law we call partitioning that couples joints [7],  [8]. The control law uses both image and kinematic joint data. The pan and tilt degrees-of-freedom (DOF) are visually servoed using image data while gantry trans- lations are kinematically servoed using pan and tilt angle data. In this paper we present an analysis of partitioning to illustrate that joint coupling improves visually servoed tracking. We found that fast band- width joints, when visually servoed, physically act as lead compensators. Section 2 presents the partition- ing control architecture and linearized input-output relationships. Simulations and experiments in the fre- quency domain reveal phase characteristics and are given in Section 3. Conclusions follow in Section 4. 2 Coupling DOF Tracking can be defined as the task of centering the camera’s image plane over a moving target. For a horizontally translating target, tracking requires only visually servoing one DOF; the camera can be panned or translated. In either case, tracking is achieved by controlling camera velocity in proportion to pixel po- sition changes of the target’s image centroid. Tracking in this manner is known as piloting [3] or steering [4]. Tracking by camera pan requires mapping pixel posi- tions in the image space to target bearing angles in the task space. The radial distance between the camera and target, as measured along the optical axis, factors into this mapping. This distance cannot be measured using a single image centroid, but it can be approx- imated if the bearing angle is small and the initial camera-to-target distance is knowr:. The net effect is lower pan tracking performance as the target horizon- tally translates away from the camera. Tracking by camera translation is much simplier. The camera-to- target distance remains constant when the target hor- izontally translates in a plane parallel to the camera’s image plane and simple perspective is used for the image-to-task space mapping. Camera translations however invoke a gantry DOF which is marked by large inertias. In other words, the gantry’s long time response limits tracking performance to slow moving targets. Another way to track is to invoke both pan and trans- lational DOF and design a control law that exploits the advantages each has to offer. The PTU has a short time response and can be visually servoed to keep a fast moving target in view, whereas the gantry has the ability to transport the camera anywhere in the workcell. Thus better pan tracking performance can Figure 2 :  Coupled pan-gantry block diagram. be achieved if the gantry simultaneously translates to reduce the camera-to-target bearing angle. To real- ize simultaneous pan and translation, a joint-coupling can be defined in the underlying control law, and is illustrated in the discrete-time block diagram (Fig. 2). Both sample instants IC and units are given for addi- tional clarity. The coupling is achieved with two feedback loops and the target position input, xt [m], results in both pan angle, 0; [rad], and gantry translation, z, [m], out- puts. Pan velocity e,, is visually servoed using pixel differences s[k]  - s* and gain A, and the gantry trans- lates using camera pan angle differences 0; - e* and gain A,. s[k]  = (u ,v )  is the center-of-gravity pixel position of the target’s image centroid with U and U being the horizontal and vertical pixel coordinates re- spectively. s* = (u*,u*) is a user-defined set point, with s* = (0,O) being the pixel position of the cam- era’s image plane center. To visually servo camera pan, U must be mapped into a camera-to-target bearing angle, e t .  This mapping is modeled using Fig. 3 and described as follows. If the target is not centered in the camera’s imagc plane, then a non-zero angle difference e exists between Bt and camera angle 0,. Assuming small angles, 6 can be approximated by xt XC z * + L  z * + L  e = et -ec  -- where z* and L are the initial camera-to-target dis- tance and camera’s radius of rotation respectively. 4 is the angle the target makes with respect to the lens 2095 L* : lens-to-target distance these outputs in meters and radians. Target transla- tion input zt [m] thus yields the following closed-loop transfer functions: Figure 3: Relevant notation used for tracking by cam- era pan center and using perspective where f is the lens focal length in pixels. Combining these two equations yields f f U = -  z* ( 2 A) = $ z * + L ) B  or the following input-output relationships (3) These two relationships are represented as block el- ements before and after the top-left adder in Fig. 2. The term is a camera-to-target distance de- pendent gain and models a camera’s lens. The net ef- fect is that tracking performance by visually servoed pan is limited to small target bearing angles. The gantry DOF is invoked by kinematically servoing translations in response to camera pan angle differ- ences 8; - 8’. 8’ is a user-defined setpoint camera orientation and without loss of generality, 8* is set to zero. The net effect is that the constant gain, A, ac- tuates camera translations at  speeds proportional to its pan angle. Both the gantry and PTU stepper motors are modeled as unit delays, l / z .  Forward-rectangular integration, T / ( z  - l ) ,  with sample time T is used to yield the gantry position, zc[k] and pan angle, 8,[k], in steps. Unit conversion constants, K1, K2, K3 and K4 yield CE = t * z 4  - 22*z3 + (z* + (z* + L)A,T)z2 -(z* + L)A,Tz + (z* + L)T2A,Ag Superscript c in ‘Go and ‘G, denotes “coupled“ and subscripts 8 and g describe pan and gantry respec- tively. The net effect of the coupling in Fig. 2 is that tracking behavior (i.e pan and gantry servo action) depend on two gains A, and A,. This is because both ‘Go and ‘G, have the same characteristic equation (CE) and the closed-loop pole locations depend on the product A,A,. Tracking behavior can also bc seen by the fact that if A, = 0 then there is no camera translation (since ‘G , = 0 )  and all tracking is done exclusively by panning. The gain dependence on pan and gantry responses can be seen more clearly in the ensuing root locus diagrams and step responses. 2.1 Root Locus & Time Response Reexpressing (5) and (6) into root locus forms yields two plots (Figs. 4 and 5). The former plot is of vary- ing A, and fixed A, = 0.5 and the latter is of varying A, and fixed A, = 1.0. T was set a t  0.12 s in light of the PTU’s serial latency. New gantry velocity updates are issued at  this loop time and its serial latency (6 ms) is comparatively small. These two plots comple- ment each other. Fig. 4 reveals double poles a t  both z = 0 and 1, and a pair of zeros a t  0.0641 and 0.936. The loci breaks away at  z = 0.5 and pan response will be more oscillatory as A, increases above 1.958 with instability a t  A, > 6.242. Fig. 5 reveals yields poles a t  z = 0, 1, 0.786 and 0.214 and no zeros. Gantry over- shoot will occur for A, > 0.35 and settling time will increase. As A, is increased, oscillations will increase with unstable response occurring for A, > 2.548. The net effect of kinematic coupling pan and gantry DOF together is graphically shown in Fig. 6. Step re- sponse experiments were videotaped and a sequence of image stills are shown in Fig. 7. Fig. 8 is the ex- perimentally obtained (dash) step input responses for 8, and z, with A, = 1.0, A, = 0.5. Simulation results 2096 ' l.s/- 9 . 9 1  = t  ! / '  c- , , i - . 3  . L. ! I 3 9 1.5 -1.51 I -A..- 1 ' -1.5 - . 9  - . 3  Real z ec [final] = 0 ec w+m] x,[Iinal] = 0 Xc [k+m] - Options) A C 0 E F 6 H I J Y L N P 9 R 1 U U X 2 ?=help x c  lkl Figure 4: Gff root locus with A, = 1.0 - pan gain = 1 B -- . z = E 911 r r - 9k I -1 5 1  I -1 5 - 9  - 3  3 9 1 5  Real z Options) A C 0 E F 6 H I J Y L N P 9 A 1 U U X 2 ? = k l p  Figure 5: Gf: root locus with A, = 0.5 Figure 6: Coupled pan-gantry step response P." gain: 1.8 gantry pain: 8.5 ' Options) R B  C D E F G  H J I L )I P P I U X 2 ?=help Figure 8: Coupled pan-gantry step response. Exper- imental (dash) and simulations (solid) shown. A, = 1.0 and A, = 0.5. Pan accelerations result in gantry translations. (solid) using equations (5) and (6) match these results closely. 3 Coupling Effects Rise time is often used as a performance metric and is defined as the time it takes the system to go from 10 to 90 percent of its steady-state value. The rise times of visually servoing a gantry DOF (A, = 0.5) was experimentally measured to be 4.29 s. Visually servoing the pan DOF (A, = 1.0)) yielded a 1.13 s rise time. Coupling the pan and gantry DOF at these gains yielded a 2.50 s rise time. Visually servoing the gantry results in sluggish re- sponse because its large inertial load forces using a low gain. Partitioning introduces coupling and takes advantage of the pan% fast rise time to translate the gantry quickly. As mentioned in the introduction, the 2097 Figure 7: Coupled pan-gantry step response: The target is a robot hand mounted on a Puma end-effector (left). To create the step input, the hand first translates 0.1 m and then the camera is allowed to servo. The camera simulataneously pans and translates (middle) under partitioning to the final camera pose (right). pan DOF serves as a lead compensator and is analo- gous to skeet shooting. The pan adds positive phase when aiming a t  the target. This can be seen analyti- cally from the Bode phase and magnitude plots of (5) and (6 )  given in Fig. 9. The pan's Bode plot reveals a resonant peak at 0.189 Hz (12.95 dB). Below this value, there is phase lead with amplified gain. At higher frequencies, gain is attenuated, filtering high-frequency signals. At low frequencies (e.g. 0.02 Hz) there is little pan and larger gantry motion. In other words, most of the tracking action is done by the gantry. At higher frequencies (e.g. 0.80 Hz) the reverse is true with pan motions being more prevalent. In mid-frequencies, both pan and gantry motions contribute towards tracking. The net effect is that the pan introduces phase lead to compensate for the lower gantry bandwidth. Fig. 10 shows this pan phase lead (+55.88" and 6.37 dB gain) for a 0.1 Hz sinusoidal input. Peak-to-peak pixel error in response to a sinusoidal input is another performance metric. A small pixel error means that the target is centered in the camera's image plane. Fig. 11 illustrates the improved phase and peak-to-peak pixel error that joint-coupling of- fers. Tracking by visually servoing the gantry alone (A, = 1.0) results in a peak error of 133 pixels (5.3 cm). Tracking by coupling the pan and gantry DOF (A, = 1.0, A, = 1.0) results in a peak error of 73 pixels (2.9 cm). By increasing &, even smaller pixel errors are possible (Fig. 12). For A, = 1.0 and A, = 2.0, the peak error is 33 pixels (1.3 cm). 4 Conclusions Casual observance reveals that humans coordinate their DOF while tracking and the resulting motions Cuupled yas-gantry respome t o  E . 1  Hz. E. l - l -krye t  translation pan gain:  1 . E '  ,,,.-,,pan Iradl gantry gain:  6 . 5 '  t ''i --- ! :. i >  p 21, ; >< i \. t i  u t  . 4 /  ~L : : . 8 - -  0 5 10 IS 28 2s l i n e  I s 1  Options) I( B C D E F 6 H J K 1 I! P Q 1 U X 2 ?=help Figure 10: Coupled pan-gantry response to a 0.1 Hz, 0.1 m amplitude sinusoidal target translation zt (thick solid line). Both experimental (dash) and simulated (solid) responses shown. A, = 1.0 and A, = 0.5. suggest a joint-coupling. The partitioned scheme de- fines a joint-coupling in the control law that takes advantage of both image and kinematic data. Sim- ulations and experimcns reveal that coupling smaller peak-to-peak pixel errors, and hence improved track- ing performance. Like skeet shooting, the partitioned pan DOF adds phase to the gantry's dynamics due to its faster response and physically acts like a lead compensator. This suggests that improved lag per- formance can be achieved by kinematically coupling a robot's faster joints to its slower joints under a parti- tioned scheme. The analysis reveals that partitioned tracking performance depends on two gains which can be tuned using standard root locus techniques. We are currently exploring linear quadratic cost functions as an alternative approach for gain tuning. Such an a p  proach would prescribe the amount of pan and gantry control effort and ensure both stability and optimal 2098 Figure 9: Bode plots of coupled pan-gantry (left to right): pan magnitude and phase, gantry magnitude and phase for A, = A, = 1.0. Both Matlab-generated (solid) and experimental (dot) results shown. The pan’s magnitude peaks at 0.189 Hz with a 12.95 dB gain and 0” phase angle. The gantry 3 dB down occurs at 0.293 Hz and -166” phase angle. IO . L7Y Figure 11: Pixel errors for both gantry-only (A, = 1.0) and coupled pan-gantry (A, = A, = 1.0) systems to a 0.1 Hz, 0.1 m amplitude target translation Q. Note how coupling improves phase and has less peak- to-peak pixel error. tracking performance. References [l] Bensalah, F., Chaumette, F., “Compensation of Abrupt Motion Changes in Target Tracking by Visual Servoing”, Proc IROS ’95 p 181, Pittsburgh, 8/95. [2] Chaumette, F., Rives, P., Espiau, B., “Positioning of a Robot with respect to an Object, Tracking it and Estimating its Velocity by Visual Servoing” , IEEE Int Conf Robotics F4 Automation,, p 2248, 1991. [3] Chaumette, F., Santos, A., “Tracking a Moving Object by Visual Servoing”, Proc. of 12th World Congress IFAC, Vol9 pp 409-414, Sydney, July 1993. [4] Corke, P. I., “Dynamics of Visual Control”, in IEEE Int Conf Robotics €4 Automation, Workshop M-5, San Diego, 5/94. [5] Corke, P., “Design, Delay and Performance in Gaze Control: Engineering and Biological Approaches”, in Figure 12: As pan gain A, increases, the pixel error decreases but can yield gantry end-point vibrations due to larger accelerations The Confluence of Vision and Control, Kriegman et al, Eds. Springer-Verlag pp. 146-158, 1998. [6] Feddema, J.T., Lee, G.C.S., “Weighted Selection of Image Features for Resolved Rate Visual Feedback Control”, IEEE Trans Robotics F4 Automation, V7 N 1  2/91. [7] Oh, P.Y., Allen, P.K., “Design of a Partitioned Vi- sual Feedback Controller”, IEEE Int Conf Robotics €4 Automation, Leuven, Belgium, pp 1360 5/98. [8] Oh, P.Y., Allen, P.K., “Performance of a Partitioned Visual Feedback Controller”, IEEE Int Conf Robotics F4 Automation, Detroit, Michigan 5/99. [9] Papanikolopoulos, N.P., Khosla, P.K., “Adaptive Robotic Visual Tracking: Theory and Experiments”, IEEE Trans Automatic Control” V38 N3 3/93. [lo] Wilson, W.J., “Visual Servo Control of Robots Using Kalman Filter Estimates of Robot Pose Relative to Work-pieces” , in Visual Servoing, World Scientific, Koichi Hashimoto Editor, 1993 2099 

Joint-coupled compensation effects in visually servoed tracking

https://academiccommons.columbia.edu/doi/10.7916/D8902D10/download

Joint-coupled compensation effects in visually servoed tracking

Abstract

Similar works

Full text

Available Versions

Columbia University Academic Commons