In order to compute camera viewpoints during sensor planning, Tarabanis et al. (1991) present a group of feature detectability constraints which include six nonlinear inequalities in an eight-dimensional real space. It is difficult to compute robust viewpoints which satisfy all feature detectability constraints. In this paper, the viewpoint setting is formulated as an unconstrained optimization problem. Then a tree annealing algorithm, which is a general-purpose technique for finding minima of functions of continuously-valued variables, is applied to solve this nonlinear multiconstraint optimization problem. Our results show that the technique is quite effective to get robust viewpoints even in the presence of considerable amounts of noise

Allen, Peter K.

Yao, Yulin

Columbia University Academic Commons

Computing Robust Viewpoints with Multi-constraints Using Tree Annealing Yulin Yao* Peter Allen+ Abstract In order to compute camera viewpoints during sensor planning, Tarabanis et a1 present a group of feature detectability constraints which include six nonlinear inequalities in an eight-dimensional real space. It is difficult to compute robust viewpoints which satisfy all feature detectability Constraints. In this paper, the viewpoint setting is formulated as an unconstrainted optimization problem. Then a tree annealing algorithm, which is a general-purpose techniques for finding minima of functions of continuously-valued variables, is applied to solve this nonlinear multiconstraint optimization problem. Our results show that the technique is quite effective to get robust viewpoints even in the presence of considerable amounts of noise. I. INTRODUCTION Sensor planning involves determining strategies with which sensor parameter values can be found that will achieve a sensing task with a certain degree of satisfaction. It is a fairly new area of computer vision but has received considerable interest recently [a] [6] [7] [8]. Tarabanis, Tsai and Allen [7] have been developing a vision planning system, MVP (Machine Vision Planner), that automatically determines vision sensor parameter values so that the task requirements are satisfied. Compared to  the iterative techniques employed in the SRI system [a] and other sensor planning systems, the main contribution of the MVP system is that it provides closed-form solutions to  the individual task constraints and determines a set of sensor parameters which characterize the general viewing configurations. However, in the MVP system, it is difficult to  compute robust viewpoints which satisfy all feature detectability constraints simultaneously. As Tarabanis pointed out in [6], techniques that combine the admissible domain of individual constraints in order to  determine optimal solutions still need to be investigated. In this paper, the viewpoint setting is formulated as an unconstrainted optimization problem, then a tree annealing (TA) technique [l] which is one of simulated annealing algorithms [4] that  can handle continuously-valued variables, is applied to solve the multiple nonlinear constraints problem. Our results show that the technique is quite effective to  get robust viewpoints even in the presence of considerable amounts of noise. 11. CONSTRAINTS FOR FEATURE DETECTABILITY In the MVP system, the configurations of viewing parameters that are planned include the three posi- tional degrees of freedom of the sensor ~:(x,y,z), the two orientational degree of freedom (pan and tilt *The author was with the Columbia University. He is now with the Dow Jones & Company, Inc. 'The author is with the Columbia University 0-7803-2559-1/95 $4.00 0 1995 IEEE 993 angles) described by a unit vector v' along the viewing direction and the three optical parameters (the back nodal point to  image plane distance d, and focal length f a n d  the aperture of the lens U ) .  Thus, planning is done in eight-dimensional space and a point in this space is defined as a generalized viewpoint p(r;, 5, cl, f, a). Using knowledge from geometry and optics, each task constraint in the MVP is char- acterized by an analytical relationship [6]. As a result, the locus of generalized viewpoints that satisfies the resolution, depth-of-field and field-of-view constraints separately is expressed by a relationship of the form: gz(rG, G, d,  f, U )  2 0, specifically: .Depth of field: for farthest point: gl = D1 - Il(r5 - r'f) * GI/ 2 0, for closest point: g2 = Il(r5 - r:) .i?ll - D2 2 0 where T? is the position vector of the farthest feature vertex from the front nodal point of the lens along the viewing direction; r', is the position vector of the closest feature vertex from the front nodal point of the lens along the viewing direction; r; is the position vector of the front nodal point of the lens and where e is the minimum of the horizontal and vertical sensor element spacings, d is the back nodal point to  image plane distance, f i s  the focal length and a is the aperture of the lens. .Field of view: 93 = ( T Z  - T ; j ) .  v'- C O S ( f ) l l T k  - 2 0 where r k  = rc  - R,G, r2 is the position vector of the center of the sphere of radius Rj circumscribing the object features, R, = Rj/(sin((1!/2)),  R j  is the radius of the sphere circumscribing all the object features, (I! is the field of view angle and is given by a = 2 -t~n-~(I,;,/2d), I,,, is the minimum dimension of the sensor plane, and all other variables are as defined above.  resolution for edge feature AB; where r z ,  r i , ,  rj& are the position vectors of the front nodal point of the lens and vertices of the feature edge i to  be resolved; e; is the unit vector along to  feature edge AB; to  be resolved; I, w are the lengths of the feature to  be resolved in object and image space, respectively. All other variables are as defined above. Unit vector: g5 = llG112 - 1 = 0 It should be noted that there is a resolution constraint for each edge feature that is to  be resolved, while for other constraints, there is a single relationship for all features. 994 111. TREE ANNEALING Tree annealing [1] [3] is an extension of the familiar Metropolis algorithm [5]  of simulated annealing, but handles continuously valued variables in a natural way. In this section, we briefly introduce the tree annealing (TA) method based on [l] [3]. Following the definitions and notations of [l] [3], let us assume we are searching for the minimum of some function f(x) where the d-dimensional vector x has continuously valued elements. Furthermore, we assume a finite search space S c Rd. A k-d tree in which each level of the tree represents a binary partition of one particular degree of freedom (DOF) is used. Each node may thus be interpreted as representing a hyperrectangle, and its children therefore represent the smaller hyperrectangles resulting from dividing the parent along one particular DOF. Let a vector x be the current sample. At each node, two numbers are stored, n L  and n R ,  representing how many times in the past that an acceptable point has been found in the left and right subtrees, respectively. The TA algorithm works as follows for a finite set S: 1. Growing and searching the tree: (a) The tree is initiated by simply creating the root node, and choosing a point at random with uniform probability from the entire search space. That point becomes the first accepted point. Two daughter nodes are created, corresponding to a division of the search space in half along the first DOF. The n L  and n R  are both initialized to  1 for the root node. (b) Begin at the root and, at each node, choose either the left or right child randomly with ~ r o b a b i l i t ~  nL"+"nR or nL respectively. Descend the tree to  its leaves making left-right decisions in this way. (c) Upon reaching a leaf, generate the point y at random (uniformly) from the subspace defined by the leaf. Compare x and y and make an accept/reject decision on x (see step 2). If y is accepted, replace x by y as the current sample; if y is rejected, x remains the current sample. (d) If y was accepted, split the current leaf (containing y), and create two new daughter nodes, thus making more resolution available at this node if it is ever explored again. (e) Ascend the tree from the current sample to the root, updating n L  and n R  at  each node. 2. Accept/reject decision. Accept the point y as the new estimate with probability where the probabilities p are Gibbs, (i.e., with form p(X) o( e s p ( - p ) )  and g(y) is computed from the path of the descent down the tree by where pi = &, I represents the node visited at level I, a1 represents n L  or n R ,  according to which direction was chosen ar each 2. Similarly, bl represents the n of the direction not chosen. 3. The annealing schedule is very similar to the suggestion in [4]: T t rT where r = 1 -% and dS is a small positive constant and C, is a term easily related to the variance of the energy. 995 IV. PROBLEM FORMULATION The decisive criteria of a computed viewpoint are its robustness and stability. The measure is used to assess the goodness of a solution with respect to  the value of each constraint relationship 9;. This is appropriate since a large positive value of g; indicates that a constraint is satisfied comfortably, a small positive value indicates marginal satisfaction, while inadmissible solutions give rise to a negative value. We want to search a globally admissible eight-dimensional viewpoint which is near the center of the admissible domain and far from the bounded hypersurfaces described by the constraints. Such a generalized viewpoint is desirable, since it is robust in the event of inaccuracy. Similarly, the measure for the visibility constraint is also formulated. For this purpose, the minimum distance, d,, from the viewpoint to the polyhedron describing the visibility region is chosen: $3 = Ad,, where +d, or -d, depending on whether the point is inside or outside the visibility volume respectively. The optimization function is taken to be a weighted sum of the above component criteria, each of which characterize the quality of the solution with respect to  each associated requirement separately. If we take two edge features then we will have two resolution constraints gda and 946, each of them with respect to an edge feature. Thus, the optimization function is written as: maxobj = Ea; eg;, (i = 1,2,3,4~,4b,6) ,orminobj  = --Ea; -g ; ,  ( i  = 1,2,3,4a,4b,6) i S i S Subject to: where a; are weights and s is a point of the finite eight-dimensional space S. g; 2 0; i = 1, 2, 3, 4a, 4b, 6; and g5 = 0 We convert the above set of constraints into a penalty function. For each g;, ( i  = 1,2,3,4a,4b, S), the penalty term ezp(  -,&go) is assigned, where p; is a positive real number which represents the degree of penalty (penalty factor). It is appropriate since, for g; < 0, the value of the eap(-P;g;) will be (exponentially) very large; for g, 2 0, the value of the ezp(-P;g;) will be small. For g5, the penalty term ezp(/?s[ggI) is assigned, where ,& is a positive penalty factor. In our experiments, we choose same penalty factor (=1) for each constraint g;, (i =1, 2 ,  3, 4a,4b). It is also appropriate since, for g5 = 0, esp(P5lgsl) = 1; for g5 # 0, the value of ezp(Pslg51) will be (exponentially) very large. In our experiments, we choose P.5 = 1000, which is larger than any other penalty factor, in order to  get more accurate unit vector. So we know that the penalty function will appropriately penalize any infeasible/inadmissible constraint. Thus the constrained problem is reformulated as an unconstrainted optimization: minobj = - SES where i = 1, 2, 3 ,  4a, 4b, 6. We use the TA algorithms described in the previous section to  solve this unconstrainted optimization. V. EXPERIMENTAL RESULTS As part of the MVP system, we have implemented the vision planning algorithms that are given in Section I1 and 111 using the TA algorithm. In the experiments, we will demonstrate the effectiveness of applying the technique to compute the robust general viewpoints with multiple feature detectability constraints. The features to be observed are the two edges ( U  and b) of an enclosed cube. In our experiments, we choose the parameters as in [6]: r>= (0,0,0), which coincides with the origin of object coordinates system; c = 13.5 microns, I = 2.54 mm, w = 0.02112 mm, I,,, = 6.5 mm, where 996 c is the minimum of the horizontal and vertical sensor element spacings, I, w and Imin are defined in Section 11. The values of the lens aperture a and the intrinsic focal length f a r e  chosen a priori (f = 12.5 mm and a = f/16 = 0.78125 mm) and thus, values for the remaining imaging space parameters v' and d are computed. All measured units are expressed in millimeters in the experiments. The values of the weights a; in the objective function are taken to be: a1 = 0.1, a2 = a3= 0.01, a4a = Q4b= 1000 . The values of the weights p; in the penalty function are taken to be: PI = P 2  = ,& = = 1000 (explained in previous section). An initial viewpoint V, that is chosen to start the optimization, and the corresponding camera viewpoint Vj that is computed by the TA algorithm, is listed in Tables 1. For V, ,  the gl and g3 constraints are violated (refer to the first column in the Tables 2). All feature detectability of constraints of the computed viewpoint Vj that is determined by the TA algorithm are satisfied. = p46 = Ps= 1, Table 1: The initial and final generalized viewpoints V, and Vj (unit: mm) X I Y I  z I v(1) I v(2) I v(3) I f I d I  a V,  11 80.0 I -5.0 I 160.0 I -0.58 I 0.2 I -0.8 I 12.5 I 14.0 I f / l 6  I Vf 11 124.34 I -4.44 1 207.60 I -0.61 I -0.03 I -0.79 I 12.5 I 13.13 I f/16 I In order to check the robustness and stability of the computed viewpoints, the camera is approached to the object along the view direction to see whether the constraints are still satisfied. Let PI be the computed viewpoint, P2 be a point at which the camera approaches the object and C be the center of the sphere of circumscribing the object features. The approaching scale factor is defined as follows: IPlP21 scale factor = the projection of PI-C on the view directionPI?. The value of constraints with different scale factors are given in Tables 3 and 4. We find all constraints but gz and 93 are satisfied. The constraint $2 (depth of field for closest point) is isolated when the distance between the viewpoint and the center of the object is less than the certain value (D2);  and the constrain g3 (focus of view) is isolated when the angle between the view direction PI32 and P2-C is greater than certain value. These values are determined by the intrinsic parameters of camera (see the definition of 9 2  and g3 in section 11). The interesting result - the maximum reachable viewpoint V,,,, which still satisfy simultaneously all constraints, from the current computed viewpoint Vf respectively along a reverse view direction - is given in the third column in Table 2. Another factor that will affect the stability and robustness of the computed viewpoint is the presence of noise, for example, the slight perturbation of manipulator on which the camera is mounted (we can imagine that the manipulator is teleoperated and many conditions around it are unpredictable). In order to check the stability and robustness of viewpoint planning in the presence of noise, independent random noise with 10 %, 20 % and 30 % are added to each component of the position vector r: and the orientation vector v'. The values of constraints under the different noise levels is listed in Tables 2. We can find from the table that aU constraints are still satisfied in these cases, that is, the computed viewpoint Vj are stable and robust even in the presence of noise. Thus we can conclude that the viewpoint Vi which is computed by the TA algorithm is robust and stable. ACKNOWLEDGMENTS The authors would like to thank Steven Abrams and Paul Michelman for their valuable help and comments. 997 Table 2: The values of constraints with the different scale factors and noises forV,, Vf and V,,,(unit: ") References [l] Griff Bilbro and Wesley Snyder. Optimization of functions with many minima. IEEE Transactions on  Systems, Man and Cybernetic, 21(4):840-849, 1991. [a] C.K. Cowan and A. Bergman. Determining the camera and light source location for a visual task. In Proceedings of IEEE International Conference on Robotics and Automation, pages 509-514, 1989. [3] Younsik Han, Griff Bilbro, and Wesley Snyder. Pose determination using tree annealing. In Proceed- ings of I E E E  International Conference on  Robotics and Automation, Cincinnati, OH, 1990. [4] S. Kirkpatrick, C. Gelatt, and M. Vecchi. Optimization by simulated annealing. Science, 220(4598):671-680,1983. [5] N.  Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller. Equation of state calculations by fast computing machines. Jouranl of Chemical Physics, 21(6):1087-1091,1953. [6] K. Tarabanis. Automatic synthesis of camera views that satisfy feature detectability constraints. PhD thesis, Computer Science Department, Columbia University, New York, NY, 1991. [7] K. Tarabanis, R. Y. Tsai, and P. Allen. Automated sensor planning for robotic vision tasks. In Proceedings of IEEE International Conference on  Robotics and Automation, 1991. [8] S. Yi, R. M. Haralick, and L. G. Shapiro. Automatic sensor and light source positioning for machine vision. In Proceedings of IEEE International Conference on  Pattern Recognition, pages 55-59, 1990. 998 

Computing robust viewpoints with multi-constraints using tree annealing

https://academiccommons.columbia.edu/doi/10.7916/D8GH9T6K/download

Computing robust viewpoints with multi-constraints using tree annealing

Abstract

Similar works

Full text

Available Versions

Columbia University Academic Commons