The Influence of Autofocus Lenses in the Camera Calibration Process

[EN] Camera calibration is a crucial step in robotics and computer vision. Accurate camera parameters are necessary to achieve robust applications. Nowadays, camera calibration process consists of adjusting a set of data to a pin-hole model, assuming that with a reprojection error close to zero, camera parameters are correct. Since all camera parameters are unknown, computed results are considered true. However, the pin-hole model does not represent the camera behavior accurately if the autofocus is considered. Real cameras with autofocus lenses change the focal length slightly to obtain sharp objects in the image, and this feature skews the calibration result if a unique pin-hole model is computed with a constant focal length. In this article, a deep analysis of the camera calibration process is done to detect and strengthen its weaknesses when autofocus lenses are used. To demonstrate that significant errors exist in computed extrinsic parameters, the camera is mounted in a robot arm to know true extrinsic camera parameters with an accuracy under 1 mm. It is also demonstrated that errors in extrinsic camera parameters are compensated with bias in intrinsic camera parameters. Since significant errors exist with autofocus lenses, a modification of the widely accepted camera calibration method using images of a planar template is presented. A pin-hole model with distance-dependent focal length is proposed to improve the calibration process substantially.Ricolfe Viala, C.; Esparza Peidro, A. (2021). The Influence of Autofocus Lenses in the Camera Calibration Process. IEEE Transactions on Instrumentation and Measurement. 70:1-15. https://doi.org/10.1109/TIM.2021.30557931157

Depth-Dependent High Distortion Lens Calibration

[EN] Accurate correction of high distorted images is a very complex problem. Several lens distortion models exist that are adjusted using different techniques. Usually, regardless of the chosen model, a unique distortion model is adjusted to undistort images and the camera-calibration template distance is not considered. Several authors have presented the depth dependency of lens distortion but none of them have treated it with highly distorted images. This paper presents an analysis of the distortion depth dependency in strongly distorted images. The division model that is able to represent high distortion with only one parameter is modified to represent a depth-dependent high distortion lens model. The proposed calibration method obtains more accurate results when compared to existing calibration methods.The Instituto de Automatica e Informatica Industrial (ai2) of the Universitat Politecnica de Valencia has financed the open access fees of this paper.Ricolfe Viala, C.; Esparza Peidro, A. (2020). Depth-Dependent High Distortion Lens Calibration. Sensors. 20(13):1-12. https://doi.org/10.3390/s20133695S1122013Ricolfe-Viala, C., & Sanchez-Salmeron, A.-J. (2010). Lens distortion models evaluation. Applied Optics, 49(30), 5914. doi:10.1364/ao.49.005914Wieneke, B. (2008). Volume self-calibration for 3D particle image velocimetry. Experiments in Fluids, 45(4), 549-556. doi:10.1007/s00348-008-0521-5Magill, A. A. (1955). Variation in Distortion with Magnification*. Journal of the Optical Society of America, 45(3), 148. doi:10.1364/josa.45.000148Fryer, J. G., & Fraser, C. S. (2006). ON THE CALIBRATION OF UNDERWATER CAMERAS. The Photogrammetric Record, 12(67), 73-85. doi:10.1111/j.1477-9730.1986.tb00539.xAlvarez, L., Gómez, L., & Sendra, J. R. (2010). Accurate Depth Dependent Lens Distortion Models: An Application to Planar View Scenarios. Journal of Mathematical Imaging and Vision, 39(1), 75-85. doi:10.1007/s10851-010-0226-2Ricolfe-Viala, C., Sanchez-Salmeron, A.-J., & Martinez-Berti, E. (2011). Accurate calibration with highly distorted images. Applied Optics, 51(1), 89. doi:10.1364/ao.51.000089Ricolfe-Viala, C., & Sánchez-Salmerón, A.-J. (2010). Robust metric calibration of non-linear camera lens distortion. Pattern Recognition, 43(4), 1688-1699. doi:10.1016/j.patcog.2009.10.003Devernay, F., & Faugeras, O. (2001). Straight lines have to be straight. Machine Vision and Applications, 13(1), 14-24. doi:10.1007/pl0001326

Neural networks in virtual reference tuning

This paper discusses the application of the virtual reference tuning (VRT) techniques to tune neural controllers from batch inputoutput data, by particularising nonlinear VRT and suitably computing gradients backpropagating in time. The flexibility of gradient computation with neural networks also allows alternative block diagrams with extra inputs to be considered. The neural approach to VRT in a closed-loop setup is compared to the linear VRFT one in a simulated crane example. © 2011 Elsevier Ltd. All rights reserved.A. Esparza is grateful to the project GVPRE/2008/116 financed by Generalitat Valenciana. The authors are also grateful to the financial support of Grants dpi2008-06731-c02-01, dpi2011-27845-c02-01 (Spanish Government) and prometeo/2008/088 (Generalitat Valenciana).Esparza Peidro, A.; Sala, A.; Albertos Pérez, P. (2011). Neural networks in virtual reference tuning. Engineering Applications of Artificial Intelligence. 24(6):983-995. https://doi.org/10.1016/j.engappai.2011.04.00398399524

Robust fulfillment of constraints in robot visual servoing

[EN] In this work, an approach based on sliding mode ideas is proposed to satisfy constraints in robot visual servoing. In particular, different types of constraints are defined in order to: fulfill the visibility constraints (camera fieldof-view and occlusions) for the image features of the detected object; to avoid exceeding the joint range limits and maximum joint speeds; and to avoid forbidden areas in the robot workspace. Moreover, another task with low-priority is considered to track the target object. The main advantages of the proposed approach are low computational cost, robustness and fully utilization of the allowed space for the constraints. The applicability and effectiveness of the proposed approach is demonstrated by simulation results for a simple 2D case and a complex 3D case study. Furthermore, the feasibility and robustness of the proposed approach is substantiated by experimental results using a conventional 6R industrial manipulator.This work was supported in part by the Spanish Government under grants BES-2010-038486 and Project DPI2013-42302-R, and the Generalitat Valenciana under grants VALi+d APOSTD/2016/044 and BEST/2017/029.Muñoz-Benavent, P.; Gracia Calandin, LI.; Solanes Galbis, JE.; Esparza Peidro, A.; Tornero Montserrat, J. (2018). Robust fulfillment of constraints in robot visual servoing. Control Engineering Practice. 71(1):79-95. https://doi.org/10.1016/j.conengprac.2017.10.017S799571

Asymptotic statistical analysis for model-based control design strategies

In this paper, we generalize existing fundamental limitations on the accuracy of the estimation of dynamic models. In addition, we study the large sample statistical behavior of different estimation-based controller design strategies. In particular, fundamental limitations on the closed-loop performance using a controller obtained by Virtual Reference Feedback Tuning (VRFT) are studied. We also extend our results to more general estimation-based control design strategies. We present numerical examples to show the application of our results. © 2011 Elsevier Ltd. All rights reserved.This work has been partially supported by the project GVPRE/2008/116 financed by Generalitat Valenciana (Spain). This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor Guoxiang Gu under the direction of Editor Torsten Soderstrom.Esparza Peidro, A.; Agüero, JC.; Rojas, CR.; Godoy, BI. (2011). Asymptotic statistical analysis for model-based control design strategies. Automatica. 47(5):1041-1046. https://doi.org/10.1016/j.automatica.2011.01.0581041104647