Image based visual servoing using bitangent points applied to planar shape alignment

We present visual servoing strategies based on bitangents for aligning planar shapes. In order to acquire bitangents we use convex-hull of a curve. Bitangent points are employed in the construction of a feature vector to be used in visual control. Experimental results obtained on a 7 DOF Mitsubishi PA10 robot, verifies the proposed method

Positioning and trajectory following tasks in microsystems using model free visual servoing

In this paper, we explore model free visual servoing algorithms by experimentally evaluating their performances for various tasks performed on a microassembly workstation developed in our lab. Model free or so called uncalibrated visual servoing does not need the system calibration (microscope-camera-micromanipulator) and the model of the observed scene. It is robust to parameter changes and disturbances. We tested its performance in point-to-point positioning and various trajectory following tasks. Experimental results validate the utility of model free visual servoing in microassembly tasks

A fast algorithm for vision-based hand gesture recognition for robot control

We propose a fast algorithm for automatically recognizing a limited set of gestures from hand images for a robot control application. Hand gesture recognition is a challenging problem in its general form. We consider a fixed set of manual commands and a reasonably structured environment, and develop a simple, yet effective, procedure for gesture recognition. Our approach contains steps for segmenting the hand region, locating the fingers, and finally classifying the gesture. The algorithm is invariant to translation, rotation, and scale of the hand. We demonstrate the effectiveness of the technique on real imagery

İki noktada teğetler kullanarak görüntü tabanlı görsel geri beslemeli kontrol ile düzlemsel şekil hizalama

Bu bildiride, düzlemsel şekilleri hizalamak için iki noktada teğetlerden (bitangent) faydalanan görsel geri beslemeli kontrol stratejileri sunulmuştur. Bitangentları elde etmek ic¸in bir eğrinin dışbükey zarfından (convex-hull) yararlanılmıştır. İmge nitelik vektörü bitangent noktalarından oluşturularak görsel kontrolde kullanılmıştır. 7 serbestlik dereceli Mitsubishi PA10 robotu üzerinde gerçekleştirilen deneyler önerilen metodun geçerliliğini göstermiştir

Model-based vs. model-free visual servoing: A Performance evaluation in microsystems

In this paper, model-based and model-free image based visual servoing (VS) approaches are implemented on a microassembly workstation, and their regulation and tracking performances are evaluated. A precise image based VS relies on computation of the image jacobian. In the model-based visual servoing, the image Jacobian is computed via calibrating the optical system. Precisely calibrated model based VS promises better positioning and tracking performance than the model-free approach. However, in the model-free approach, optical system calibration is not required due to the dynamic Jacobian estimation, thus it has the advantage of adapting to the different operating modes

Model free visual servoing in macro and micro domain robotic applications

This thesis explores model free visual servoing algorithms by experimentally evaluating their performances for various tasks performed both in macro and micro domains. Model free or so called uncalibrated visual servoing does not need the system (vision system + robotic system) calibration and the model of the observed scene, since it provides an online estimation of the composite (image + robot) Jacobian. It is robust to parameter changes and disturbances. A model free visual servoing scheme is tested on a 7 DOF Mitsubishi PA10 robotic arm and on a microassembly workstation which is developed in our lab. In macro domain, a new approach for planar shape alignment is presented. The alignment task is performed based on bitangent points which are acquired using convex-hull of a curve. Both calibrated and uncalibrated visual servoing schemes are employed and compared. Furthermore, model free visual servoing is used for various trajectory following tasks such as square, circle, sine etc. and these reference trajectories are generated by a linear interpolator which produces midway targets along them. Model free visual servoing can provide more exibility in microsystems, since the calibration of the optical system is a tedious and error prone process, and recalibration is required at each focusing level of the optical system. Therefore, micropositioning and three di erent trajectory following tasks are also performed in micro world. Experimental results validate the utility of model free visual servoing algorithms in both domains

Lip segmentation using adaptive color space training

In audio-visual speech recognition (AVSR), it is beneficial to use lip boundary information in addition to texture-dependent features. In this paper, we propose an automatic lip segmentation method that can be used in AVSR systems. The algorithm consists of the following steps: face detection, lip corners extraction, adaptive color space training for lip and non-lip regions using Gaussian mixture models (GMMs), and curve evolution using level-set formulation based on region and image gradients fields. Region-based fields are obtained using adapted GMM likelihoods. We have tested the proposed algorithm on a database (SU-TAV) of 100 facial images and obtained objective performance results by comparing automatic lip segmentations with hand-marked ground truth segmentations. Experimental results are promising and much work has to be done to improve the robustness of the proposed method

Lattice-based shape tracking and servoing of elastic objects

In this paper, we propose a general unified tracking-servoing approach for controlling the shape of elastic deformable objects using robotic arms. Our approach works by forming a lattice around the object, binding the object to the lattice, and tracking and servoing the lattice instead of the object. This makes our approach have full 3D control over deformable objects of any general form (linear, thin-shell, volumetric). Furthermore, it decouples the runtime complexity of the approach from the objects' geometric complexity. Our approach is based on the As-Rigid-As-Possible (ARAP) deformation model. It requires no mechanical parameter of the object to be known and can drive the object toward desired shapes through large deformations. The inputs to our approach are the point cloud of the object's surface in its rest shape and the point cloud captured by a 3D camera in each frame. Overall, our approach is more broadly applicable than existing approaches. We validate the efficiency of our approach through numerous experiments with deformable objects of various shapes and materials (paper, rubber, plastic, foam). Experiment videos are available on the project website: https://sites.google.com/view/tracking-servoing-approach.Comment: This is the arXiv version of an article published in IEEE Transactions on Robotics. Please cite the accepted version: M. Shetab-Bushehri, M. Aranda, Y. Mezouar and E. \"Ozg\"ur, "Lattice-based Shape Tracking and Servoing of Elastic Objects," in IEEE Transactions on Robotics, doi: 10.1109/TRO.2023.333159

Laser-assisted Cellular Electrophysiology Measurement System

Patch-clamp technique is the gold standard for cellular electrophysiological measurements, which is capable of measuring single ion transport events across the cell membrane. However, the measurement possesses significant complexity, and it requires a high level of expertise, while its experimental throughput is nevertheless considerably low. Here, we suggest and experimentally demonstrate a laser-assisted method for performing cellular electrophysiological measurements. Femtosecond laser pulses, coupled to an optical microscope, are used to form a sub-micrometer hole on a thin polymer membrane separating two electrodes, where a nearby cell is subsequently placed onto the hole by negative pressure. Afterwards, the cell is punctured using subsequent laser exposure, revealing the cell membrane over the hole for electrophysiological recording. This system could be used to increase the output amount of the electrophysiological measurements substantially.Comment: 4 pages, 4 figure

ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library

Tracking the 3D shape of a deforming object using only monocular 2D vision is a challenging problem. This is because one should (i) infer the 3D shape from a 2D image, which is a severely underconstrained problem, and (ii) implement the whole solution pipeline in real-time. The pipeline typically requires feature detection and matching, mismatch filtering, 3D shape inference and feature tracking algorithms. We propose ROBUSfT, a conventional pipeline based on a template containing the object's rest shape, texturemap and deformation law. ROBUSfT is ready-to-use, wide-baseline, capable of handling large deformations, fast up to 30 fps, free of training, and robust against partial occlusions and discontinuity in video frames. It outperforms the state-of-the-art methods in challenging datasets. ROBUSfT is implemented as a publicly available C++ library and we provide a tutorial on how to use it in https://github.com/mrshetab/ROBUSfTComment: This is the arXiv version of an article published in Image and Vision Computing. Please cite the accepted version: M. Shetab-Bushehri, M. Aranda, E. Ozgur, Y. Mezouar and Adrien Bartoli "ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library," in Image and Vision Computing, doi: 10.1016/j.imavis.2023.10486