Real-Time Salient Closed Boundary Tracking via Line Segments Perceptual Grouping

This paper presents a novel real-time method for tracking salient closed boundaries from video image sequences. This method operates on a set of straight line segments that are produced by line detection. The tracking scheme is coherently integrated into a perceptual grouping framework in which the visual tracking problem is tackled by identifying a subset of these line segments and connecting them sequentially to form a closed boundary with the largest saliency and a certain similarity to the previous one. Specifically, we define a new tracking criterion which combines a grouping cost and an area similarity constraint. The proposed criterion makes the resulting boundary tracking more robust to local minima. To achieve real-time tracking performance, we use Delaunay Triangulation to build a graph model with the detected line segments and then reduce the tracking problem to finding the optimal cycle in this graph. This is solved by our newly proposed closed boundary candidates searching algorithm called "Bidirectional Shortest Path (BDSP)". The efficiency and robustness of the proposed method are tested on real video sequences as well as during a robot arm pouring experiment.Comment: 7 pages, 8 figures, The 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017) submission ID 103

Efficient and secure real-time mobile robots cooperation using visual servoing

This paper deals with the challenging problem of navigation in formation of mobiles robots fleet. For that purpose, a secure approach is used based on visual servoing to control velocities (linear and angular) of the multiple robots. To construct our system, we develop the interaction matrix which combines the moments in the image with robots velocities and we estimate the depth between each robot and the targeted object. This is done without any communication between the robots which eliminate the problem of the influence of each robot errors on the whole. For a successful visual servoing, we propose a powerful mechanism to execute safely the robots navigation, exploiting a robot accident reporting system using raspberry Pi3. In addition, in case of problem, a robot accident detection reporting system testbed is used to send an accident notification, in the form of a specifical message. Experimental results are presented using nonholonomic mobiles robots with on-board real time cameras, to show the effectiveness of the proposed method

Direct model based visual tracking and pose estimation using mutual information

International audienceThis paper deals with model-based pose estimation (or camera localization). The model is rendered as a virtual image and we propose a direct approach that takes into account the image as a whole. For this, we consider a sim- ilarity measure, the mutual information. Mutual information is a measure of the quantity of information shared by two signals (or two images in our case). Exploiting this measure allows our method to deal with different image modalities (real and synthetic). Furthermore, it handles occlusions and illu- mination changes. Results with synthetic (benchmark) and real image sequences, with static or mobile camera, demonstrate the robustness of the method and its ability to produce stable and precise pose estimations

Conception and development of a mobile mixed reality medium for environment-related storytelling – a novel approach to virtual heritage

Das Ziel dieser Arbeit ist die Entwicklung eines neuartigen Virtual Heritage Mediums das User mit Hilfe von interaktiven Mixed-Reality Umgebungen und raumbezogenem Erzählen nicht nur räumlich mitten in eine Geschichte hineinversetzt, sondern auch aktiv in diese einbezieht. Dies wird erreicht, indem das Videobild eines getrackten Smartphones mit perspektivisch stimmigen Echtzeit-3D Inhalten überlagert wird. Der User kann diese Mixed-Reality Umgebung erkunden, die Handlungen von 3D Charakteren beobachten sowie mit ihnen und virtuellen Artefakten interagieren. Dieses Medium bietet folglich die Möglichkeit mediale Geschichten in echten Räumen zu erzählen sowie ein immersives und user-involvierendes Medienerlebnis. Diese Arbeit wird den Einsatz dieses Mediums speziell für Kulturvermittlungszwecke fokussieren. Diese Arbeit wird zunächst die technischen Anforderungen und Umsetzungsmöglichkeiten dieses Vorhabens mittels Unity für iPhone / iOS 7 untersuchen. Die Belegung der Ergebnisse erfolgt durch die anschließende Realisierung eines Prototypen.The goal of this thesis is to develop a novel type of virtual heritage medium that utilises the combined immersive and engaging potentials of interactive mixed reality environments and spatial narratives. Concretely, this is achieved through depth-sensitive compositing of real-time 3D content into the live-video of a tracked smartphone. The user can explore this mixed reality environment, watch the actions of staged 3D characters as well as interact with them and virtual artifacts. This medium would therefore provide possibilities for telling stories in direct context with existing environments along with an immersive and engaging media experience. This work will mainly focus on how this medium can be used as an edutainment medium in sites of cultural heritage. This thesis will focus on establishing the technical requirements and realisation possibilities for implementation in Unity on iPhone 5 / iOS 7. Subsequently, a prototype is implemented in order to prove the research results

Automated surgical light positioning system using thermal-imaging and optical techniques

Thesis (MEng)--Stellenbosch University, 2019.ENGLISH ABSTRACT: Surgical light systems (SLS) are used to ensure optimal light conditions during surgical procedures. At present, these light sources are adjusted manually by the surgeon or other operating room (OR) personnel. Manual adjustment of the SLS is problematic due to the necessity for a sterile environment. Surgeons must either adjust the SLS themselves, or communicate their requirements so that the OR assistants can position the lights to ensure optimal surgical conditions. Other complications with current SLS include mechanical problems, collisions, inaccessibility and illumination issues. It would be beneficial if the SLS could be automated to illuminate the wound without input from the surgeon. Therefore, the aim of this project was to test whether it is possible to identify a heat source (simulating a surgical wound), track this heat source in real time, and adjust a laser indicator (simulating a surgical light beam) mounted on an articulating assembly (analogous to an SLS). A system was developed that used an algorithm that identified and tracked a heat source and communicated to an automated articulating assembly to keep the laser indicator pointed at the heat source. The heat source was identified using thermal cameras and tracked using stereo optical cameras in three-dimensional space. The tracking accuracy and the manipulation accuracy were tested, and the results demonstrated that the combination of optical and thermal cameras with stereo image-processing techniques could be used to identify and track a heat source. This could further be used to guide an articulated assembly to keep a light beam pointed at the heat source with good accuracy. Therefore, this technology will contribute towards achieving full automation of SLS in the future. Following from the conclusions of this thesis, aspects have been identified and recommended for future research to achieve full automation and solve all SLS complications in the future.AFRIKAANSE OPSOMMING: irurgiese ligstelsels (CLS’e) word gebruik om optimale ligtoestande tydens chirurgiese prosedures te verseker. Tans word dit deur die chirurg of ander personeel in die operasiesaal (OS) verstel. Om die CLS met die hand te verstel, is problematies weens die vereistes van 'n steriele omgewing. Chirurge wat nie die CLS self verstel nie, moet hul vereistes so kommunikeer sodat die verstelling deur die OS-assistente voldoende is om optimale chirurgiese omstandighede te verseker. Ander komplikasies met die huidige CLS sluit in: meganiese probleme, botsings, ontoeganklikheid en beligtingskwessies. Dit sal dus voordelig wees as die CLS geoutomatiseer kan word om die wond te verlig sonder insette van die chirurg. Die doel van hierdie projek was dus om te toets of dit moontlik is om 'n hittebron (wat die chirurgiese wond voorstel) te identifiseer, hierdie hittebron te volg en 'n laseraanwyser (simulasie van die chirurgiese ligstraal) aan te pas met 'n ge-artikulerende arm (voorstelling van 'n CLS). 'n Stelsel is ontwikkel wat 'n algoritme gebruik om 'n hittebron te identifiseer en te volg. Die inligting is dan oorgedra na 'n gemotoriseerde artikulerende arm, om die laseraanwyser na die hittebron te rig. Termiese kameras het die hittebron geïdentifiseer terwyl stereo-optiese kameras gebruik is om die hittebron te volg soos wat dit beweeg het. Die akkuraatheid van die arm se beheer is getoets en die resultate het getoon dat die kombinasie van optiese en termiese kameras met stereobeeld-verwerkingstegnieke gebruik kan word om 'n hittebron te identifiseer en te volg. Dit kan verder gebruik word om 'n artikulerende arm aan te pas om 'n ligstraal te rig na die hittebron, met voldoende akkuraatheid. As gevolg van die akkuraatheid sal hierdie tegnologie in die toekoms bydra tot die volle outomatisering van CLS. Na aanleiding van die gevolgtrekkings van hierdie tesis, is daar sekere aspekte geïdentifiseer en aanbevelings gemaak vir toekomstige navorsing om volle outomatisering te bereik en alle SLS-komplikasies in die toekoms op te los

Real-time Hybrid Tracking using Edge and Texture Information

This paper proposes a real-time, robust and effective tracking framework for visual servoing applications. The algorithm is based on the fusion of visual cues and on the estimation of a transformation (either a homography or a 3D pose). The parameters of this transformation are estimated using a non-linear minimization of a unique criterion that integrates information both on the texture and the edges of the tracked object. The proposed tracker is more robust and performs well in conditions where methods based on a single cue fail. The framework has been tested for 2D object motion estimation and pose computation. The method presented in this paper has been validated on several video sequences as well as in visual servoing experiments considering various objects. Results show the method to be robust to occlusions or textured backgrounds and suitable for visual servoing applications

Real-time Hybrid Tracking using Edge and Texture Information

This paper proposes a real-time, robust and effective tracking framework for visual servoing applications. The algorithm is based on the fusion of visual cues and on the estimation of a transformation (either a homography or a 3D pose). The parameters of this transformation are estimated using a non-linear mini-mization of a unique criterion that integrates information both on the texture and the edges of the tracked object. The proposed tracker is more robust and performs well in conditions where methods based on a single cue fail. The framework has been tested for 2D object motion estimation and pose computation. The method presented in this paper has been validated on several video sequences as well as in visual servoing experiments considering various objects. Results show the method to be robust to occlusions or textured backgrounds and suitable for visual servoing applications