Deteksi Kedipan dengan Metode CNN dan Percentage of Eyelid Closure (PERCLOS)

Pengembangan teknologi mengenai face detection dan eyes detection melaju sangat pesat, sehingga peneliti berlomba-lomba meneliti metode dan algoritma yang optimal untuk pengaplikasian di kehidupan sehari-hari, mulai dari pengamanan biometrics sampai identifikasi wajah secara au- tomasi. Di tugas akhir ini diusulkan penggunaan metode Convo- lutional Neural Network (CNN) dan Percentage of Eyelid Closure (PERCLOS) pada pendeteksi kedipan mata. Sistem dibangun menggunakan webcam personal computer sebagai kamera dan mendeteksi secara real-time. Sistem dapat mengenali kondisi ketika mata tertutup atau mata terbuka dan menentukan lebar bukaan mata dengan menggunakan Eye Aspect Ratio (EAR) serta dapat mengestimasi skor tatapan dengan menggunakan Percentage of Eyelid Closure (PERCLOS). Sistem dapat menge- nali wajah dari objek bukan wajah dengan jarak pendeteksian optimal antara 40-70 cm. Model hasil training dapat mengk- lasifikasikan kondisi mata terbuka dan mata tertutup dengan menggunakan Convolutional Neural Network dengan arstitektur yang memiliki 3 layer mendapatkan hasil accuracy 98% dan loss 2.05

Human Recognition and Identification: identification of Persons in the social context based on image processing

The aim of this Bachelor’s Thesis is to provide the therapeutic robot TUK with an identification software, so that it is able to recognize people in its environment. Moreover, it is intended that TUK can identify who it is interacting with, and thus adapt its behaviour depending on the situation. In the first part, the theoretical principles that have been used during the development of the software are presented. Several kinds of image processing techniques as well as classification algorithms are explained. In the second part, the implementation is shown step by step in order to give an overview of the whole system. Finally, the results obtained during several tests are presented and discussed. In conclusion, several guidelines for tackling some of the challenges are proposed, setting a possible way for further work

Fast and Accurate Algorithm for Eye Localization for Gaze Tracking in Low Resolution Images

Iris centre localization in low-resolution visible images is a challenging problem in computer vision community due to noise, shadows, occlusions, pose variations, eye blinks, etc. This paper proposes an efficient method for determining iris centre in low-resolution images in the visible spectrum. Even low-cost consumer-grade webcams can be used for gaze tracking without any additional hardware. A two-stage algorithm is proposed for iris centre localization. The proposed method uses geometrical characteristics of the eye. In the first stage, a fast convolution based approach is used for obtaining the coarse location of iris centre (IC). The IC location is further refined in the second stage using boundary tracing and ellipse fitting. The algorithm has been evaluated in public databases like BioID, Gi4E and is found to outperform the state of the art methods.Comment: 12 pages, 10 figures, IET Computer Vision, 201

Computer Vision Based Early Intraocular Pressure Assessment From Frontal Eye Images

Intraocular Pressure (IOP) in general, refers to the pressure in the eyes. Gradual increase of IOP and high IOP are conditions or symptoms that may lead to certain diseases such as glaucoma, and therefore, must be closely monitored. While the pressure in the eye increases, different parts of the eye may become affected until the eye parts are damaged. An effective way to prevent rise in eye pressure is by early detection. Exiting IOP monitoring tools include eye tests at clinical facilities and computer-aided techniques from fundus and optic nerves images. In this work, a new computer vision-based smart healthcare framework is presented to evaluate the intraocular pressure risk from frontal eye images early-on. The framework determines the status of IOP by analyzing frontal eye images using image processing and machine learning techniques. A database of images from the Princess Basma Hospital was used in this work. The database contains 400 eye images; 200 images with normal IOP and 200 high eye pressure case images. This study proposes novel features for IOP determination from two experiments. The first experiment extracts the sclera using circular hough transform, after which four features are extracted from the whole sclera. These features are mean redness level, red area percentage, contour area and contour height. The pupil/iris diameter ratio feature is also extracted from the frontal eye image after a series of pre-processing techniques. The second experiment extracts the sclera and iris segment using a fully conventional neural network technique, after which six features are extracted from only part of the segmented sclera and iris. The features include mean redness level, red area percentage, contour area, contour distance and contour angle along with the pupil/iris diameter ratio. Once the features are extracted, classification techniques are applied in order to train and test the images and features to obtain the status of the patients in terms of eye pressure. For the first experiment, neural network and support vector machine algorithms were adopted in order to detect the status of intraocular pressure. The second experiment adopted support vector machine and decision tree algorithms to detect the status of intraocular pressure. For both experiments, the framework detects the status of IOP (normal or high IOP) with high accuracies. This computer vison-based approach produces evidence of the relationship between the extracted frontal eye image features and IOP, which has not been previously investigated through automated image processing and machine learning techniques from frontal eye images

Automated Vision-Based High Intraocular Pressure Detection Using Frontal Eye Images

Glaucoma, the silent thief of vision, is mostly caused by the gradual increase of pressure in the eye which is known as intraocular pressure (IOP). An effective way to prevent the rise in eye pressure is by early detection. Prior computer vision-based work regarding IOP relies on fundus images of the optic nerves. This paper provides a novel vision-based framework to help in the initial IOP screening using only frontal eye images. The framework first introduces the utilization of a fully convolutional neural (FCN) network on frontal eye images for sclera and iris segmentation. Using these extracted areas, six features that include mean redness level of the sclera, red area percentage, Pupil/Iris diameter ratio, and three sclera contour features (distance, area, and angle) are computed. A database of images from the Princess Basma Hospital is used in this work, containing 400 facial images; 200 cases with normal IOP; and 200 cases with high IOP. Once the features are extracted, two classifiers (support vector machine and decision tree) are applied to obtain the status of the patients in terms of IOP (normal or high). The overall accuracy of the proposed framework is over 97.75% using the decision tree. The novelties and contributions of this work include introducing a fully convolutional network architecture for eye sclera segmentation, in addition to scientifically correlating the frontal eye view (image) with IOP by introducing new sclera contour features that have not been previously introduced in the literature from frontal eye images for IOP status determination.https://doi.org/10.1109/JTEHM.2019.291553

Robust and real-time hand detection and tracking in monocular video

In recent years, personal computing devices such as laptops, tablets and smartphones have become ubiquitous. Moreover, intelligent sensors are being integrated into many consumer devices such as eyeglasses, wristwatches and smart televisions. With the advent of touchscreen technology, a new human-computer interaction (HCI) paradigm arose that allows users to interface with their device in an intuitive manner. Using simple gestures, such as swipe or pinch movements, a touchscreen can be used to directly interact with a virtual environment. Nevertheless, touchscreens still form a physical barrier between the virtual interface and the real world. An increasingly popular field of research that tries to overcome this limitation, is video based gesture recognition, hand detection and hand tracking. Gesture based interaction allows the user to directly interact with the computer in a natural manner by exploring a virtual reality using nothing but his own body language. In this dissertation, we investigate how robust hand detection and tracking can be accomplished under real-time constraints. In the context of human-computer interaction, real-time is defined as both low latency and low complexity, such that a complete video frame can be processed before the next one becomes available. Furthermore, for practical applications, the algorithms should be robust to illumination changes, camera motion, and cluttered backgrounds in the scene. Finally, the system should be able to initialize automatically, and to detect and recover from tracking failure. We study a wide variety of existing algorithms, and propose significant improvements and novel methods to build a complete detection and tracking system that meets these requirements. Hand detection, hand tracking and hand segmentation are related yet technically different challenges. Whereas detection deals with finding an object in a static image, tracking considers temporal information and is used to track the position of an object over time, throughout a video sequence. Hand segmentation is the task of estimating the hand contour, thereby separating the object from its background. Detection of hands in individual video frames allows us to automatically initialize our tracking algorithm, and to detect and recover from tracking failure. Human hands are highly articulated objects, consisting of finger parts that are connected with joints. As a result, the appearance of a hand can vary greatly, depending on the assumed hand pose. Traditional detection algorithms often assume that the appearance of the object of interest can be described using a rigid model and therefore can not be used to robustly detect human hands. Therefore, we developed an algorithm that detects hands by exploiting their articulated nature. Instead of resorting to a template based approach, we probabilistically model the spatial relations between different hand parts, and the centroid of the hand. Detecting hand parts, such as fingertips, is much easier than detecting a complete hand. Based on our model of the spatial configuration of hand parts, the detected parts can be used to obtain an estimate of the complete hand's position. To comply with the real-time constraints, we developed techniques to speed-up the process by efficiently discarding unimportant information in the image. Experimental results show that our method is competitive with the state-of-the-art in object detection while providing a reduction in computational complexity with a factor 1 000. Furthermore, we showed that our algorithm can also be used to detect other articulated objects such as persons or animals and is therefore not restricted to the task of hand detection. Once a hand has been detected, a tracking algorithm can be used to continuously track its position in time. We developed a probabilistic tracking method that can cope with uncertainty caused by image noise, incorrect detections, changing illumination, and camera motion. Furthermore, our tracking system automatically determines the number of hands in the scene, and can cope with hands entering or leaving the video canvas. We introduced several novel techniques that greatly increase tracking robustness, and that can also be applied in other domains than hand tracking. To achieve real-time processing, we investigated several techniques to reduce the search space of the problem, and deliberately employ methods that are easily parallelized on modern hardware. Experimental results indicate that our methods outperform the state-of-the-art in hand tracking, while providing a much lower computational complexity. One of the methods used by our probabilistic tracking algorithm, is optical flow estimation. Optical flow is defined as a 2D vector field describing the apparent velocities of objects in a 3D scene, projected onto the image plane. Optical flow is known to be used by many insects and birds to visually track objects and to estimate their ego-motion. However, most optical flow estimation methods described in literature are either too slow to be used in real-time applications, or are not robust to illumination changes and fast motion. We therefore developed an optical flow algorithm that can cope with large displacements, and that is illumination independent. Furthermore, we introduce a regularization technique that ensures a smooth flow-field. This regularization scheme effectively reduces the number of noisy and incorrect flow-vector estimates, while maintaining the ability to handle motion discontinuities caused by object boundaries in the scene. The above methods are combined into a hand tracking framework which can be used for interactive applications in unconstrained environments. To demonstrate the possibilities of gesture based human-computer interaction, we developed a new type of computer display. This display is completely transparent, allowing multiple users to perform collaborative tasks while maintaining eye contact. Furthermore, our display produces an image that seems to float in thin air, such that users can touch the virtual image with their hands. This floating imaging display has been showcased on several national and international events and tradeshows. The research that is described in this dissertation has been evaluated thoroughly by comparing detection and tracking results with those obtained by state-of-the-art algorithms. These comparisons show that the proposed methods outperform most algorithms in terms of accuracy, while achieving a much lower computational complexity, resulting in a real-time implementation. Results are discussed in depth at the end of each chapter. This research further resulted in an international journal publication; a second journal paper that has been submitted and is under review at the time of writing this dissertation; nine international conference publications; a national conference publication; a commercial license agreement concerning the research results; two hardware prototypes of a new type of computer display; and a software demonstrator

Eye Status Based on Eyelid Detection: A Driver Assistance System

Fatigue and driver drowsiness monitoring is an important subject for designing driver assistance systems. The measurement of eye closure is a fundamental step for driver awareness detection. We propose a method which is based on eyelid detection and the measurement of the distance between the eyelids. First, the face and the eyes of the driver are localized. After extracting the eye region, the proposed algorithm detects eyelids and computes the percentage of eye closure. Experimental results are performed on the BioID database. Our comparisons show that the proposed method outperforms state-of-the-art methods

Automatic Pipeline Surveillance Air-Vehicle

This thesis presents the developments of a vision-based system for aerial pipeline Right-of-Way surveillance using optical/Infrared sensors mounted on Unmanned Aerial Vehicles (UAV). The aim of research is to develop a highly automated, on-board system for detecting and following the pipelines; while simultaneously detecting any third-party interference. The proposed approach of using a UAV platform could potentially reduce the cost of monitoring and surveying pipelines when compared to manned aircraft. The main contributions of this thesis are the development of the image-analysis algorithms, the overall system architecture and validation of in hardware based on scaled down Test environment. To evaluate the performance of the system, the algorithms were coded using Python programming language. A small-scale test-rig of the pipeline structure, as well as expected third-party interference, was setup to simulate the operational environment and capture/record data for the algorithm testing and validation. The pipeline endpoints are identified by transforming the 16-bits depth data of the explored environment into 3D point clouds world coordinates. Then, using the Random Sample Consensus (RANSAC) approach, the foreground and background are separated based on the transformed 3D point cloud to extract the plane that corresponds to the ground. Simultaneously, the boundaries of the explored environment are detected based on the 16-bit depth data using a canny detector. Following that, these boundaries were filtered out, after being transformed into a 3D point cloud, based on the real height of the pipeline for fast and accurate measurements using a Euclidean distance of each boundary point, relative to the plane of the ground extracted previously. The filtered boundaries were used to detect the straight lines of the object boundary (Hough lines), once transformed into 16-bit depth data, using a Hough transform method. The pipeline is verified by estimating a centre line segment, using a 3D point cloud of each pair of the Hough line segments, (transformed into 3D). Then, the corresponding linearity of the pipeline points cloud is filtered within the width of the pipeline using Euclidean distance in the foreground point cloud. Then, the segment length of the detected centre line is enhanced to match the exact pipeline segment by extending it along the filtered point cloud of the pipeline. The third-party interference is detected based on four parameters, namely: foreground depth data; pipeline depth data; pipeline endpoints location in the 3D point cloud; and Right-of-Way distance. The techniques include detection, classification, and localization algorithms. Finally, a waypoints-based navigation system was implemented for the air- vehicle to fly over the course waypoints that were generated online by a heading angle demand to follow the pipeline structure in real-time based on the online identification of the pipeline endpoints relative to a camera frame