A survey on mouth modeling and analysis for Sign Language recognition

© 2015 IEEE.Around 70 million Deaf worldwide use Sign Languages (SLs) as their native languages. At the same time, they have limited reading/writing skills in the spoken language. This puts them at a severe disadvantage in many contexts, including education, work, usage of computers and the Internet. Automatic Sign Language Recognition (ASLR) can support the Deaf in many ways, e.g. by enabling the development of systems for Human-Computer Interaction in SL and translation between sign and spoken language. Research in ASLR usually revolves around automatic understanding of manual signs. Recently, ASLR research community has started to appreciate the importance of non-manuals, since they are related to the lexical meaning of a sign, the syntax and the prosody. Nonmanuals include body and head pose, movement of the eyebrows and the eyes, as well as blinks and squints. Arguably, the mouth is one of the most involved parts of the face in non-manuals. Mouth actions related to ASLR can be either mouthings, i.e. visual syllables with the mouth while signing, or non-verbal mouth gestures. Both are very important in ASLR. In this paper, we present the first survey on mouth non-manuals in ASLR. We start by showing why mouth motion is important in SL and the relevant techniques that exist within ASLR. Since limited research has been conducted regarding automatic analysis of mouth motion in the context of ALSR, we proceed by surveying relevant techniques from the areas of automatic mouth expression and visual speech recognition which can be applied to the task. Finally, we conclude by presenting the challenges and potentials of automatic analysis of mouth motion in the context of ASLR

Dipolarization Fronts in the Jovian Magnetotail: Statistical Survey of Ion Intensity Variations Using Juno Observations

Energetic particle acceleration and energization in planetary magnetotails are often associated with dipolarization fronts characterized by a rapid increase of the meridional component of the magnetic field. Despite many studies of dipolarization events in Earth's magnetotail, Jupiter’s magnetotail provides an almost ideal environment to study high-energetic ion acceleration by dipolarization fronts because of its large spatial scales and plasma composition of heavy and light ions. In this study, we focus on the response of different high-energetic ion intensities (H, He, S, and O) to prominent magnetic dipolarization fronts inside the Jovian magnetotail. We investigate if ion energization and acceleration are present in the observations around the identified dipolarization fronts. Therefore, we present a statistical study of 87 dipolarization front signatures, which are identified in the magnetometer data of the Juno spacecraft from July 2016 to July 2021. For the ion intensity analysis, we use the energetic particle observations from the Jupiter Energetic Particle Detector Instrument. Our statistical study reveals that less than half of the identified events are accompanied by an increase of the ion intensities, while most of the other events show no significant change in the ion intensity dynamics. In about 40% of the events located in the dawn sector a significant decrease of the energy spectral index is detected indicating ion acceleration by the dipolarization fronts

Discovery of a transient radiation belt at Saturn

Radiation belts have been detected in situ at five planets. Only at Earth however has any variability in their intensity been heretofore observed, in indirect response to solar eruptions and high altitude nuclear explosions. The Cassini spacecraft's MIMI/LEMMS instrument has now detected systematic radiation belt variability elsewhere. We report three sudden increases in energetic ion intensity around Saturn, in the vicinity of the moons Dione and Tethys, each lasting for several weeks, in response to interplanetary events caused by solar eruptions. However, the intensifications, which could create temporary satellite atmospheres at the aforementioned moons, were sharply restricted outside the orbit of Tethys. Unlike Earth, Saturn has almost unchanging inner ion radiation belts: due to Saturn's near-symmetrical magnetic field, Tethys and Dione inhibit inward radial transport of energetic ions, shielding the planet's main, inner radiation belt from solar wind influences

Magnetospheric considerations for solar system ice state

The current lattice configuration of the water ice on the surfaces of the inner satellites of Jupiter and Saturn is likely shaped by many factors. But laboratory experiments have found that energetic proton irradiation can cause a transition in the structure of pure water ice from crystalline to amorphous. It is not known to what extent this process is competitive with other processes in solar system contexts. For example, surface regions that are rich in water ice may be too warm for this effect to be important, even if the energetic proton bombardment rate is very high. In this paper, we make predictions, based on particle flux levels and other considerations, about where in the magnetospheres of Jupiter and Saturn the ∼MeV proton irradiation mechanism should be most relevant. Our results support the conclusions of Hansen and McCord (2004), who related relative level of radiation on the three outer Galilean satellites to the amorphous ice content within the top 1 mm of surface. We argue here that if magnetospheric effects are considered more carefully, the correlation is even more compelling. Crystalline ice is by far the dominant ice state detected on the inner Saturnian satellites and, as we show here, the flux of bombarding energetic protons onto these bodies is much smaller than at the inner Jovian satellites. Therefore, the ice on the Saturnian satellites also corroborates the correlation

A Two-Player Game of Life

We present a new extension of Conway's game of life for two players, which we call p2life. P2life allows one of two types of token, black or white, to inhabit a cell, and adds competitive elements into the birth and survival rules of the original game. We solve the mean-field equation for p2life and determine by simulation that the asymptotic density of p2life approaches 0.0362.Comment: 7 pages, 3 figure

3D Reconstruction of 'In-the-Wild' Faces in Images and Videos

This is the author accepted manuscript. The final version is available from IEEE via the DOI in this record 3D Morphable Models (3DMMs) are powerful statistical models of 3D facial shape and texture, and are among the state-of-the-art methods for reconstructing facial shape from single images. With the advent of new 3D sensors, many 3D facial datasets have been collected containing both neutral as well as expressive faces. However, all datasets are captured under controlled conditions. Thus, even though powerful 3D facial shape models can be learnt from such data, it is difficult to build statistical texture models that are sufficient to reconstruct faces captured in unconstrained conditions ('in-the-wild'). In this paper, we propose the first 'in-the-wild' 3DMM by combining a statistical model of facial identity and expression shape with an 'in-the-wild' texture model. We show that such an approach allows for the development of a greatly simplified fitting procedure for images and videos, as there is no need to optimise with regards to the illumination parameters. We have collected three new benchmarks that combine 'in-the-wild' images and video with ground truth 3D facial geometry, the first of their kind, and report extensive quantitative evaluations using them that demonstrate our method is state-of-the-art.Engineering and Physical Sciences Research Council (EPSRC

The Menpo benchmark for multi-pose 2D and 3D facial landmark localisation and tracking

In this article, we present the Menpo 2D and Menpo 3D benchmarks, two new datasets for multi-pose 2D and 3D facial landmark localisation and tracking. In contrast to the previous benchmarks such as 300W and 300VW, the proposed benchmarks contain facial images in both semi-frontal and profile pose. We introduce an elaborate semi-automatic methodology for providing high-quality annotations for both the Menpo 2D and Menpo 3D benchmarks. In Menpo 2D benchmark, different visible landmark configurations are designed for semi-frontal and profile faces, thus making the 2D face alignment full-pose. In Menpo 3D benchmark, a united landmark configuration is designed for both semi-frontal and profile faces based on the correspondence with a 3D face model, thus making face alignment not only full-pose but also corresponding to the real-world 3D space. Based on the considerable number of annotated images, we organised Menpo 2D Challenge and Menpo 3D Challenge for face alignment under large pose variations in conjunction with CVPR 2017 and ICCV 2017, respectively. The results of these challenges demonstrate that recent deep learning architectures, when trained with the abundant data, lead to excellent results. We also provide a very simple, yet effective solution, named Cascade Multi-view Hourglass Model, to 2D and 3D face alignment. In our method, we take advantage of all 2D and 3D facial landmark annotations in a joint way. We not only capitalise on the correspondences between the semi-frontal and profile 2D facial landmarks but also employ joint supervision from both 2D and 3D facial landmarks. Finally, we discuss future directions on the topic of face alignment

Effect of heliox breathing on flow limitation in chronic heart failure patients

Patients with chronic heart failure (CHF) exhibit orthopnoea and tidal expiratory flow limitation in the supine position. It is not known whether the flow-limiting segment occurs in the peripheral or central part of the tracheobronchial tree. The location of the flow-limiting segment can be inferred from the effects of heliox (80% helium/20% oxygen) administration. If maximal expiratory flow increases with this low-density mixture, the choke point should be located in the central airways, where the wave-speed mechanism dominates. If the choke point were located in the peripheral airways, where maximal flow is limited by a viscous mechanism, heliox should have no effect on flow limitation and dynamic hyperinflation. Tidal expiratory flow limitation, dynamic hyperinflation and breathing pattern were assessed in 14 stable CHF patients during air and heliox breathing at rest in the sitting and supine position. No patient was flow-limited in the sitting position. In the supine posture, eight patients exhibited tidal expiratory flow limitation on air. Heliox had no effect on flow limitation and dynamic hyperinflation and only minor effects on the breathing pattern. The lack of density dependence of maximal expiratory flow implies that, in CHF patients, the choke point is located in the peripheral airways

Estimating correspondences of deformable objects "in-the-wild"

This is the author accepted manuscript. The final version is available from IEEE via the DOI in this recordDuring the past few years we have witnessed the development of many methodologies for building and fitting Statistical Deformable Models (SDMs). The construction of accurate SDMs requires careful annotation of images with regards to a consistent set of landmarks. However, the manual annotation of a large amount of images is a tedious, laborious and expensive procedure. Furthermore, for several deformable objects, e.g. human body, it is difficult to define a consistent set of landmarks, and, thus, it becomes impossible to train humans in order to accurately annotate a collection of images. Nevertheless, for the majority of objects, it is possible to extract the shape by object segmentation or even by shape drawing. In this paper, we show for the first time, to the best of our knowledge, that it is possible to construct SDMs by putting object shapes in dense correspondence. Such SDMs can be built with much less effort for a large battery of objects. Additionally, we show that, by sampling the dense model, a part-based SDM can be learned with its parts being in correspondence. We employ our framework to develop SDMs of human arms and legs, which can be used for the segmentation of the outline of the human body, as well as to provide better and more consistent annotations for body joints.Engineering and Physical Sciences Research Council (EPSRC)TekesEuropean Community Horizon 202