Visual units and confusion modelling for automatic lip-reading

Automatic lip-reading (ALR) is a challenging task because the visual speech signal is known to be missing some important information, such as voicing. We propose an approach to ALR that acknowledges that this information is missing but assumes that it is substituted or deleted in a systematic way that can be modelled. We describe a system that learns such a model and then incorporates it into decoding, which is realised as a cascade of weighted finite-state transducers. Our results show a small but statistically significant improvement in recognition accuracy. We also investigate the issue of suitable visual units for ALR, and show that visemes are sub-optimal, not but because they introduce lexical ambiguity, but because the reduction in modelling units entailed by their use reduces accuracy

Enhancing Approximations for Regular Reachability Analysis

This paper introduces two mechanisms for computing over-approximations of sets of reachable states, with the aim of ensuring termination of state-space exploration. The first mechanism consists in over-approximating the automata representing reachable sets by merging some of their states with respect to simple syntactic criteria, or a combination of such criteria. The second approximation mechanism consists in manipulating an auxiliary automaton when applying a transducer representing the transition relation to an automaton encoding the initial states. In addition, for the second mechanism we propose a new approach to refine the approximations depending on a property of interest. The proposals are evaluated on examples of mutual exclusion protocols

Tree Regular Model Checking for Lattice-Based Automata

Tree Regular Model Checking (TRMC) is the name of a family of techniques for analyzing infinite-state systems in which states are represented by terms, and sets of states by Tree Automata (TA). The central problem in TRMC is to decide whether a set of bad states is reachable. The problem of computing a TA representing (an over- approximation of) the set of reachable states is undecidable, but efficient solutions based on completion or iteration of tree transducers exist. Unfortunately, the TRMC framework is unable to efficiently capture both the complex structure of a system and of some of its features. As an example, for JAVA programs, the structure of a term is mainly exploited to capture the structure of a state of the system. On the counter part, integers of the java programs have to be encoded with Peano numbers, which means that any algebraic operation is potentially represented by thousands of applications of rewriting rules. In this paper, we propose Lattice Tree Automata (LTAs), an extended version of tree automata whose leaves are equipped with lattices. LTAs allow us to represent possibly infinite sets of interpreted terms. Such terms are capable to represent complex domains and related operations in an efficient manner. We also extend classical Boolean operations to LTAs. Finally, as a major contribution, we introduce a new completion-based algorithm for computing the possibly infinite set of reachable interpreted terms in a finite amount of time.Comment: Technical repor