An original framework for understanding human actions and body language by using deep neural networks

The evolution of both fields of Computer Vision (CV) and Artificial Neural Networks (ANNs) has allowed the development of efficient automatic systems for the analysis of people's behaviour. By studying hand movements it is possible to recognize gestures, often used by people to communicate information in a non-verbal way. These gestures can also be used to control or interact with devices without physically touching them. In particular, sign language and semaphoric hand gestures are the two foremost areas of interest due to their importance in Human-Human Communication (HHC) and Human-Computer Interaction (HCI), respectively. While the processing of body movements play a key role in the action recognition and affective computing fields. The former is essential to understand how people act in an environment, while the latter tries to interpret people's emotions based on their poses and movements; both are essential tasks in many computer vision applications, including event recognition, and video surveillance. In this Ph.D. thesis, an original framework for understanding Actions and body language is presented. The framework is composed of three main modules: in the first one, a Long Short Term Memory Recurrent Neural Networks (LSTM-RNNs) based method for the Recognition of Sign Language and Semaphoric Hand Gestures is proposed; the second module presents a solution based on 2D skeleton and two-branch stacked LSTM-RNNs for action recognition in video sequences; finally, in the last module, a solution for basic non-acted emotion recognition by using 3D skeleton and Deep Neural Networks (DNNs) is provided. The performances of RNN-LSTMs are explored in depth, due to their ability to model the long term contextual information of temporal sequences, making them suitable for analysing body movements. All the modules were tested by using challenging datasets, well known in the state of the art, showing remarkable results compared to the current literature methods

Lisp, Jazz, Aikido -- Three Expressions of a Single Essence

The relation between Science (what we can explain) and Art (what we can't) has long been acknowledged and while every science contains an artistic part, every art form also needs a bit of science. Among all scientific disciplines, programming holds a special place for two reasons. First, the artistic part is not only undeniable but also essential. Second, and much like in a purely artistic discipline, the act of programming is driven partly by the notion of aesthetics: the pleasure we have in creating beautiful things. Even though the importance of aesthetics in the act of programming is now unquestioned, more could still be written on the subject. The field called "psychology of programming" focuses on the cognitive aspects of the activity, with the goal of improving the productivity of programmers. While many scientists have emphasized their concern for aesthetics and the impact it has on their activity, few computer scientists have actually written about their thought process while programming. What makes us like or dislike such and such language or paradigm? Why do we shape our programs the way we do? By answering these questions from the angle of aesthetics, we may be able to shed some new light on the art of programming. Starting from the assumption that aesthetics is an inherently transversal dimension, it should be possible for every programmer to find the same aesthetic driving force in every creative activity they undertake, not just programming, and in doing so, get deeper insight on why and how they do things the way they do. On the other hand, because our aesthetic sensitivities are so personal, all we can really do is relate our own experiences and share it with others, in the hope that it will inspire them to do the same. My personal life has been revolving around three major creative activities, of equal importance: programming in Lisp, playing Jazz music, and practicing Aikido. But why so many of them, why so different ones, and why these specifically? By introspecting my personal aesthetic sensitivities, I eventually realized that my tastes in the scientific, artistic, and physical domains are all motivated by the same driving forces, hence unifying Lisp, Jazz, and Aikido as three expressions of a single essence, not so different after all. Lisp, Jazz, and Aikido are governed by a limited set of rules which remain simple and unobtrusive. Conforming to them is a pleasure. Because Lisp, Jazz, and Aikido are inherently introspective disciplines, they also invite you to transgress the rules in order to find your own. Breaking the rules is fun. Finally, if Lisp, Jazz, and Aikido unify so many paradigms, styles, or techniques, it is not by mere accumulation but because they live at the meta-level and let you reinvent them. Working at the meta-level is an enlightening experience. Understand your aesthetic sensitivities and you may gain considerable insight on your own psychology of programming. Mine is perhaps common to most lispers. Perhaps also common to other programming communities, but that, is for the reader to decide..

Liberating Composition from Language Dictatorship

Historically, programming languages have been—although benevolent—dictators: fixing a lot of semantics into built-in language constructs. Over the years, (some) programming languages have freed the programmers from restrictions to use only built-in libraries, built-in data types, or built-in type checking rules. Even though, arguably, such freedom could lead to anarchy, or people shooting themselves in the foot, the contrary tends to be the case: a language that does not allow for extensibility, is depriving software engineers from the ability to construct proper abstractions and to structure software in the most optimal way. Instead, the software becomes less structured and maintainable than would be possible if the software engineer could express the behavior of the program with the most appropriate abstractions. The new idea proposed by this paper is to move composition from built-in language constructs to programmable, first-class abstractions in the language. As an emerging result, we present the Co-op concept of a language, which shows that it is possible with a relatively simple model to express a wide range of compositions as first-class concepts

An Exploration of the Roles Values Play in Design Decision-Making

The paper presents the findings of a study into design decision-making and specifically the use of values during design decision-making. It briefly describes the development of a taxonomy of values used in design decision-making developed from a series of pilot interviews, protocol analysis and focus groups. This was necessary because although the values agenda is not new, previous studies were found to have gaps, or did not reflect the current state of play. From this more in-depth case studies were carried out to explore the influence of values in design decision-making. Eight designers were asked to design a lectern out of sustainable materials. They were given one day to complete the project. For one hour during the day they were asked to ‘talk aloud’ while being videoed, also known as concurrent verbalisation and protocol analysis. They also took part in a 40 minute retrospective interview about their design work, at the end of the day. One designer was asked to complete a ten day design project in order to verify the results against a longitudinal project. They also took part in a 40 minute retrospective interview at the end of the project. The paper presents some of the rich data collected during the study. And illustrates the ability to research the role of values in design decision-making. The data generated shows values driving many of the decisions designers make including the way in which they cognitively organise their design activity and through which they can reduce avenues of enquiry. Keywords: Design Decision-Making, Knowledge, Skills, Values, Empirical Evidence, Research Methods</p

AO-OpenCom: an AO-Middleware architecture supporting flexible dynamic reconfiguration

Middleware has emerged as a key technology in the construction of distributed systems. As a consequence, middleware is increasingly required to be highly modular and configurable, to support separation of concerns between services, and, crucially, to support dynamic reconfiguration: i.e. to be capable of being changed while running. Aspect-oriented middleware is a promising technology for the realisation of distributed reconfiguration in distributed systems. In this paper we propose an aspect-oriented middleware platform called AO-OpenCom that builds AO-based reconfiguration on top of a dynamic component approach to middleware system composition. The goal is to support extremely flexible dynamic reconfiguration that can be applied at all levels of the system and uniformly across the distributed environment. We evaluate our platform by the capability in meeting flexible reconfiguration and the impact of these overheads

Free composition instead of language dictatorship

Historically, programming languages have been—benevolent—dictators: reducing all possible semantics to specific ones offered by a few built-in language constructs. Over the years, some programming languages have freed the programmers from the restrictions to use only built-in libraries, built-in data types, and builtin type-checking rules. Even though—arguably—such freedom could lead to anarchy, or people shooting themselves in the foot, the contrary tends to be the case: a language that does not allow for extensibility is depriving software engineers of the ability to construct proper abstractions and to structure software in the most optimal way. Therefore the software becomes less structured and maintainable than would be possible if the software engineer could express the behavior of the program with the most appropriate abstractions. The idea proposed by this paper is to move composition from built-in language constructs to programmable, first-class abstractions in a language. We discuss several prototypes of the Co-op language, which show that it is possible, with a relatively simple model, to express a wide range of compositions as first-class concepts

Open Programming Language Interpreters

Context: This paper presents the concept of open programming language interpreters and the implementation of a framework-level metaobject protocol (MOP) to support them. Inquiry: We address the problem of dynamic interpreter adaptation to tailor the interpreter's behavior on the task to be solved and to introduce new features to fulfill unforeseen requirements. Many languages provide a MOP that to some degree supports reflection. However, MOPs are typically language-specific, their reflective functionality is often restricted, and the adaptation and application logic are often mixed which hardens the understanding and maintenance of the source code. Our system overcomes these limitations. Approach: We designed and implemented a system to support open programming language interpreters. The prototype implementation is integrated in the Neverlang framework. The system exposes the structure, behavior and the runtime state of any Neverlang-based interpreter with the ability to modify it. Knowledge: Our system provides a complete control over interpreter's structure, behavior and its runtime state. The approach is applicable to every Neverlang-based interpreter. Adaptation code can potentially be reused across different language implementations. Grounding: Having a prototype implementation we focused on feasibility evaluation. The paper shows that our approach well addresses problems commonly found in the research literature. We have a demonstrative video and examples that illustrate our approach on dynamic software adaptation, aspect-oriented programming, debugging and context-aware interpreters. Importance: To our knowledge, our paper presents the first reflective approach targeting a general framework for language development. Our system provides full reflective support for free to any Neverlang-based interpreter. We are not aware of any prior application of open implementations to programming language interpreters in the sense defined in this paper. Rather than substituting other approaches, we believe our system can be used as a complementary technique in situations where other approaches present serious limitations