Structuring visual exploratory analysis of skill demand

The analysis of increasingly large and diverse data for meaningful interpretation and question answering is handicapped by human cognitive limitations. Consequently, semi-automatic abstraction of complex data within structured information spaces becomes increasingly important, if its knowledge content is to support intuitive, exploratory discovery. Exploration of skill demand is an area where regularly updated, multi-dimensional data may be exploited to assess capability within the workforce to manage the demands of the modern, technology- and data-driven economy. The knowledge derived may be employed by skilled practitioners in defining career pathways, to identify where, when and how to update their skillsets in line with advancing technology and changing work demands. This same knowledge may also be used to identify the combination of skills essential in recruiting for new roles. To address the challenges inherent in exploring the complex, heterogeneous, dynamic data that feeds into such applications, we investigate the use of an ontology to guide structuring of the information space, to allow individuals and institutions to interactively explore and interpret the dynamic skill demand landscape for their specific needs. As a test case we consider the relatively new and highly dynamic field of Data Science, where insightful, exploratory data analysis and knowledge discovery are critical. We employ context-driven and task-centred scenarios to explore our research questions and guide iterative design, development and formative evaluation of our ontology-driven, visual exploratory discovery and analysis approach, to measure where it adds value to users’ analytical activity. Our findings reinforce the potential in our approach, and point us to future paths to build on

The Whole World in Your Hand: Active and Interactive Segmentation

Object segmentation is a fundamental problem in computer vision and a powerful resource for development. This paper presents three embodied approaches to the visual segmentation of objects. Each approach to segmentation is aided by the presence of a hand or arm in the proximity of the object to be segmented. The first approach is suitable for a robotic system, where the robot can use its arm to evoke object motion. The second method operates on a wearable system, viewing the world from a human's perspective, with instrumentation to help detect and segment objects that are held in the wearer's hand. The third method operates when observing a human teacher, locating periodic motion (finger/arm/object waving or tapping) and using it as a seed for segmentation. We show that object segmentation can serve as a key resource for development by demonstrating methods that exploit high-quality object segmentations to develop both low-level vision capabilities (specialized feature detectors) and high-level vision capabilities (object recognition and localization)

Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli

In natural vision both stimulus features and task-demands affect an observer's attention. However, the relationship between sensory-driven (“bottom-up”) and task-dependent (“top-down”) factors remains controversial: Can task-demands counteract strong sensory signals fully, quickly, and irrespective of bottom-up features? To measure attention under naturalistic conditions, we recorded eye-movements in human observers, while they viewed photographs of outdoor scenes. In the first experiment, smooth modulations of contrast biased the stimuli's sensory-driven saliency towards one side. In free-viewing, observers' eye-positions were immediately biased toward the high-contrast, i.e., high-saliency, side. However, this sensory-driven bias disappeared entirely when observers searched for a bull's-eye target embedded with equal probability to either side of the stimulus. When the target always occurred in the low-contrast side, observers' eye-positions were immediately biased towards this low-saliency side, i.e., the sensory-driven bias reversed. Hence, task-demands do not only override sensory-driven saliency but also actively countermand it. In a second experiment, a 5-Hz flicker replaced the contrast gradient. Whereas the bias was less persistent in free viewing, the overriding and reversal took longer to deploy. Hence, insufficient sensory-driven saliency cannot account for the bias reversal. In a third experiment, subjects searched for a spot of locally increased contrast (“oddity”) instead of the bull's-eye (“template”). In contrast to the other conditions, a slight sensory-driven free-viewing bias prevails in this condition. In a fourth experiment, we demonstrate that at known locations template targets are detected faster than oddity targets, suggesting that the former induce a stronger top-down drive when used as search targets. Taken together, task-demands can override sensory-driven saliency in complex visual stimuli almost immediately, and the extent of overriding depends on the search target and the overridden feature, but not on the latter's free-viewing saliency

Detecting and Refactoring Operational Smells within the Domain Name System

The Domain Name System (DNS) is one of the most important components of the Internet infrastructure. DNS relies on a delegation-based architecture, where resolution of names to their IP addresses requires resolving the names of the servers responsible for those names. The recursive structures of the inter dependencies that exist between name servers associated with each zone are called dependency graphs. System administrators' operational decisions have far reaching effects on the DNSs qualities. They need to be soundly made to create a balance between the availability, security and resilience of the system. We utilize dependency graphs to identify, detect and catalogue operational bad smells. Our method deals with smells on a high-level of abstraction using a consistent taxonomy and reusable vocabulary, defined by a DNS Operational Model. The method will be used to build a diagnostic advisory tool that will detect configuration changes that might decrease the robustness or security posture of domain names before they become into production.Comment: In Proceedings GaM 2015, arXiv:1504.0244

Augmenting Situated Spoken Language Interaction with Listener Gaze

Collaborative task solving in a shared environment requires referential success. Human speakers follow the listener’s behavior in order to monitor language comprehension (Clark, 1996). Furthermore, a natural language generation (NLG) system can exploit listener gaze to realize an effective interaction strategy by responding to it with verbal feedback in virtual environments (Garoufi, Staudte, Koller, & Crocker, 2016). We augment situated spoken language interaction with listener gaze and investigate its role in human-human and human-machine interactions. Firstly, we evaluate its impact on prediction of reference resolution using a mulitimodal corpus collection from virtual environments. Secondly, we explore if and how a human speaker uses listener gaze in an indoor guidance task, while spontaneously referring to real-world objects in a real environment. Thirdly, we consider an object identification task for assembly under system instruction. We developed a multimodal interactive system and two NLG systems that integrate listener gaze in the generation mechanisms. The NLG system “Feedback” reacts to gaze with verbal feedback, either underspecified or contrastive. The NLG system “Installments” uses gaze to incrementally refer to an object in the form of installments. Our results showed that gaze features improved the accuracy of automatic prediction of reference resolution. Further, we found that human speakers are very good at producing referring expressions, and showing listener gaze did not improve performance, but elicited more negative feedback. In contrast, we showed that an NLG system that exploits listener gaze benefits the listener’s understanding. Specifically, combining a short, ambiguous instruction with con- trastive feedback resulted in faster interactions compared to underspecified feedback, and even outperformed following long, unambiguous instructions. Moreover, alternating the underspecified and contrastive responses in an interleaved manner led to better engagement with the system and an effcient information uptake, and resulted in equally good performance. Somewhat surprisingly, when gaze was incorporated more indirectly in the generation procedure and used to trigger installments, the non-interactive approach that outputs an instruction all at once was more effective. However, if the spatial expression was mentioned first, referring in gaze-driven installments was as efficient as following an exhaustive instruction. In sum, we provide a proof of concept that listener gaze can effectively be used in situated human-machine interaction. An assistance system using gaze cues is more attentive and adapts to listener behavior to ensure communicative success

Separable mechanisms underlying global feature-based attention

Feature-based attention is known to operate in a spatially global manner, in that the selection of attended features is not bound to the spatial focus of attention. Here we used electromagnetic recordings in human observers to characterize the spatiotemporal signature of such global selection of an orientation feature. Observers performed a simple orientation-discrimination task while ignoring task-irrelevant orientation probes outside the focus of attention. We observed that global feature-based selection, indexed by the brain response to unattended orientation probes, is composed of separable functional components. One such component reflects global selection based on the similarity of the probe with task-relevant orientation values ("template matching"), which is followed by a component reflecting selection based on the similarity of the probe with the orientation value under discrimination in the focus of attention ("discrimination matching"). Importantly, template matching occurs at similar to 150 ms after stimulus onset, similar to 80 ms before the onset of discrimination matching. Moreover, source activity underlying template matching and discrimination matching was found to originate from ventral extrastriate cortex, with the former being generated in more anterolateral and the latter in more posteromedial parts, suggesting template matching to occur in visual cortex higher up in the visual processing hierarchy than discrimination matching. We take these observations to indicate that the population-level signature of global feature-based selection reflects a sequence of hierarchically ordered operations in extrastriate visual cortex, in which the selection based on task relevance has temporal priority over the selection based on the sensory similarity between input representations

Contingent task and motion planning under uncertainty for human–robot interactions

Manipulation planning under incomplete information is a highly challenging task for mobile manipulators. Uncertainty can be resolved by robot perception modules or using human knowledge in the execution process. Human operators can also collaborate with robots for the execution of some difficult actions or as helpers in sharing the task knowledge. In this scope, a contingent-based task and motion planning is proposed taking into account robot uncertainty and human–robot interactions, resulting a tree-shaped set of geometrically feasible plans. Different sorts of geometric reasoning processes are embedded inside the planner to cope with task constraints like detecting occluding objects when a robot needs to grasp an object. The proposal has been evaluated with different challenging scenarios in simulation and a real environment.Postprint (published version

Mastering the requirements analysis for communication-intensive websites

Web application development still needs to employ effective methods to accommodate some distinctive aspects of the requirements analysis process: capturing high-level communication goals, considering several user profiles and stakeholders, defining hypermedia-specific requirements (concerning navigation, content, information structure and presentation aspects), and reusing requirements for an effective usability evaluation. Techniques should be usable by both stakeholders and the design team, require little training effort, and show relative advantage to project managers. Over the last few years, requirements methodologies applied to web-based applications have considered mainly the transactional and operational aspects typical of traditional information systems. The communicational aspects of web sites have been neglected in regards to systematic requirements methods. This thesis, starting from key achievements in Requirements Engineering (hereafter RE), introduces a model (AWARE) for defining and analyzing requirements for web applications mainly conceived as strategic communication means for an institution or organization. The model extends traditional goal and scenario-based approaches for refining highlevel goals into website requirements, by introducing the analysis of ill-defined user goals, stakeholder communication goals, and a hypermedia requirement taxonomy to facilitate web conceptual design, and paving the way for a systematic usability evaluation. AWARE comprises a conceptual toolkit and a notation for effective requirements documentation. AWARE concepts and notation represent a useful communication and analysis conceptual tool that may support in the elicitation, negotiation, analysis and validation of requirements from the relevant stakeholders (users included). The empirical validation of the model is carried out in two ways. Firstly, the model has been employed in web projects on the field. These case studies and the lessons learnt will be presented and discussed to assess advantages and limits of the proposal. Secondly, a sample of web analysts and designers has been asked to study and apply the model: the feedback gathered is positive and encouraging for further improvement.Lo sviluppo di applicazioni web necessita di strumenti efficaci per gestire alcuni aspetti essenziali del processo di analisi dei requisiti: l'identificazione di obiettivi di comunicazione strategici, la presenza di una varietà di profili utente e di stakeholders, le definizione di requisiti ipermediali (riguardanti navigazione, interazione, contenuto e presentazione), e il riuso dei requisiti per una pianificazione efficace della valutazione dell'usabilità. Sono necessarie tecniche usabili sia dagli stakeholders che dai progettisti, che richiedono un tempo breve per essere appresi ed usati con efficacia, mostrando vantaggi significativi ai gestori di progetti complessi. La tesi definisce AWARE (Analysis of Web Application Requirements) - una metodologia per l'analisi dei requisiti specifica per la gestione di siti web (ed applicazioni interattive) con forti componenti comunicative. La metodologia estende le tecniche esistenti dell''analisi dei requisiti basate su approcci goal-oriented e scenario-based, introducendo una tassonomia di requisiti specifica per siti web (che permette di dare un input strutturato all'attività di progetazione), strumenti per l'identificazione e l'analisi di obiettivi ill-defined (generici o mal-definiti) e di obiettivi comunicativi e supporto metodologico per la valutazione dell'usabilità basata sui requisiti dell'applicazione. La metodologia AWARE è stata valutata sul campo attraverso progetti con professionisti del settore (web designers e IT managers), e grazie ad interventi di formazione in aziende specializzate nella comunicazione su web