Human-Machine Collaborative Optimization via Apprenticeship Scheduling

Coordinating agents to complete a set of tasks with intercoupled temporal and resource constraints is computationally challenging, yet human domain experts can solve these difficult scheduling problems using paradigms learned through years of apprenticeship. A process for manually codifying this domain knowledge within a computational framework is necessary to scale beyond the ``single-expert, single-trainee" apprenticeship model. However, human domain experts often have difficulty describing their decision-making processes, causing the codification of this knowledge to become laborious. We propose a new approach for capturing domain-expert heuristics through a pairwise ranking formulation. Our approach is model-free and does not require enumerating or iterating through a large state space. We empirically demonstrate that this approach accurately learns multifaceted heuristics on a synthetic data set incorporating job-shop scheduling and vehicle routing problems, as well as on two real-world data sets consisting of demonstrations of experts solving a weapon-to-target assignment problem and a hospital resource allocation problem. We also demonstrate that policies learned from human scheduling demonstration via apprenticeship learning can substantially improve the efficiency of a branch-and-bound search for an optimal schedule. We employ this human-machine collaborative optimization technique on a variant of the weapon-to-target assignment problem. We demonstrate that this technique generates solutions substantially superior to those produced by human domain experts at a rate up to 9.5 times faster than an optimization approach and can be applied to optimally solve problems twice as complex as those solved by a human demonstrator.Comment: Portions of this paper were published in the Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) in 2016 and in the Proceedings of Robotics: Science and Systems (RSS) in 2016. The paper consists of 50 pages with 11 figures and 4 table

The complexities of technology-based care : telecare as perceived by care practitioners

Telecare, which offers ‘care at a distance’ (Pols, 2012) through a variety of remote monitoring technologies, has developed rapidly across health and social care policy in many developed countries. Nonetheless, approaches to this development differ; the focus of this paper is the United Kingdom, where implementation has been particularly rapid and ambitious in scope. The paper argues that, while there are clear and tangible benefits from the use of some telecare technologies, there is insufficient research about the complexities of implementation with end users. These complexities include ethical questions raised by the use of monitoring and surveillance equipment, the ability to fine tune technologies to the needs of individuals, and the way in which care relationships may be altered by remote care mediated through technologies. This paper addresses these issues through a particularly under researched area; that is, the perceptions of care practitioners who assess for, and interact with, these technologies. The research was conducted with practitioners using qualitative research methodology. The paper concludes that Telecare practice is uneven in the way it addresses complexities and that more needs to be done to understand the way in which technologies are discussed and utilised by those charged with their implementation

The actions of others act as a pseudo-reward to drive imitation in the context of social reinforcement learning

While there is no doubt that social signals affect human reinforcement learning, there is still no consensus about how this process is computationally implemented. To address this issue, we compared three psychologically plausible hypotheses about the algorithmic implementation of imitation in reinforcement learning. The first hypothesis, decision biasing (DB), postulates that imitation consists in transiently biasing the learner's action selection without affecting their value function. According to the second hypothesis, model-based imitation (MB), the learner infers the demonstrator's value function through inverse reinforcement learning and uses it to bias action selection. Finally, according to the third hypothesis, value shaping (VS), the demonstrator's actions directly affect the learner's value function. We tested these three hypotheses in 2 experiments (N = 24 and N = 44) featuring a new variant of a social reinforcement learning task. We show through model comparison and model simulation that VS provides the best explanation of learner's behavior. Results replicated in a third independent experiment featuring a larger cohort and a different design (N = 302). In our experiments, we also manipulated the quality of the demonstrators' choices and found that learners were able to adapt their imitation rate, so that only skilled demonstrators were imitated. We proposed and tested an efficient meta-learning process to account for this effect, where imitation is regulated by the agreement between the learner and the demonstrator. In sum, our findings provide new insights and perspectives on the computational mechanisms underlying adaptive imitation in human reinforcement learning

Gaze following in an asocial reptile (Eublepharis macularius)

Gaze following is the ability to utilise information from another's gaze. It is most often seen in a social context or as a reflexive response to interesting external stimuli. Social species can potentially reveal utilisable knowledge about another's future intentions by attending to the target of their gaze. However, in even more fundamental situations, being sensitive to another's gaze can also be useful such as when it can facilitate greater foraging efficiency or lead to earlier predator detection. While gaze sensitivity has been shown to be prevalent in a number of social species, little is currently known about the potential for gaze following in asocial species. The current study investigated whether an asocial reptile, the leopard gecko (Eublepharis macularius), could reliably use the visual indicators of attention to follow the gaze of a conspecific around a barrier. We operated three trial conditions and found subjects (N = 6) responded significantly more to the conspecific demonstrator looking up at a laser stimulus projected onto an occluder during the experimental condition compared to either of two control conditions. The study's findings point toward growing evidence for gaze-following ability in reptiles, who are typically categorised as asocial. Furthermore, our findings support developing comparative social cognition research showing the origins of gaze following and other cognitive behaviours that may be more widely distributed across taxonomic groups than hitherto thought

From Things to Services: A Social IoT Approach for Tourist Service Management

In the context of Internet of Things (IoT), the cooperation and synergy between varied and disparate communicating objects is strained by trustworthiness, confidentiality and interoperability concerns. These restrictions can limit the development of IoT-based applications especially considering the emergent boost in the number of communicating objects and their growing itinerant nature in a collective service context. A new perspective arises with the paradigm of Social Internet of Things (SIoT), that relies on the implementation of semi-independent communicating objects with cooperation assessed by social relations and social feed-back. In this article, we present the development and expansion of the IoT concept towards SIoT in the context of the interactions between tourist services as communicating objects. As a proof-of-concept we propose a composition of services as virtualized social objects and the interaction between them, by taking into consideration the balance, trustworthiness, cooperation and synergy of services. Furthermore we present a solution to integrate also accessibility in SIoT services. The presented concept is presented using a demonstrator build for tourist services

Human factors for dementia: Evidence based design

Designing care environments for people living with dementia is a complex challenge as the key stakeholder may have difficulty communicating their capabilities, limitations and preferences. This paper describes the use of evidence-based design personas in a multi-disciplinary team with architects and chartered human factors specialists. Four individual personas (Alison, Barry, Christine and David) and a couple persona (Chris and Sally) were used to bring the voices of the people living with different stages of dementia to the design process. Their changing/fluctuating symptoms were communicated in two formats (wheel and matrix) within an inclusive design process to adapt a Victorian semi-detached house. The demonstrator house presents evidence based design, adaptation and support solutions to support people living with dementia to age well at home

Collaborative R&D and technology transfer: a rapid and effective route to the success of innovation?: a system-oriented assessment for Portugal 2020's demonstrator projects

Technology transfer - defined as the reciprocal movement of know-how between an institutional technological environment and an industrial environment - has operated as an enabler of the business innovation process. Its socio-economic benefits have warranted the interest of organizations, since it allows them to acquire a larger set of skills, to complement their learning and to more rapidly overcome any obstacles that might arise in technology development and commercialization. In this regard, within the 2014-2020 Framework, policy-makers reinforced their innovation support programmes and policies, for the purpose of stimulating this phenomenon and thereby foster a structural transformation based on competitiveness and economy specialization in multiple and interconnected spaces. Nevertheless, a holistic assessment of those instruments, which might foster coherent and transparent decision-making in Portugal, is plainly lacking. In the light of this, the current dissertation puts forwards a systemic assessment methodology and applies it to one of the measures that promotes technology transfer – the Demonstrator projects. This assessment model makes it possible to monitor, inform and disclose the efficiency with which the activities are coordinated, the quality of the processes, the prospective effects and externalities, as well as their market potential, in an accurate and comprehensive way. The results were quite clear: innovation, by means of technology transfer, promotes more qualified employment, dissemination actions, desire for business expansion, international connections, interregional and intersectoral collaboration and, consequently, technologies with larger innovative and commercial capacity. Emphasis is therefore put on fostering its practice in Portugal, which entails the concertation of the existing deficit between all the economic agents comprising it.A transferência tecnológica - caraterizada como o movimento recíproco de know-how entre um ambiente institucional tecnológico e um ambiente industrial - tem atuado como facilitadora do processo de inovação empresarial. Os seus benefícios socioeconómicos têm despertado o interesse das organizações, uma vez que lhes permite adquirir uma maior base de skills, complementar aprendizagens e mais rapidamente superar quaisquer obstáculos com que se deparem no desenvolvimento e comercialização tecnológica. Neste contexto, no Quadro 2014-2020, os policy-makers reforçaram os seus programas e instrumentos de apoio à inovação, visando estimular este fenómeno e, assim, fomentar uma transformação estrutural baseada na competitividade e na especialização da economia em espaços múltiplos e interligados. Todavia, denota-se a ausência de uma avaliação holística desses instrumentos, com vista a tomadas de decisão coerentes e transparentes em Portugal. Face a esta evidência, a presente dissertação propõe uma metodologia de avaliação sistémica e aplica-a a uma das medidas que promove a transferência tecnológica - os projetos Demonstradores. Este modelo permite monitorizar, informar e divulgar sobre a eficiência com que as atividades são coordenadas, a qualidade dos processos, os possíveis efeitos e externalidades, bem como o seu potencial de mercado, de forma precisa e abrangente. Os resultados foram bastante claros: a inovação, através da transferência tecnológica, promove mais emprego qualificado, ações de difusão, vontade de expansão de negócio, ligações internacionais, colaboração intersectorial e interregional e, por conseguinte, tecnologias com maior capacidade inovadora e comercial. Enfatiza-se, assim, um maior estímulo à sua prática em Portugal, o que implica a concertação do défice existente entre todos os agentes económicos seus constituintes

Making Telecare desirable rather than a last resort

Despite reported benefits of Telecare use for older adults, uptake of Telecare in the United Kingdom remains relatively low. Non-users of Telecare are an under-researched group in the Telecare field. We conducted 22 qualitative individual semi-structured interviews to explore the views and opinions of current non-users of Telecare regarding barriers and facilitators to its use, and explored considerations which may precede their decision to accept, or reject, Telecare. Framework analysis identified a number of themes which influence the outcome and timing of this decision, including peace of mind (for the individual and their family), the strength and composition of an individual's support network, the impact of changing personal and health circumstances, and lack of communication about Telecare (e.g. advertising). A cost–benefit decision process appears to take place for the potential user, whereby the benefit of peace of mind is weighed against perceived ‘costs’ of using Telecare. Telecare is often perceived as a last resort rather than a preventative measure. A number of barriers to Telecare use need to be addressed if individuals are to make fully informed decisions regarding their Telecare use, and to begin using Telecare at a time when it could provide them with optimal benefit. Although the study was set in England, the findings may be relevant for other countries where Telecare is used