Improved Reinforcement Learning with Curriculum

Humans tend to learn complex abstract concepts faster if examples are presented in a structured manner. For instance, when learning how to play a board game, usually one of the first concepts learned is how the game ends, i.e. the actions that lead to a terminal state (win, lose or draw). The advantage of learning end-games first is that once the actions which lead to a terminal state are understood, it becomes possible to incrementally learn the consequences of actions that are further away from a terminal state - we call this an end-game-first curriculum. Currently the state-of-the-art machine learning player for general board games, AlphaZero by Google DeepMind, does not employ a structured training curriculum; instead learning from the entire game at all times. By employing an end-game-first training curriculum to train an AlphaZero inspired player, we empirically show that the rate of learning of an artificial player can be improved during the early stages of training when compared to a player not using a training curriculum.Comment: Draft prior to submission to IEEE Trans on Games. Changed paper slightl

The role of employee engagement and communication in gaining competitive advantage

There are many challenges facing today’s modern organisations, not least of which is to keep the employee workforce motivated and performing in the way in which the organisation requires in order to attain and sustain competitive advantage, and to satisfy stakeholder expectation. This can be understood as leveraging employee engagement in the organisational workplace. This dissertation reported on a real live and current organisational issue within the Authors most recent employer (The Author left Organisation X on 26/3/08 to complete this dissertation)- Organisation X (The Author has been asked to use any reference to the subject organisation within the dissertation as ‘Organisation X’, in order to maintain anonymity, and preserve any sensitive commercial intelligence regarding strategy and direction), a large UK Plc and a member of the FTSE 250 with c£10bn turnover, and c2300 retail locations; many of them prime High Street. The dissertation debated the impact of employee engagement, what it is, how it links to competitive advantage, the tools and behaviours of employee engagement and the merit to Organisation X of having an engaged workforce in their retail estate, (through which 85% of their profitability is delivered). The paper crucially offers recommendations on what Organisation X can do to progress their employee engagement. Concluding with a critical synthesis of whether Organisation X is following the correct route to leverage their internal resource – the employees, and thus provide maximum value to their stakeholders. The outcomes from this research greatly assisted the Author in developing and communicating his own employee engagement strategy within his area of responsibility thus impacting positively on his domain business and trading performance. These outcomes yielded an opportunity to be adopted in the peer areas of Organisation X’s retail network thus building a firm platform for competitive advantage, which in turn drives performance ahead of the sector market, and delivers growth in shareholder value. A robust research methodology is provided, discussing aspects of research philosophy (epistemology; ontology), research methodology (qualitative; phenomenalist and inductive approach), research methods (two-way feedback questionnaire and focus groups) and ethical considerations. A literature review of the main content and process theories of employee engagement is set out and discussed. This proposal would be useful for anyone interested in organisational employee engagement, (specifically aligned to the case organisation), how it links to competitive advantage and to those who want to know how to write a coherent dissertation with a strong methodology and literature review

A framework for the design of privacy preserving pervasive healthcare

Privacy is an important aspect of pervasive and ubiquitous computing systems, and, in particular, pervasive healthcare. With reference to previous approaches on developing privacy sensitive pervasive healthcare applications, we detail a framework for the design of such systems that aims to minimise the impact of privacy on such systems. In reviewing previous approaches, we extract and combine common elements in order to unify the approaches and create a more formal methodology for designing privacy mechanisms in pervasive healthcare applications. In doing so we also consider the manner in which ubiquitous technologies impact on privacy and methods for reducing this impact. We demonstrate how the framework can be applied by using examples from the previous approaches. In addressing privacy issues, the framework aims to remove a large obstacle to deployment of pervasive healthcare systems, acceptance of the technology.<br /

Context aware privacy in visual surveillance

In this paper we present preliminary work implementing dynamic privacy in public surveillance. The aim is to maximise the privacy of those under surveillance, while giving an observer access to sufficient information to perform their duties. As these aspects are in conflict, a dynamic approach to privacy is required to balance the system\u27s purpose with the system\u27s privacy. Dynamic privacy is achieved by accounting for the situation, or context, within the environment. The context is determined by a number of visual features that are combined and then used to determine an appropriate level of privacy.<br /

Dynamic privacy in a smart house environment

A smart house can be regarded as a surveillance environment in which the person being observed carries out activities that range from intimate to more public. What can be observed depends on the activity, the person observing (e.g. a carer) and policy. In assisted living smart house environments, a single privacy policy, applied throughout, would be either too invasive for an occupant, or too restrictive for an observer, due to the conflicting goals of surveillance and private environments. Hence, we propose a dynamic method for altering the level of privacy in the environment based on the context, the situation within the environment, encompassing factors relevant to ensuring the occupant\u27s safety and privacy. The context is mapped to an appropriate level of privacy, which is implemented by controlling access to data sources (e.g. video) using data hiding techniques. The aim of this work is to decrease the invasiveness of the technology, while retaining the purpose of the system.<br /

Unifying background models over complex audio using entropy

In this paper we extend an existing audio background modelling technique, leading to a more robust application to complex audio environments. The determination of background audio is used as an initial stage in the analysis of audio for surveillance and monitoring applications. Knowledge of the background serves to highlight unusual or infrequent sounds. An existing modelling approach uses an online, adaptive Gaussian Mixture model technique that uses multiple distributions to model variations in the background. The method used to determine the background distributions of the GMM leads to a failure mode of the existing technique when applied to complex audio. We propose a method incorporating further information, the proximity of distributions determined using entropy, to determine a more complete background model. The method was successful in more robustly modelling the background for complex audio scenes.<br /

Persistent audio modelling for background determination

This paper is concerned with modelling background audio online to detect foreground sounds in complex audio environments for surveillance and smart home applications. We examine and expand upon previous work in the audio and video domains, and propose a new implementation of an audio background modelling algorithm, addressing the complexities of audio data. A number of audio features characterising different aspects of the audio content were analysed to determine the factors relevant to the determination of the background audio. We test the algorithms on three audio data sets of varying complexity. The new approach was successful in modelling the background audio for the test data