678 research outputs found

    Speech wave-form driven motion synthesis for embodied agents

    Get PDF
    The main objective of this thesis is to synthesise motion from speech, especially in conversation. Based on previous research into different acoustic features or the combination of them were investigated, no one has investigated in estimating head motion from waveform directly, which is the stem of the speech. Thus, we study the direct use of speech waveform to generate head motion. We claim that creating a task-specific feature from waveform to generate head motion leads to better performance than using standard acoustic features to generate head motion overall. At the same time, we completely abandon the handcrafted feature extraction process, leading to more effectiveness. However, there are a few problems if we would like to apply speech waveform, 1) high dimensional, where the dimension of the waveform data is much higher than those common acoustic features and thus making the training of the model more difficult, and 2) irrelevant information, which refers to the full information in the original waveform implicating potential cumbrance for neural network training. To resolve these problems, we applied a deep canonical correlated constrainted auto-encoder (DCCCAE) to compress the waveform into low dimensional and highly correlated embedded features with head motion. The estimated head motion was evaluated both objectively and subjectively. In objective evaluation, the result confirmed that DCCCAE enables the creation of a more correlated feature with the head motion than standard AE and other popular spectral features such as MFCC and FBank, and is capable of being used in achieving state-of-the-art results for predicting natural head motion with the advantage of the DCCCAE. Besides investigating the representation learning of the feature, we also explored the LSTM-based regression model for the proposed feature. The LSTM-based models were able to boost the overall performance in the objective evaluation and adapt better to the proposed feature than MFCC. MUSHRA-liked subjective evaluation results suggest that the animations generated by models with the proposed feature were chosen to be better than the other models by the participants of MUSHRA-liked test. A/B test further that the LSTM-based regression model adapts better to the proposed feature. Furthermore, we extended the architecture to estimate the upper body motion as well. We submitted our result to GENEA2020 and our model achieved a higher score than BA in both aspects (human-likeness and appropriateness) according to the participant’s preference, suggesting that the highly correlated feature pair and the sequential estimation helped in improving the model generalisation

    Intentional dialogues in multi-agent systems based on ontologies and argumentation

    Get PDF
    Some areas of application, for example, healthcare, are known to resist the replacement of human operators by fully autonomous systems. It is typically not transparent to users how artificial intelligence systems make decisions or obtain information, making it difficult for users to trust them. To address this issue, we investigate how argumentation theory and ontology techniques can be used together with reasoning about intentions to build complex natural language dialogues to support human decision-making. Based on such an investigation, we propose MAIDS, a framework for developing multi-agent intentional dialogue systems, which can be used in different domains. Our framework is modular so that it can be used in its entirety or just the modules that fulfil the requirements of each system to be developed. Our work also includes the formalisation of a novel dialogue-subdialogue structure with which we can address ontological or theory-of-mind issues and later return to the main subject. As a case study, we have developed a multi-agent system using the MAIDS framework to support healthcare professionals in making decisions on hospital bed allocations. Furthermore, we evaluated this multi-agent system with domain experts using real data from a hospital. The specialists who evaluated our system strongly agree or agree that the dialogues in which they participated fulfil Cohen’s desiderata for task-oriented dialogue systems. Our agents have the ability to explain to the user how they arrived at certain conclusions. Moreover, they have semantic representations as well as representations of the mental state of the dialogue participants, allowing the formulation of coherent justifications expressed in natural language, therefore, easy for human participants to understand. This indicates the potential of the framework introduced in this thesis for the practical development of explainable intelligent systems as well as systems supporting hybrid intelligence

    Improving Problem-Oriented Policing with Natural Language Processing

    Get PDF
    The policing approach known as Problem oriented policing (POP) was outlined by Herman Goldstein in 1979. Despite POP being shown as an effective method to reduce crime it is difficult to implement because of the high analytical burden that accompanies it. This analytical burden is centred on understanding the mechanism by which a crime took place. One of the factors that contributes to this high burden is that a lot of the required information is stored in free- text data, which has traditionally not been in a format suitable for aggregate analysis. However, advances in machine learning, in particular natural language processing, are lowering the barriers for extracting information from free-text data. This thesis explores the potential for pre-trained language models (PTMs) to efficiently unlock the information in police crime free-text data. PTMs are a new class of machine learning model that are ‘pre-trained’ to recognise the meaning of language. This allows the PTM to interrogate large quantities of free-text data. Thanks to this pre-training, PTMs can be adapted to specific natural language processing tasks with much less effort. Efficiently unlocking the information in the police free-text crime data should reduce the analytical burden for POP. In turn, the lower analytical burden should facilitate the wider adoption of POP. The thesis concludes that the evidence suggests PTMs are potentially an efficient method for extracting useful information from police free text data

    Data journeys in the sciences

    Get PDF
    This is the final version. Available from Springer via the DOI in this record. This groundbreaking, open access volume analyses and compares data practices across several fields through the analysis of specific cases of data journeys. It brings together leading scholars in the philosophy, history and social studies of science to achieve two goals: tracking the travel of data across different spaces, times and domains of research practice; and documenting how such journeys affect the use of data as evidence and the knowledge being produced. The volume captures the opportunities, challenges and concerns involved in making data move from the sites in which they are originally produced to sites where they can be integrated with other data, analysed and re-used for a variety of purposes. The in-depth study of data journeys provides the necessary ground to examine disciplinary, geographical and historical differences and similarities in data management, processing and interpretation, thus identifying the key conditions of possibility for the widespread data sharing associated with Big and Open Data. The chapters are ordered in sections that broadly correspond to different stages of the journeys of data, from their generation to the legitimisation of their use for specific purposes. Additionally, the preface to the volume provides a variety of alternative “roadmaps” aimed to serve the different interests and entry points of readers; and the introduction provides a substantive overview of what data journeys can teach about the methods and epistemology of research.European CommissionAustralian Research CouncilAlan Turing Institut

    Designing hybridization: alternative education strategies for fostering innovation in communication design for the territory

    Get PDF
    Within the broad context of design studies, Communication Design for the Territory stands as a hybrid discipline constantly interfacing with other fields of knowledge. It assumes the territorial theme as its specific dimension, aiming to generate communication systems capable of reading the stratifications of places. From an educational perspective, teaching activities are closely linked to research and can take on different levels of complexity: from the various forms of cartographic translation to the design of sophisticated transmedia digital systems. In the wake of COVID-19, this discipline has come to terms with a profoundly changed scenario in terms of limited access to the physical space and the emergence of new technologies for remote access. In this unique context, we propose a pedagogical strategy that focuses on the hybridization of communication artifacts with the aim of fostering design experimentation. As a creative tool, hybridization leads to the design of innovative systems by strategically combining the characteristics of different artifacts to achieve specific communication goals. By experimenting with these creative strategies, students are led to critically reflect on existing communication artifacts’ features and explore original designs that deliberately combine different media, contents, and communication languages in innovative ways. Through hybridization, the methods for territorial knowledge production appear more effective, effectively combining the skills and knowledge embodied in multiple subject areas. The paper presents the experience developed in the teaching laboratories of the DCxT (Communication Design for the Territory) research group of the Design Department of Politecnico di Milano. The teaching experience highlights how hybridization strategies can increase the effectiveness in learning about territorial specificities, in acquiring critical knowledge about communication systems, and in developing innovation strategies that allow to influence the evolution of traditional communication models

    A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

    Full text link
    The inclusion of physical information in machine learning frameworks has revolutionized many application areas. This involves enhancing the learning process by incorporating physical constraints and adhering to physical laws. In this work we explore their utility for reinforcement learning applications. We present a thorough review of the literature on incorporating physics information, as known as physics priors, in reinforcement learning approaches, commonly referred to as physics-informed reinforcement learning (PIRL). We introduce a novel taxonomy with the reinforcement learning pipeline as the backbone to classify existing works, compare and contrast them, and derive crucial insights. Existing works are analyzed with regard to the representation/ form of the governing physics modeled for integration, their specific contribution to the typical reinforcement learning architecture, and their connection to the underlying reinforcement learning pipeline stages. We also identify core learning architectures and physics incorporation biases (i.e., observational, inductive and learning) of existing PIRL approaches and use them to further categorize the works for better understanding and adaptation. By providing a comprehensive perspective on the implementation of the physics-informed capability, the taxonomy presents a cohesive approach to PIRL. It identifies the areas where this approach has been applied, as well as the gaps and opportunities that exist. Additionally, the taxonomy sheds light on unresolved issues and challenges, which can guide future research. This nascent field holds great potential for enhancing reinforcement learning algorithms by increasing their physical plausibility, precision, data efficiency, and applicability in real-world scenarios

    A geo-informatics approach to sustainability assessments of floatovoltaic technology in South African agricultural applications

    Get PDF
    South African project engineers recently pioneered the first agricultural floating solar photovoltaic tech nology systems in the Western Cape wine region. This effort prepared our country for an imminent large scale diffusion of this exciting new climate solver technology. However, hydro-embedded photovoltaic sys tems interact with environmentally sensitive underlying aquatic ecosystems, causing multiple project as sessment uncertainties (energy, land, air, water) compared to ground-mounted photovoltaics. The dissimi lar behaviour of floatovoltaic technologies delivers a broader and more diversified range of technical advan tages, environmental offset benefits, and economic co-benefits, causing analytical modelling imperfections and tooling mismatches in conventional analytical project assessment techniques. As a universal interna tional real-world problem of significance, the literature review identified critical knowledge and methodology gaps as the primary causes of modelling deficiencies and assessment uncertainties. By following a design thinking methodology, the thesis views the sustainability assessment and modelling problem through a geo graphical information systems lens, thus seeing an academic research opportunity to fill critical knowledge gaps through new theory formulation and geographical knowledge creation. To this end, this philosophi cal investigation proposes a novel object-oriented systems-thinking and climate modelling methodology to study the real-world geospatial behaviour of functioning floatovoltaic systems from a dynamical system thinking perspective. As an empirical feedback-driven object-process methodology, it inspired the thesis to create new knowledge by postulating a new multi-disciplinary sustainability theory to holistically characterise agricultural floatovoltaic projects through ecosystems-based quantitative sustainability profiling criteria. The study breaks new ground at the frontiers of energy geo-informatics by conceptualising a holistic theoretical framework designed for the theoretical characterisation of floatovoltaic technology ecosystem operations in terms of the technical energy, environmental and economic (3E) domain responses. It campaigns for a fully coupled model in ensemble analysis that advances the state-of-the-art by appropriating the 3E theo retical framework as underpinning computer program logic blueprint to synthesise the posited theory in a digital twin simulation. Driven by real-world geo-sensor data, this geospatial digital twin can mimic the geo dynamical behaviour of floatovoltaics through discrete-time computer simulations in real-time and lifetime digital project enactment exercises. The results show that the theoretical 3E framing enables project due diligence and environmental impact assessment reporting as it uniquely incorporates balanced scorecard performance metrics, such as the water-energy-land-food resource impacts, environmental offset benefits and financial feasibility of floatovoltaics. Embedded in a geoinformatics decision-support platform, the 3E theory, framework and model enable numerical project decision-supporting through an analytical hierarchy process. The experimental results obtained with the digital twin model and decision support system show that the desktop-based parametric floatovoltaic synthesis toolset can uniquely characterise the broad and diverse spectrum of performance benefits of floatovoltaics in a 3E sustainability profile. The model uniquely predicts important impact aspects of the technology’s land, air and water preservation qualities, quantifying these impacts in terms of the water, energy, land and food nexus parameters. The proposed GIS model can quantitatively predict most FPV technology unknowns, thus solving a contemporary real-world prob lem that currently jeopardises floating PV project licensing and approvals. Overall, the posited theoretical framework, methodology model, and reported results provide an improved understanding of floating PV renewable energy systems and their real-world behaviour. Amidst a rapidly growing international interest in floatovoltaic solutions, the research advances fresh philosophical ideas with novel theoretical principles that may have far-reaching implications for developing electronic, photovoltaic performance models worldwide.GeographyPh. D. (Geography
    • 

    corecore