Learning and adaptation in speech production without a vocal tract

How is the complex audiomotor skill of speaking learned? To what extent does it depend on the specific characteristics of the vocal tract? Here, we developed a touchscreen-based speech synthesizer to examine learning of speech production independent of the vocal tract. Participants were trained to reproduce heard vowel targets by reaching to locations on the screen without visual feedback and receiving endpoint vowel sound auditory feedback that depended continuously on touch location. Participants demonstrated learning as evidenced by rapid increases in accuracy and consistency in the production of trained targets. This learning generalized to productions of novel vowel targets. Subsequent to learning, sensorimotor adaptation was observed in response to changes in the location-sound mapping. These findings suggest that participants learned adaptable sensorimotor maps allowing them to produce desired vowel sounds. These results have broad implications for understanding the acquisition of speech motor control.Published versio

Principles in Patterns (PiP) : Project Evaluation Synthesis

Evaluation activity found the technology-supported approach to curriculum design and approval developed by PiP to demonstrate high levels of user acceptance, promote improvements to the quality of curriculum designs, render more transparent and efficient aspects of the curriculum approval and quality monitoring process, demonstrate process efficacy and resolve a number of chronic information management difficulties which pervaded the previous state. The creation of a central repository of curriculum designs as the basis for their management as "knowledge assets", thus facilitating re-use and sharing of designs and exposure of tacit curriculum design practice, was also found to be highly advantageous. However, further process improvements remain possible and evidence of system resistance was found in some stakeholder groups. Recommendations arising from the findings and conclusions include the need to improve data collection surrounding the curriculum approval process so that the process and human impact of C-CAP can be monitored and observed. Strategies for improving C-CAP acceptance among the "late majority", the need for C-CAP best practice guidance, and suggested protocols on the knowledge management of curriculum designs are proposed. Opportunities for further process improvements in institutional curriculum approval, including a re-engineering of post-faculty approval processes, are also recommended

Model of distributed software development using system dynamics

Distributed Software Development today is one of the most widely used and implemented software development strategies in the industry [1]. Some of the major advantages of this methodology are 24 hour work-cycle [2], increased diversity of resources, reduced labor costs, decreased time of iteration cycle and diverse skillset of the workforce [3]. Although it has proven to be quite efficient and practical, there are ample reasons from previous research [4][5] in this field that show that this development approach is uncertain in terms of quality of product developed, speed and expenses. Factors such as presence of multiple stakeholders, lack of effective communication among sites, cultural differences among the workforce and presence of a diverse range of system variables brings a level of uncertainty into the system. A method is required to simulate iterations of the software development lifecycle and understand the effect of changes in system variables/stakeholders involved. This would help project managers, business analysts and other parties involved from different sites to examine the effect of changes in one variable at any point to the other variables and inspect its short and long term consequence on the project plan and deliverables. Problems leading to faulty product development, failure in conforming to all the lifecycle requirements, decreased customer satisfaction, unforeseen expenses and inability to meet deadlines can be avoided by predicting changes using those predictions to make better decisions. In this thesis, I have created a simulated model of Distributed Software Development using the concept of System Dynamics [6]. My main purpose is to define the different variables, and stakeholders involved in this methodology. Furthermore, I aim to define relationships among them, analyze and draw sufficient conclusions that would help understand and decrease uncertainty. As an example, the results of the simulation show prediction of change of the number of customers, features released with time for the given product as other variables in the system change. This can help project directors, managers and leads to make better informed decisions about the steps they can take to maximize their product growth in the market

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Neural Dynamics of Delayed Feedback in Robot Teleoperation: Insights from fNIRS Analysis

As robot teleoperation increasingly becomes integral in executing tasks in distant, hazardous, or inaccessible environments, the challenge of operational delays remains a significant obstacle. These delays are inherent in signal transmission and processing and can adversely affect the operators performance, particularly in tasks requiring precision and timeliness. While current research has made strides in mitigating these delays through advanced control strategies and training methods, a crucial gap persists in understanding the neurofunctional impacts of these delays and the efficacy of countermeasures from a cognitive perspective. Our study narrows this gap by leveraging functional Near-Infrared Spectroscopy (fNIRS) to examine the neurofunctional implications of simulated haptic feedback on cognitive activity and motor coordination under delayed conditions. In a human-subject experiment (N=41), we manipulated sensory feedback to observe its influences on various brain regions of interest (ROIs) response during teleoperation tasks. The fNIRS data provided a detailed assessment of cerebral activity, particularly in ROIs implicated in time perception and the execution of precise movements. Our results reveal that certain conditions, which provided immediate simulated haptic feedback, significantly optimized neural functions related to time perception and motor coordination, and improved motor performance. These findings provide empirical evidence about the neurofunctional basis of the enhanced motor performance with simulated synthetic force feedback in the presence of teleoperation delays.Comment: Submitted to Frontiers in Human Neuroscienc

Identifying and mitigating the cognitive implications of semi-natural virtual locomotion techniques

Users of virtual reality systems often need to navigate to distant parts of the virtual environment in order to perform their desired tasks. Unfortunately, physical space restrictions as well as tracker range limitations preclude the use of fully natural techniques for navigation through an infinite virtual environment. This necessitates the use of a locomotion interface, and the closer that interface matches the analogous real world actions, the easier it will be for the user. Unnatural techniques require cognitive effort on the part of the users. Many authors have attempted to address this problem by creating locomotion interfaces and techniques that more closely approximate real world counterparts to the extent possible. In addition to requiring these unnatural movements, current virtual reality systems are incapable of providing the high-fidelity sensory feedback used to guide real-world movements. This may cause users to resort to more cognitively demanding strategies. There is a large body of research in the psychology domain regarding the structure of cognitive resources. In particular, Baddeley\u27s multi-component model of working memory describes a separation between the resources used for verbal and non-verbal storage and processing. It is likely that semi-natural locomotion techniques require some of these resources, which will then be unavailable for concurrent tasks. A pair of studies was conducted, investigating the cognitive resource requirements of several atomic locomotion movements by manipulating the user interface and field of view. The results indicate that semi-natural locomotion interfaces generally require a user\u27s spatial cognitive resources. Based on the conclusions from the working memory studies, an adaptive system was designed that can learn how to adjust parameters of the locomotion technique according to a user\u27s present cognitive task load

DDX7: Differentiable FM Synthesis of Musical Instrument Sounds

FM Synthesis is a well-known algorithm used to generate complex timbre from a compact set of design primitives. Typically featuring a MIDI interface, it is usually impractical to control it from an audio source. On the other hand, Differentiable Digital Signal Processing (DDSP) has enabled nuanced audio rendering by Deep Neural Networks (DNNs) that learn to control differentiable synthesis layers from arbitrary sound inputs. The training process involves a corpus of audio for supervision, and spectral reconstruction loss functions. Such functions, while being great to match spectral amplitudes, present a lack of pitch direction which can hinder the joint optimization of the parameters of FM synthesizers. In this paper, we take steps towards enabling continuous control of a well-established FM synthesis architecture from an audio input. Firstly, we discuss a set of design constraints that ease spectral optimization of a differentiable FM synthesizer via a standard reconstruction loss. Next, we present Differentiable DX7 (DDX7), a lightweight architecture for neural FM resynthesis of musical instrument sounds in terms of a compact set of parameters. We train the model on instrument samples extracted from the URMP dataset, and quantitatively demonstrate its comparable audio quality against selected benchmarks

Analysis of Disengagements in Semi-Autonomous Vehicles: Drivers’ Takeover Performance and Operational Implications

This report analyzes the reactions of human drivers placed in simulated Autonomous Technology disengagement scenarios. The study was executed in a human-in-the-loop setting, within a high-fidelity integrated car simulator capable of handling both manual and autonomous driving. A population of 40 individuals was tested, with metrics for control takeover quantification given by: i) response times (considering inputs of steering, throttle, and braking); ii) vehicle drift from the lane centerline after takeover as well as overall (integral) drift over an S-turn curve compared to a baseline obtained in manual driving; and iii) accuracy metrics to quantify human factors associated with the simulation experiment. Independent variables considered for the study were the age of the driver, the speed at the time of disengagement, and the time at which the disengagement occurred (i.e., how long automation was engaged for). The study shows that changes in the vehicle speed significantly affect all the variables investigated, pointing to the importance of setting up thresholds for maximum operational speed of vehicles driven in autonomous mode when the human driver serves as back-up. The results shows that the establishment of an operational threshold could reduce the maximum drift and lead to better control during takeover, perhaps warranting a lower speed limit than conventional vehicles. With regards to the age variable, neither the response times analysis nor the drift analysis provide support for any claim to limit the age of drivers of semi-autonomous vehicles