Quantitative Characteristics of Human-Written Short Stories as a Metric for Automated Storytelling

Evaluating the extent to which computer-produced stories are structured like human-invented narratives can be an important component of the quality of a story plot. In this paper, we report on an empirical experiment in which human subjects have invented short plots in a constrained scenario. The stories were annotated according to features commonly found in existing automatic story generators. The annotation was designed to measure the proportion and relations of story components that should be used in automatic computational systems for matching human behaviour. Results suggest that there are relatively common patterns that can be used as input data for identifying similarity to human-invented stories in automatic storytelling systems. The found patterns are in line with narratological models, and the results provide numerical quantification and layout of story components. The proposed method of story analysis is tested over two additional sources, the ROCStories corpus and stories generated by automated storytellers, to illustrate the valuable insights that may be derived from them

Quantitative Framework For Social Cultural Interactions

For an autonomous robot or software agent to participate in the social life of humans, it must have a way to perform a calculus of social behavior. Such a calculus must have explanatory power (it must provide a coherent theory for why the humans act the way they do), and predictive power (it must provide some plausible events from the predicted future actions of the humans). This dissertation describes a series of contributions that would allow agents observing or interacting with humans to perform a calculus of social behavior taking into account cultural conventions and socially acceptable behavior models. We discuss the formal components of the model: culture-sanctioned social metrics (CSSMs), concrete beliefs (CBs) and action impact functions. Through a detailed case study of a crooked seller who relies on the manipulation of public perception, we show that the model explains how the exploitation of social conventions allows the seller to finalize transactions, despite the fact that the clients know that they are being cheated. In a separate study, we show that how the crooked seller can find an optimal strategy with the use of reinforcement learning. We extend the CSSM model for modeling the propagation of public perception across multiple social interactions. We model the evolution of the public perception both over a single interaction and during a series of interactions over an extended period of time. An important aspect for modeling the public perception is its propagation - how the propagation is affected by the spatio-temporal context of the interaction and how does the short-term and long-term memory of humans affect the overall public perception. We validated the CSSM model through a user study in which participants cognizant with the modeled culture had to evaluate the impact on the social values. The scenarios used in the experiments modeled emotionally charged social situations in a cross-cultural setting and with the presence of a robot. The scenarios model conflicts of cross-cultural communication as well as ethical, social and financial choices. This study allowed us to study whether people sharing the same culture evaluate CSSMs at the same way (the inter-cultural uniformity conjecture). By presenting a wide range of possible metrics, the study also allowed us to determine whether any given metric can be considered a CSSM in a given culture or not

Doing pedagogical research in engineering

This is a book

Exploring the Biocybernetic loop: Classifying Psychophysiological Responses to Cultural Artefacts using Physiological Computing

The aim of this research project was to provide a bio-sensing component for a real-time adaptive technology in the context of cultural heritage. The proposed system was designed to infer the interest or intention of the user and to augment elements of the cultural heritage experience interactively through implicit interaction. Implicit interaction in this context is the process whereby the system observes the user while they interact with artefacts; recording psychophysiological responses to cultural heritage artefacts or materials and acting upon these responses to drive adaptations in content in real-time.Real-time biocybernetic control is the central component of physiological computing wherein physiological data are converted into a control input for a technological system. At its core the bio-sensing component is a biocybernetic control loop that utilises an inference of user interest as its primary driver. A biocybernetic loop is composed of four main stages: inference, classification, adaptation and interaction. The programme of research described in this thesis is concerned primarily with exploration of the inference and classification elements of the biocybernetic loop but also encompasses an element of adaptation and interaction. These elements are explored first through literature review and discussion (presented in chapters 1-5) and then through experimental studies (presented in chapters 7-11)

A cost estimate maturity benchmark method to support early concept design decision-making: a case study application to the small modular nuclear reactor

Constructing large Nuclear Power Plants (NPPs) is synonymous with significant cost and schedule uncertainty. Innovative Small Modular Reactors (SMRs) have been identified as a way of increasing certainty of delivery, whilst also maintaining a competitive Life Cycle Cost (LCC). Previous research into the cost of SMRs has focused on the economics of a design from the perspective of an owner or investor. There is a significant gap in the literature associated with cost estimating SMRs at the early concept development stage from the perspective of a reactor developer. Early design stage cost estimates are inherently uncertain. Design teams, therefore, need to make decisions that will achieve a cost competitive product by considering uncertainty. Existing cost uncertainty analysis methods lack standardisation in their application, often relying on the subjective assessment of experts. The central argument presented in this research is that the SMR vendor can make more effective decisions related to achieving cost certainty by understanding the drivers of knowledge uncertainty associated with early design stage cost estimates. This thesis describes research spanning the concept design phase of the UK SMR development programme. The research investigation is divided into two distinct phases. The first phase identifies the requirements for cost information from the perspective of the SMR vendor through interviews, a participatory case study investigation and surveys. Limited access to cost information means that early design cost assessment is highly subjective. Cost uncertainty analysis should provide decision makers with an understanding of the level of confidence associated with the estimate. A survey investigating how cost information is interpreted revealed that providing more granular detail about cost uncertainty would support the design team with additional rationale for selecting a design option. The main requirement identified from phase 1 of the research is the need for a standardised method to identify how sources of cost uncertainty influence the maturity of the estimate at each stage of the design development process. The second phase of the research involved a participatory research approach where the Acceptable Cost Uncertainty Benchmark Assessment (ACUBA) method was developed and then implemented retrospectively on the case study cost data. The ACUBA method uses a qualitative measure to assess the quality and impact of engineering definition, manufacturing process knowledge and supply chain knowledge on the cost estimate confidence. The maturity rating is then assessed against a benchmark to determine the acceptability of the estimate uncertainty range. Focus groups were carried out in the vendor organisation to investigate whether the design team could clarify their reasoning for decisions related to reducing cost uncertainty when given insight into the sources of cost uncertainty. The rationale for a decision is found to be clearer using the ACUBA method compared with existing cost uncertainty analysis methods used by the case study organisation. This research has led to the development of a novel method which standardises and improves the communication of cost information across different functions within a design team. By establishing a benchmark acceptable level of cost maturity for a decision, the cost maturity metric can be employed to measure the performance of the SMR development programme towards achieving product cost maturity. In addition, the ACUBA method supports the more effective allocation of limited resources available at the early design stage, by identifying design activities which could lead to an acceptable cost maturity.</div

Use of automated coding methods to assess motivational behaviour in education

Teachers’ motivational behaviour is related to important student outcomes. Assessing teachers’ motivational behaviour has been helpful to improve teaching quality and enhance student outcomes. However, researchers in educational psychology have relied on self-report or observer ratings. These methods face limitations on accurately and reliably assessing teachers’ motivational behaviour; thus restricting the pace and scale of conducting research. One potential method to overcome these restrictions is automated coding methods. These methods are capable of analysing behaviour at a large scale with less time and at low costs. In this thesis, I conducted three studies to examine the applications of an automated coding method to assess teacher motivational behaviours. First, I systematically reviewed the applications of automated coding methods used to analyse helping professionals’ interpersonal interactions using their verbal behaviour. The findings showed that automated coding methods were used in psychotherapy to predict the codes of a well-developed behavioural coding measure, in medical settings to predict conversation patterns or topics, and in education to predict simple concepts, such as the number of open/closed questions or class activity type (e.g., group work or teacher lecturing). In certain circumstances, these models achieved near human level performance. However, few studies adhered to best-practice machine learning guidelines. Second, I developed a dictionary of teachers’ motivational phrases and used it to automatically assess teachers’ motivating and de-motivating behaviours. Results showed that the dictionary ratings of teacher need support achieved a strong correlation with observer ratings of need support (rfull dictionary = .73). Third, I developed a classification of teachers’ motivational behaviour that would enable more advanced automated coding of teacher behaviours at each utterance level. In this study, I created a classification that includes 57 teacher motivating and de-motivating behaviours that are consistent with self-determination theory. Automatically assessing teachers’ motivational behaviour with automatic coding methods can provide accurate, fast pace, and large scale analysis of teacher motivational behaviour. This could allow for immediate feedback and also development of theoretical frameworks. The findings in this thesis can lead to the improvement of student motivation and other consequent student outcomes

Filtering News from Document Streams: Evaluation Aspects and Modeled Stream Utility

Events like hurricanes, earthquakes, or accidents can impact a large number of people. Not only are people in the immediate vicinity of the event affected, but concerns about their well-being are shared by the local government and well-wishers across the world. The latest information about news events could be of use to government and aid agencies in order to make informed decisions on providing necessary support, security and relief. The general public avails of news updates via dedicated news feeds or broadcasts, and lately, via social media services like Facebook or Twitter. Retrieving the latest information about newsworthy events from the world-wide web is thus of importance to a large section of society. As new content on a multitude of topics is continuously being published on the web, specific event related information needs to be filtered from the resulting stream of documents. We present in this thesis, a user-centric evaluation measure for evaluating systems that filter news related information from document streams. Our proposed evaluation measure, Modeled Stream Utility (MSU), models users accessing information from a stream of sentences produced by a news update filtering system. The user model allows for simulating a large number of users with different characteristic stream browsing behavior. Through simulation, MSU estimates the utility of a system for an average user browsing a stream of sentences. Our results show that system performance is sensitive to a user population's stream browsing behavior and that existing evaluation metrics correspond to very specific types of user behavior. To evaluate systems that filter sentences from a document stream, we need a set of judged sentences. This judged set is a subset of all the sentences returned by all systems, and is typically constructed by pooling together the highest quality sentences, as determined by respective system assigned scores for each sentence. Sentences in the pool are manually assessed and the resulting set of judged sentences is then used to compute system performance metrics. In this thesis, we investigate the effect of including duplicates of judged sentences, into the judged set, on system performance evaluation. We also develop an alternative pooling methodology, that given the MSU user model, selects sentences for pooling based on the probability of a sentences being read by modeled users. Our research lays the foundation for interesting future work for utilizing user-models in different aspects of evaluation of stream filtering systems. The MSU measure enables incorporation of different user models. Furthermore, the applicability of MSU could be extended through calibration based on user behavior

Mapping numerical magnitude into behaviour

The importance of spatial models for numerical representations and the functional relation between number and space in the parietal cortex are suggested by the evidence that numerical information may affect spatial processing. It has been hypothesized that number maps onto a unidimensional continuum, the mental number line, and that number and space share a common metric. An investigation of the metric for numerical magnitudes, whether it is shared with space, and how this relation is reflected in behaviour, represent the main topic of the thesis. The hypothesis of shared metric is evaluated by the experimental work in the context of two topics: a) the subjective scale for numerical representation and b) the origin of spatial numerical interactions in visuomotor behaviour. Chapter 2 addresses an issue whether number, similarly to some physical magnitudes, may be represented on the logarithmically scaled continuum. The method for differentiating between logarithmic and linear hypotheses about the scale for number is implemented using novel variants of the number-line task, with results supporting the linear scaling schema. In Chapter 3, the method of transcranial magnetic stimulation was used to investigate whether the parietal areas, known to process numerical distance and allegedly implementing the mental number line, are involved in ratio scale computations, which are not compatible with mental number line model. Chapter 4 proposes a structural similarity between scales for number and space as a criterion to support the common metric between number and space. The scale analysis of number mapping onto space demonstrated discrepancy between spatial and numerical metrics for the performance in the manual estimation. Chapter 5 was designed to differentiate between the effect of number on the automatic visuomotor adaptation and on the response selection. The results show no evidence for the effect of number on the on-line motor corrections but reveal the signatures of non-sequential number mapping onto space at the stage of response selection. The findings in Chapter 5 are contrasted with the findings from Chapter 6, showing a pronounced effect of spatially non-specific expectations on the speed of the visuomotor coordination and spatial discrimination. The overall results do not support the hypothesis of the common metric for number and space and suggest that spatial models for number are deployed flexibly according to task demands