Do Deep Generative Models Know What They Don't Know?

A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data. A plethora of work has demonstrated that it is easy to find or synthesize inputs for which a neural network is highly confident yet wrong. Generative models are widely viewed to be robust to such mistaken confidence as modeling the density of the input features can be used to detect novel, out-of-distribution inputs. In this paper we challenge this assumption. We find that the density learned by flow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses (i.e. CIFAR-10) from those of house numbers (i.e. SVHN), assigning a higher likelihood to the latter when the model is trained on the former. Moreover, we find evidence of this phenomenon when pairing several popular image data sets: FashionMNIST vs MNIST, CelebA vs SVHN, ImageNet vs CIFAR-10 / CIFAR-100 / SVHN. To investigate this curious behavior, we focus analysis on flow-based generative models in particular since they are trained and evaluated via the exact marginal likelihood. We find such behavior persists even when we restrict the flows to constant-volume transformations. These transformations admit some theoretical analysis, and we show that the difference in likelihoods can be explained by the location and variances of the data and the model curvature. Our results caution against using the density estimates from deep generative models to identify inputs similar to the training distribution until their behavior for out-of-distribution inputs is better understood.Comment: ICLR 201

Using simulations and artificial life algorithms to grow elements of construction

'In nature, shape is cheaper than material', that is a common truth for most of the plants and other living organisms, even though they may not recognize that. In all living forms, shape is more or less directly linked to the influence of force, that was acting upon the organism during its growth. Trees and bones concentrate their material where thy need strength and stiffness, locating the tissue in desired places through the process of self-organization. We can study nature to find solutions to design problems. That’s where inspiration comes from, so we pick a solution already spotted somewhere in the organic world, that closely resembles our design problem, and use it in constructive way. First, examining it, disassembling, sorting out conclusions and ideas discovered, then performing an act of 'reverse engineering' and putting it all together again, in a way that suits our design needs. Very simple ideas copied from nature, produce complexity and exhibit self-organization capabilities, when applied in bigger scale and number. Computer algorithms of simulated artificial life help us to capture them, understand well and use where needed. This investigation is going to follow the question : How can we use methods seen in nature to simulate growth of construction elements? Different ways of extracting ideas from world of biology will be presented, then several techniques of simulated emergence will be demonstrated. Specific focus will be put on topics of computational modelling of natural phenomena, and differences in developmental and non-developmental techniques. Resulting 3D models will be shown and explained

What Can Artificial Intelligence Do for Scientific Realism?

The paper proposes a synthesis between human scientists and artificial representation learning models as a way of augmenting epistemic warrants of realist theories against various anti-realist attempts. Towards this end, the paper fleshes out unconceived alternatives not as a critique of scientific realism but rather a reinforcement, as it rejects the retrospective interpretations of scientific progress, which brought about the problem of alternatives in the first place. By utilising adversarial machine learning, the synthesis explores possibility spaces of available evidence for unconceived alternatives providing modal knowledge of what is possible therein. As a result, the epistemic warrant of synthesised realist theories should emerge bolstered as the underdetermination by available evidence gets reduced. While shifting the realist commitment away from theoretical artefacts towards modalities of the possibility spaces, the synthesis comes out as a kind of perspectival modelling

Developing Generative Leadership through Emergent Learning

This thesis is the current synthesis of a deep exploration of the foundations of collaborative, transformational learning within organizations. I begin with a basic assumption which informs all the thinking that unfolds throughout this thesis: the sustainability of our organizations, and quite possibly the survival of our species, is dependent not on the leadership and the development of a chosen few, but on our collective ability to deeply listen for and sense what most needs to happen within a given group of people and then to act on this. We live our lives with deeply entrenched, mostly tacit beliefs about deferring to experts and the need for strong, charismatic leaders. These tacit beliefs have largely disempowered and disconnected us from accessing our most fully creative, generative selves. The deepest reservoirs of learning are found in collaborative, emergent learning experiences. In essence, the question becomes: what can happen when groups of people gather together as teachers and learners to share their thinking, their imaginings, their hopes and fears? What new thinking can be born? And how might this impact our sense of leadership and collective action? There are many forms which emergent learning can take. Contemporary structures for emergent learning have many of their roots in the group sensitivity training movement of the 1960s and \u2770s. Present structures for emergent learning include: the dialogue process, Community Building, Open Space Technology and various hybrid forms of both verbal and non-verbal collaborative, co-creative processes. The essence of \u27emergent learning is an experiential immersion in many of the foundational skills of critical and creative thinking: systems thinking, metacognition, inquiry, empathic and reflective listening, and seeing from multiple perspectives. While emergent learning structures can have many purposes, I believe the greatest value of these learning experiences is developing the capacity for what I refer to as generative leadership. Generative leadership is about developing what I call advanced group sensitivities -- listening for what is wanting and needing to happen within the collective and then having the courage to act on this. It is about engendering a new quality of leadership within organizations -- unfolding, shared leadership as an alternative to traditional, hierarchical control, and authority

Stative sentences in Japanese and the role of the nominative marker "ga" : a thesis submitted in partial fulfillment of the requirements for the degree of Master of Arts in Japanese at Massey University, Palmerston North, New Zealand

The Japanese nominative particle ga is normally associated with the marking of subjects. However, there are several constructions involving stative predicates, where it has been claimed, notably by those working within a generative framework, that a ga-marked NP can be an object and that such sentences are transitive. Such an analysis has particularly arisen in the case of sentences with more than one ga-marked NP, exhibiting so-called double ga marking. The following study makes two claims. Firstly, that one of the functions of ga in such sentences is to provide a discourse frame akin to the topic marking function of the postpositional particle wa. Secondly it argues that stative sentences associated with double ga-marking are in fact intransitive and that the ga-marked NP's that have been claimed to be objects are in fact subjects

Chatbots with Personality Using Deep Learning

Natural Language Processing (NLP) requires the computational modelling of the complex relationships of the syntax and semantics of a language. While traditional machine learning methods are used to solve NLP problems, they cannot imitate the human ability for language comprehension. With the growth in deep learning, these complexities within NLP are easier to model, and be used to build many computer applications. A particular example of this is a chatbot, where a human user has a conversation with a computer program, that generates responses based on the user’s input. In this project, we study the methods used in building chatbots, what they lack and what can be improved

Conversational Agent: Developing a Model for Intelligent Agents with Transient Emotional States

The inclusion of human characteristics (i.e., emotions, personality) within an intelligent agent can often increase the effectiveness of information delivery and retrieval. Chat-bots offer a plethora of benefits within an eclectic range of disciplines (e.g., education, medicine, clinical and mental health). Hence, chatbots offer an effective way to observe, assess, and evaluate human communication patterns. Current research aims to develop a computational model for conversational agents with an emotional component to be applied to the army leadership training program that will allow for the examination of interpersonal skills in future research. Overall, the current research explores the application of the deep learning algorithm to the development of a generalized framework that will be based upon modeling empathetic conversation between an intelligent conversational agent (chatbot) and a human user in order to allow for higher level observation of interpersonal communication skills. Preliminary results demonstrate the promising potential of the seq2seq technique (e.g., through the use of Dialog Flow Chatbot platform) when applied to emotion-oriented conversational tasks. Both the classification and generative conversational modeling tasks demonstrate the promising potential of the current research for representing human to agent dialogue. However, this implementation may be extended by utilizing, a larger more high-quality dataset

Masculinity and Social Change: Exploring Generative Masculinity Development in Resident Assistant Men through the Social Change Model of Leadership Development

In this study, men’s identity development among Resident Assistants (RAs) at Louisiana State University is investigated using a constructivist approach. Societal expectations of men tend to value hegemonic masculinity, which reinforces a drive for dominance, objectification, and high-risk behaviors (Edwards & Jones, 2009). Whereas, generative masculinity is characterized by a sense of responsibility, desire to give back, comfort with self, willingness to confront and break gender stereotypes, and the use of personal strengths to foster wellbeing (Badaszewski, 2014). Many characteristics of generative masculinity align with the Seven C’s of Social Change as described in the Social Change Model of Leadership Development. The Social Change Model is designed to describe how students cultivate leadership skills though service to others (Higher Education Research Institute, 1996). Resident Assistants (RAs) serve as mentors and role models to students living on campus, help to foster community amongst on-campus student residents, and enforce building security. For the purposes of this study, the researcher uses the Social Change Model of Leadership Development to examine how being a Resident Assistant contributes to the generative masculinity development of RA men