How do humans mediate with the external physical world? From perception to control of articulated objects

Many actions in our daily life involve operation with articulated tools. Despite the ubiquity of articulated objects in daily life, human ability in perceiving the properties and control of articulated objects has been merely studied. Articulated objects are composed of links and revolute or prismatic joints. Moving one part of the linkage results in the movement of the other ones. Reaching a position with the tip of a tool requires adapting the motor commands to the change of position of the endeffector different from the action of reaching the same position with the hand. The dynamic properties are complex and variant in the movement of articulated bodies. For instance, apparent mass, a quantity that measures the dynamic interaction of the articulated object, varies as a function of the changes in configuration. An actuated articulated system can generate a static, but position-dependent force field with constant torques about joints. There are evidences that internal models are involved in the perception and control of tools. In the present work, we aim to investigate several aspects of the perception and control of articulated objects and address two questions, The first question is how people perceive the kinematic and dynamic properties in the haptic interaction with articulated objects? And the second question is what effect has seeing the tool on the planning and execution of reaching movements with a complex tool? Does the visual representation of mechanism structures help in the reaching movement and how? To address these questions, 3D printed physical articulated objects and robotic systems have been designed and developed for the psychophysical studies. The present work involves three studies in different aspects of perception and control of articulated objects. We first did haptic size discrimination tasks using three different types of objects, namely, wooden boxes, actuated apparatus with two movable flat surfaces, and large-size pliers, in unimanual, bimanual grounded and bimanual free conditions. We found bimanual integration occurred in particular in the free manipulation of objects. The second study was on the visuo-motor reaching with complex tools. We found that seeing the mechanism of the tool, even briefly at the beginning of the trial, improved the reaching performance. The last study was about force perception, evidences showed that people could take use of the force field at the end-effector to induce the torque about the joints generated by the articulated system

How the brain controls hand actions: TMS, fMRI and behavioural studies

This thesis focused on testing the predictions made in Milner and Goodale’s model and reports finding from experiments investigating how inputs from both the dorsal and the ventral streams are required when we perform hand actions with objects (Chapter 2) and tools (Chapter 3 & 4) using different paradigms such as real and pantomimed grasping and techniques such as transcranial magnetic stimulation, motion-tracking of hand movements and cutting-edge fMRI multivoxel pattern analysis. The primary aim was to gain a new insight on the role of the dorsal and the ventral visual streams in real grasping and pantomiming and to understand what specific aspects of objects and movements associated with them are represented within the two streams. The first experiment (Chapter 2) examined the causal role of the anterior intraparietal and the lateral occipital in object’s real and pantomimed grasping using TMS. The results showed that real object grasping and pantomime actions without the objects in hand require the left dorsal stream but that information from the ventral stream is additionally required for pantomiming. The experiments in Chapter 3 and 4 investigated how tools and tool related actions are represented within the dorsal and the ventral stream (Chapter 3) and whether different action end-goals affected early grasping kinematics (Chapter 4). Using MVPA we showed that both dorsal and ventral stream regions represent information about functional and structural manipulation knowledge of tools. Moreover, we showed that both streams represent tool identity, which seems in line with our behavioural findings that tool identity affects grasping kinematics. The current work provided a detailed understanding of how the dorsal and the ventral streams interact in tool processing and propose a more sophisticated view of the distributed representations across the two streams. These findings open up a number of research avenues as well as help understanding how actions are disrupted in brain-damaged patients and advance the development of neural prosthetics

Robot manipulation in human environments

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.Includes bibliographical references (p. 211-228).Human environments present special challenges for robot manipulation. They are often dynamic, difficult to predict, and beyond the control of a robot engineer. Fortunately, many characteristics of these settings can be used to a robot's advantage. Human environments are typically populated by people, and a robot can rely on the guidance and assistance of a human collaborator. Everyday objects exhibit common, task-relevant features that reduce the cognitive load required for the object's use. Many tasks can be achieved through the detection and control of these sparse perceptual features. And finally, a robot is more than a passive observer of the world. It can use its body to reduce its perceptual uncertainty about the world. In this thesis we present advances in robot manipulation that address the unique challenges of human environments. We describe the design of a humanoid robot named Domo, develop methods that allow Domo to assist a person in everyday tasks, and discuss general strategies for building robots that work alongside people in their homes and workplaces.by Aaron Ladd Edsinger.Ph.D

Haptic working memory: performance in interference tasks and span tasks with everyday objects

Tese de doutoramento em Psicologia Especialidade em Psicologia Experimental e Ciências CognitivasIn this work, human haptic working memory is analysed in interference and span tasks, aiming to systematically study memory for touch in natural conditions, and to examine how haptic information is encoded and processed in human memory. The first part of this thesis consists in a literature review of data on haptic perception and haptic memory. Our approach to this review implied a systematisation of the main variables manipulated in tactile and haptic studies, followed by a critical review of studies on memory for touch. The first group of experiments, evaluating the impact of interference tasks in haptic recognition is described in Part II. Participants were required to sequentially touch lists of 50 common (e.g., comb) or uncommon (e.g., plastic onion) everyday objects, either in single task or interference task conditions, and then were asked to perform an immediate incidental haptic recognition memory task, by touching a new list of 50 objects (25 presented and 25 non-presented objects) and to indicate if each object had been touched before. The interference tasks were always performed at encoding and could be haptic (evaluating paper samples), motor (performing a concurrent movement), verbal (evaluating pseudo-word pairs), or visuospatial (evaluating three-dimensional stimuli images). Results showed that participants were able to perform these tasks with a high accuracy rate, either for common or uncommon objects. For common objects, only haptic interference impaired haptic recognition. For uncommon objects, haptic recognition was affected by haptic, verbal and visual interference tasks, but not by motor interference. A final analysis, considering direct comparisons between object types for each interference condition revealed that common objects systematically present higher recognition rates, and are less affected by interference than uncommon objects. Overall, these studies suggest a haptic specificity in working memory, and an object familiarity mediation of the participant’s performance. The third part of this thesis reports a group of exploratory studies on haptic span, recurring to immediate serial recall and reconstruction of order tasks. Considering the relevance of verbal representations for everyday objects, the participant’s performance was studied in single task and articulatory suppression conditions. In immediate serial recall tasks participants were asked to touch lists of common objects (e.g., ball), starting with a list extension of two objects, and going up to ten objects, according to the participant’s performance. Results revealed an haptic span of approximately five items in single task and of four items in articulatory suppression conditions. In reconstruction of order tasks, after the list presentation, all touched objects were available again, and the participants had to order them according to the initial presentation. This task, not implying item identification (naming), allowed a comparison between common and uncommon object’s span. Results showed that similar spans exist for both conditions, with about six items being recalled in single task, and about five items in the articulatory suppression conditions. The present work is a contribute to the field of haptic cognition, specifically haptic memory, by presenting a first attempt to systematically study working memory for touch by adapting classical paradigms in the study of human memory to the haptic sensory modality. Throughout this work, empirical and theoretical topics regarding touch experiments are discussed and future research paths in this field are suggested.Neste trabalho analisa-se a memória de trabalho háptica em humanos, procurando estudá-la de forma sistemática e em condições naturais, explorando a forma como a informação háptica é codificada e processada na memória humana. A primeira parte desta tese consiste numa revisão da literatura existente nas áreas de percepção e memória táctil e háptica. A abordagem à revisão teórica passou pela sistematização das variáveis centrais nos estudos sobre tacto, seguida de uma revisão crítica do estudo da memória táctil. O primeiro grupo de experiências avalia o impacto de tarefas de interferência no reconhecimento háptico e é descrito na Parte II. Os participantes tocaram sequencialmente uma lista de 50 objectos quotidianos comuns (e.g., pente) ou incomuns (e.g., cebola de plástico), em condições de tarefa simples ou em condições de interferência. De seguida foi-lhes solicitada a realização de uma tarefa de reconhecimento háptico imediata e incidental, tocando uma nova lista de 50 objectos (25 apresentados previamente e 25 não apresentados) e indicando para cada objecto se este havia sido tocado anteriormente. A tarefa de interferência foi realizada durante a fase de codificação dos objectos e poderia ser de natureza háptica (avaliação de amostras de papel), motora (realizar um movimento específico), verbal (avaliação de pseudo-palavras), ou visuo-espacial (avaliação de imagens de estímulos tridimensionais). Os resultados demonstram que os participantes foram capazes de realizar a tarefa de reconhecimento háptico com elevadas taxas de acerto, quer para objectos comuns, quer para objectos incomuns. Para os objectos comuns, apenas a interferência háptica prejudicou o reconhecimento háptico. Para os objectos incomuns, o reconhecimento háptico foi prejudicado pela interferência háptica, verbal e visuo-espacial, mas não pela interferência motora. Uma última análise, comparando directamente o desempenho para objectos comuns e incomuns para cada tipo de interferência, revelou que os objectos comuns produzem sistematicamente melhores taxas de reconhecimento e são menos prejudicados pela interferência do que os objectos incomuns. De uma forma geral, os estudos apresentados na segunda parte deste trabalho sugerem a existência de especificidade háptica na memória de trabalho, assim como um efeito mediador da familiaridade dos estímulos no desempenho dos participantes. A terceira parte desta tese descreve um conjunto de estudos exploratórios sobre tarefas de capacidade de memória háptica, recorrendo a tarefas de evocação serial imediata e a tarefas de ordenação. Considerando a relevância das representações verbais para os objectos quotidianos, os participantes nestes estudos realizaram as tarefas de capacidade em condições de tarefa simples e com supressão articulatória Nas tarefas de evocação serial imediata, os participantes tocaram listas de objectos comuns (e.g., bola), começando a tarefa com listas com a extensão de dois objectos, que podiam ser aumentadas até um máximo de 10 objectos, dependendo do desempenho individual. Os resultados demonstram que a capacidade háptica é de aproximadamente cinco itens em condições de tarefa simples, e de quatro itens em condições de supressão articulatória. Nas tarefas de ordenação, após a apresentação das listas, todos os objectos tocados estão disponíveis para o participante, que deverá ordená-los de acordo com a apresentação inicial. Nesta tarefa, uma vez que não é necessária a identificação dos itens, foi possível comparar o desempenho com objectos comuns e incomuns. Os resultados mostram que o número de itens correctamente ordenados é idêntico para objectos comuns e incomuns e corresponde a aproximadamente seis itens em condições de tarefa simples e a cinco itens em condições de supressão articulatória. Este trabalho apresenta-se como um contributo para o campo da cognição háptica, especificamente da memória háptica, apresentando uma primeira tentativa de estudar sistematicamente a memória para o tacto, adaptando paradigmas clássicos no estudo da memória humana à modalidade sensorial háptica. Ao longo deste trabalho, tópicos empíricos e teóricos acerca das experiências com tacto serão discutidos e serão sugeridos caminhos de investigação futura nesta área

Perceptual Strategies and Neuronal Underpinnings underlying Pattern Recognition through Visual and Tactile Sensory Modalities in Rats

The aim of my PhD project was to investigate multisensory perception and multimodal recognition abilities in the rat, to better understand the underlying perceptual strategies and neuronal mechanisms. I have chosen to carry out this project on the laboratory rat, for two reasons. First, the rat is a flexible and highly accessible experimental model, where it is possible to combine state-of-the-art neurophysiological approaches (such as multi-electrode neuronal recordings) with behavioral investigation of perception and (more in general) cognition. Second, extensive research concerning multimodal integration has already been conducted in this species, both at the neurophysiological and behavioral level. My thesis work has been organized in two projects: a psychophysical assessment of object categorization abilities in rats, and a neurophysiological study of neuronal tuning in the primary visual cortex of anaesthetized rats. In both experiments, unisensory (visual and tactile) and multisensory (visuo-tactile) stimulation has been used for training and testing, depending on the task. The first project has required development of a new experimental rig for the study of object categorization in rat, using solid objects, so as to be able to assess their recognition abilities under different modalities: vision, touch and both together. The second project involved an electrophysiological study of rat primary visual cortex, during visual, tactile and visuo-tactile stimulation, with the aim of understanding whether any interaction between these modalities exists, in an area that is mainly deputed to one of them. The results of both of the studies are still preliminary, but they already offer some interesting insights on the defining features of these abilities

Neural and motor basis of inter-individual interactions

The goal of my Ph.D. work was to investigate the behavioral markers and the brain activities responsible for the emergence of sensorimotor communication. Sensorimotor communication can be defined as a form of communication consisting into flexible exchanges based on bodily signals, in order to increase the efficiency of the inter-individual coordination. For instance, a soccer player carving his movements to inform another player about his intention. This form of interaction is highly dependent of the motor system and the ability to produce appropriate movements but also of the ability of the partner to decode these cues. To tackle these facets of human social interaction, we approached the complexity of the problem by splitting my research activities into two separate lines of research. First, we pursued the examination of motor-based humans\u2019 capability to perceive and \u201cread\u201d other\u2019s behaviors in focusing on single-subject experiment. The discovery of mirror neurons in monkey premotor cortex in the early nineties (di Pellegrino et al. 1992) motivated a number of human studies on this topic (Rizzolatti and Craighero 2004). The critical finding was that some ventral premotor neurons are engaged during visual presentation of actions performed by conspecifics. More importantly, those neurons were shown to encode also the actual execution of similar actions (i.e. irrespective of who the acting individual is). This phenomenon has been highly investigated in humans by using cortical and cortico-spinal measures (for review see, fMRI: Molenberghs, Cunnington, and Mattingley 2012; TMS: Naish et al. 2014; EEG: Pineda 2008). During single pulse TMS (over the primary motor cortex), the amplitude of motor evoked potentials (MEPs) provides an index of corticospinal recruitment. During action observation the modulation of this index follow the expected changes during action execution (Fadiga et al. 1995). However, dozens of studies have been published on this topic and revealed important inconsistencies. For instance, MEPs has been shown to be dependent on observed low-level motor features (e.g. kinematic features or electromyography temporal coupling; Gangitano, Mottaghy, and Pascual-Leone 2001; Borroni et al. 2005; Cavallo et al. 2012) as well as high level movement properties (e.g. action goals; Cattaneo et al. 2009; Cattaneo et al. 2013). Furthermore, MEPs modulations do not seem to be related to the observed effectors (Borroni and Baldissera 2008; Finisguerra et al. 2015; Senna, Bolognini, and Maravita 2014), suggesting their independence from low-level movement features. These contradictions call for new paradigms. Our starting hypothesis here is that the organization and function of the mirror mechanism should follow that of the motor system during action execution. Hence, we derived three action observation protocols from classical motor control theories: 1) The first study was motivated by the fact that motor redundancy in action execution do not allow the presence of a one-to-one mapping between (single) muscle activation and action goals. Based on that, we showed that the effect of action observation (observation of an actor performing a power versus a precision grasp) are variable at the single muscle level (MEPs; motor evoked potentials) but robust when evaluating the kinematic of TMS-evoked movements. Considering that movements are based on the coordination of multiple muscle activations (muscular synergies), MEPs may represent a partial picture of the real corticospinal activation. Inversely, movement kinematics is both the final functional byproduct of muscles coordination and the sole visual feedback that can be extracted from action observation (i.e. muscle recruitment is not visible). We conclude that TMS-evoked kinematics may be more reliable in representing the state of the motor system during action observation. 2) In the second study, we exploited the inter-subject variability inherent to everyday whole-body human actions, to evaluate the link between individual motor signatures (or motor styles) and other\u2019s action perception. We showed no group-level effect but a robust correlation between the individual motor signature recorded during action execution and the subsequent modulations of corticospinal excitability during action observation. However, results were at odds with a strict version of the direct matching hypothesis that would suggest the opposite pattern. In fact, the more the actor\u2019s movement was similar to the observer\u2019s individual motor signature, the smaller was the MEPs amplitude, and vice versa. These results conform to the predictive coding hypothesis, suggesting that during AO, the motor system compares our own way of doing the action (individual motor signature) with the action displayed on the screen (actor\u2019s movement). 3) In the third study, we investigated the neural mechanisms underlying the visual perception of action mistakes. According to a strict version of the direct matching hypothesis, the observer should potentially reproduce the neural activation present during the actual execution of action errors (van Schie et al. 2004). Here, instead of observing an increase of cortical inhibition, we showed an early (120 ms) decrease of intracortical inhibition (short intracortical inhibition) when a mismatch was present between the observed action (erroneous) and the observer\u2019s expectation. As proposed by the predictive coding framework, the motor system may be involved in the generation of an error signal potentially relying on an early decrease of intracortical inhibition within the corticomotor system. The second line of research aimed at the investigation of how sensorimotor communication flows between agents engaged in a complementary action coordination task. In this regard, measures of interest where related to muscle activity and/or kinematics as the recording of TMS-related indexes would be too complicated in a joint-action scenario. 1) In the first study, we exploited the known phenomenon of Anticipatory Postural Adjustments (APAs). APAs refers to postural adjustments made in anticipation of a self- or externally-generated disturbance in order to cope for the predicted perturbation and stabilize the current posture. Here we examined how observing someone else lifting an object we hold can affect our own anticipatory postural adjustments of the arm. We showed that the visual information alone (joint action condition), in the absence of efference copy (present only when the subject is unloading by himself the object situated on his hand), were not sufficient to fully deploy the needed anticipatory muscular activations. Rather, action observation elicited a dampened APA response that is later augmented by the arrival of tactile congruent feedback. 2) In a second study, we recorded the kinematic of orchestra musicians (one conductor and two lines of violinists). A manipulation was added to perturb the normal flow of information conveyed by the visual channel. The first line of violinist where rotated 180\ub0, and thus faced the second line. Several techniques were used to extract inter-group (Granger Causality method) and intra-group synchronization (PCA for musicians and autoregression for conductors). The analyses were directed to two kinematic features, hand and head movements, which are central for functionally different action. The hand is essential for instrumental actions, whereas head movements encode ancillary expressive actions. During the perturbation, we observed a complete reshaping of the whole patterns of communication going in the direction of a distribution of the leadership between conductor and violinists, especially for what regards head movements. In fact, in the perturbed condition, the second line acts as an informational hub connecting the first line to the conductor they no longer can see. This study evidences different forms of communications (coordination versus synchronization) flowing via different channels (ancillary versus instrumental) with different time-scales

Applied aspirations : design and applied art at the Ballarat Technical Art School during the early twentieth century

Applied art and design schools operate at the nexus of art, industry, and education. During the early decades of the twentieth century, the regionally located Ballarat Technical Art School (BTAS) was the leading institution of its kind in Victoria, Australia, amid shifting economic, cultural, and pedagogical conditions. Emerging from a 1907 amalgamation of institutions, and subsequently administrated by the School of Mines Ballarat (SMB), BTAS was equipped with the assets, experience, and historic reputation necessary to surpass its provincial and metropolitan rivals. This micro-historical case-study employs qualitative analysis of primary sources to investigate the aims, outputs, and importance of BTAS, contextualised by the expectations and influences it operated under during the inaugural principalship of artist and educator, Herbert Henry Smith. Smith oversaw the training of designers, craftspeople, artists, and teachers from 1907 until his retirement in early 1940—a period of tumultuous events, fiscal obstacles, and social and cultural debate. The institution was accountable to diverse stakeholders and arbiters of taste, and successive cohorts learned in a contested space between tradition, origination, and modernisation. Pierre Bourdieu’s cultural theory serves to navigate this web of hierarchies, assumptions, and tensions, while secondary sources help contextualise findings. This thesis also discusses the suite of drawing, design and material-based disciplines delivered at BTAS as single subjects, full courses, and supplementary art-trade training. Throughout, featured students provide examples of regionally trained, Australian designer-maker and artist-teacher experiences. BTAS students learned from ambitious and skilled men and women, benefited from strong professional networks, and fostered a notable esprit-de-corps. The school was significant for its contribution to female technical training. The school’s pre-eminent position was modified during the late 1920s, when much art and art-teacher training was re-centred in Melbourne. Yet, the valuable, compelling, and widespread influence of Ballarat Technical Art School graduates resonated for decades.Doctor of Philosoph