Understanding of Object Manipulation Actions Using Human Multi-Modal Sensory Data

Object manipulation actions represent an important share of the Activities of Daily Living (ADLs). In this work, we study how to enable service robots to use human multi-modal data to understand object manipulation actions, and how they can recognize such actions when humans perform them during human-robot collaboration tasks. The multi-modal data in this study consists of videos, hand motion data, applied forces as represented by the pressure patterns on the hand, and measurements of the bending of the fingers, collected as human subjects performed manipulation actions. We investigate two different approaches. In the first one, we show that multi-modal signal (motion, finger bending and hand pressure) generated by the action can be decomposed into a set of primitives that can be seen as its building blocks. These primitives are used to define 24 multi-modal primitive features. The primitive features can in turn be used as an abstract representation of the multi-modal signal and employed for action recognition. In the latter approach, the visual features are extracted from the data using a pre-trained image classification deep convolutional neural network. The visual features are subsequently used to train the classifier. We also investigate whether adding data from other modalities produces a statistically significant improvement in the classifier performance. We show that both approaches produce a comparable performance. This implies that image-based methods can successfully recognize human actions during human-robot collaboration. On the other hand, in order to provide training data for the robot so it can learn how to perform object manipulation actions, multi-modal data provides a better alternative

Elongated Intimacy: The intimate experience of owning / commissioning a craft object

‘How will you (craftspeople) make things that others will value, give a place in their intimate space and include in the rituals of their daily life?’ (Unger 2007) Little has been written in either social science or material culture research about the way contemporary craft objects are encountered and consumed and the meanings and values that they subsequently inherit. In my research as a silversmith and jeweller the made object embodies a set of intentions with symbolic significance and narrative agendas. Until now only anecdotal data existed to support whether the reception was equal to the intentions. This paper reports on the findings of primary empirical data gathered through intimate in-depth interviews. The respondents (unlike many studies) were invited to participate because they had purchased, commissioned or acquired an object created by the author. The complex results elicited knowledge about the life of the objects and the values and meanings they hold for those who own them. The findings are presented in the context of current critical debate in contemporary craft and describe how they inform creative practice.</p

Spartan Daily, February 5, 1936

Volume 24, Issue 75https://scholarworks.sjsu.edu/spartandaily/2403/thumbnail.jp

SmartFABER: Recognizing fine-grained abnormal behaviors for early detection of mild cognitive impairment

Objective: In an ageing world population more citizens are at risk of cognitive impairment, with negative consequences on their ability of independent living, quality of life and sustainability of healthcare systems. Cognitive neuroscience researchers have identified behavioral anomalies that are significant indicators of cognitive decline. A general goal is the design of innovative methods and tools for continuously monitoring the functional abilities of the seniors at risk and reporting the behavioral anomalies to the clinicians. SmartFABER is a pervasive system targeting this objective. Methods: A non-intrusive sensor network continuously acquires data about the interaction of the senior with the home environment during daily activities. A novel hybrid statistical and knowledge-based technique is used to analyses this data and detect the behavioral anomalies, whose history is presented through a dashboard to the clinicians. Differently from related works, SmartFABER can detect abnormal behaviors at a fine-grained level. Results: We have fully implemented the system and evaluated it using real datasets, partly generated by performing activities in a smart home laboratory, and partly acquired during several months of monitoring of the instrumented home of a senior diagnosed with MCI. Experimental results, including comparisons with other activity recognition techniques, show the effectiveness of SmartFABER in terms of recognition rates

Collect Her

Many writers, including Virgina Woolf and Marcel Proust, structure memory out of the space and furnishings of the home. Drawers, beds ands cupboards become secret hiding places for untold personal stories.......