Gaze-based Attention Recognition for Human-Robot Collaboration

Attention (and distraction) recognition is a key factor in improving human-robot collaboration. We present an assembly scenario where a human operator and a cobot collaborate equally to piece together a gearbox. The setup provides multiple opportunities for the cobot to adapt its behavior depending on the operator's attention, which can improve the collaboration experience and reduce psychological strain. As a first step, we recognize the areas in the workspace that the human operator is paying attention to, and consequently, detect when the operator is distracted. We propose a novel deep-learning approach to develop an attention recognition model. First, we train a convolutional neural network to estimate the gaze direction using a publicly available image dataset. Then, we use transfer learning with a small dataset to map the gaze direction onto pre-defined areas of interest. Models trained using this approach performed very well in leave-one-subject-out evaluation on the small dataset. We performed an additional validation of our models using the video snippets collected from participants working as an operator in the presented assembly scenario. Although the recall for the Distracted class was lower in this case, the models performed well in recognizing the areas the operator paid attention to. To the best of our knowledge, this is the first work that validated an attention recognition model using data from a setting that mimics industrial human-robot collaboration. Our findings highlight the need for validation of attention recognition solutions in such full-fledged, non-guided scenarios.Comment: Accepted to PETRA 202

Dense-core granules: a specific hallmark of the neuronal/neurosecretory cell phenotype.

Expression of dense-core granules, a typical exocytic organelle, is widely believed to be controlled by coordinate gene expression mechanisms specific to neurones and neurosecretory cells. Recent studies in PC12 cells, however, have suggested the number of granules/cells depends on the levels of only one of their cargo proteins, chromogranin A, regulating the metabolism of the other proteins, and thus the composition of the organelles, by an on/off switch mechanism. In addition, transfection of chromogranin A was reported to induce appearance of dense-core granules in the non-neurosecretory fibroblasts of the CV-1 line. Here the role of chromogranin A has been reinvestigated using not the heterogeneous PC12 line but several clones isolated therefrom. In these clones, investigated as such or after transfection with chromogranin A antisense sequences, the ratio between chromogranin A and its secretory protein mate, chromogranin B, was not constant but highly and apparently randomly variable. Variability of the chromogranin A/chromogranin B ratio was seen by confocal immunofluorescence also among the cells of single clones and subclones and among the granules of single cells. Moreover, stable and transient transfections of chromogranin A in a PC12 clone characterised by a low number of dense-core granules (one fifth of the reference clone) failed to modify significantly the number of the organelles, despite the several-fold increase of the granin. Finally, in three types of non-neurosecretory cells (CV-1, adenocarcinoma TS/A and a clone of PC12 incompetent for secretion) the transfected chromogranin A accumulated mostly in the Golgi/transGolgi area and was released rapidly from resting cells (constitutive secretion) as revealed by both immunofluorescence during cycloheximide treatment and pulse-chase experiments. Only a minor fraction was sorted to discrete organelles that were not dense-core granules, but primarily lysosomes because they contained no chromogranin B, and were largely positive for the late endosomal-lysosomal markers, lamp1 and lamp3. Dense-core granules are therefore true hallmarks of neurones and neurosecretory cells. Their number/cell appears independent of chromogranin A and their composition does not appear to be constant; in particular, they exhibit considerable, and so far unexplained variability in the chromogranin A/chromogranin B ratio

The effects of robotic assistance on upper limb spatial muscle synergies in healthy people during planar upper-limb training

Background Robotic rehabilitation is a commonly adopted technique used to restore motor functionality of neurological patients. However, despite promising results were achieved, the effects of human-robot interaction on human motor control and the recovery mechanisms induced with robot assistance can be further investigated even on healthy subjects before translating to clinical practice. In this study, we adopt a standard paradigm for upper-limb rehabilitation (a planar device with assistive control) with linear and challenging curvilinear trajectories to investigate the effect of the assistance in human-robot interaction in healthy people. Methods Ten healthy subjects were instructed to perform a large set of radial and curvilinear movements in two interaction modes: 1) free movement (subjects hold the robot handle with no assistance) and 2) assisted movement (with a force tunnel assistance paradigm). Kinematics and EMGs from representative upper-limb muscles were recorded to extract phasic muscle synergies. The free and assisted interaction modes were compared assessing the level of assistance, error, and muscle synergy comparison between the two interaction modes. Results It was found that in free movement error magnitude is higher than with assistance, proving that task complexity required assistance also on healthy controls. Moreover, curvilinear tasks require more assistance than standard radial paths and error is higher. Interestingly, while assistance improved task performance, we found only a slight modification of phasic synergies when comparing assisted and free movement. Conclusions We found that on healthy people, the effect of assistance was significant on task performance, but limited on muscle synergies. The findings of this study can find applications for assessing human-robot interaction and to design training to maximize motor recovery

Towards social embodied cobots: The integration of an industrial cobot with a social virtual agent

The integration of the physical capabilities of an industrial collaborative robot with a social virtual character may represent a viable solution to enhance the workers' perception of the system as an embodied social entity and increase social engagement and well-being at the workplace. An online study was setup using prerecorded video interactions in order to pilot potential advantages of different embodied configurations of the cobot-avatar system in terms of perceptions of Social Presence, cobot-avatar Unity and Social Role of the system, and explore the relation of these. In particular, two different configurations were explored and compared: the virtual character was displayed either on a tablet strapped onto the base of the cobot or on a large TV screen positioned at the back of the workcell. The results imply that participants showed no clear preference based on the constructs, and both configurations fulfill these basic criteria. In terms of the relations between the constructs, there were strong correlations between perception of Social Presence, Unity and Social Role (Collegiality). This gives a valuable insight into the role of these constructs in the perception of cobots as embodied social entities, and towards building cobots that support well-being at the workplace

Biomechanical Assessments of the Upper Limb for Determining Fatigue, Strain and Effort from the Laboratory to the Industrial Working Place: A Systematic Review

Recent human-centered developments in the industrial field (Industry 5.0) lead companies and stakeholders to ensure the wellbeing of their workers with assessments of upper limb performance in the workplace, with the aim of reducing work-related diseases and improving awareness of the physical status of workers, by assessing motor performance, fatigue, strain and effort. Such approaches are usually developed in laboratories and only at times they are translated to on-field applications; few studies summarized common practices for the assessments. Therefore, our aim is to review the current state-of-the-art approaches used for the assessment of fatigue, strain and effort in working scenarios and to analyze in detail the differences between studies that take place in the laboratory and in the workplace, in order to give insights on future trends and directions. A systematic review of the studies aimed at evaluating the motor performance, fatigue, strain and effort of the upper limb targeting working scenarios is presented. A total of 1375 articles were found in scientific databases and 288 were analyzed. About half of the scientific articles are focused on laboratory pilot studies investigating effort and fatigue in laboratories, while the other half are set in working places. Our results showed that assessing upper limb biomechanics is quite common in the field, but it is mostly performed with instrumental assessments in laboratory studies, while questionnaires and scales are preferred in working places. Future directions may be oriented towards multi-domain approaches able to exploit the potential of combined analyses, exploitation of instrumental approaches in workplace, targeting a wider range of people and implementing more structured trials to translate pilot studies to real practice

The effects of robotic assistance on upper limb spatial muscle synergies in healthy people during planar upper-limb training

BackgroundRobotic rehabilitation is a commonly adopted technique used to restore motor functionality of neurological patients. However, despite promising results were achieved, the effects of human-robot interaction on human motor control and the recovery mechanisms induced with robot assistance can be further investigated even on healthy subjects before translating to clinical practice. In this study, we adopt a standard paradigm for upper-limb rehabilitation (a planar device with assistive control) with linear and challenging curvilinear trajectories to investigate the effect of the assistance in human-robot interaction in healthy people.MethodsTen healthy subjects were instructed to perform a large set of radial and curvilinear movements in two interaction modes: 1) free movement (subjects hold the robot handle with no assistance) and 2) assisted movement (with a force tunnel assistance paradigm). Kinematics and EMGs from representative upper-limb muscles were recorded to extract phasic muscle synergies. The free and assisted interaction modes were compared assessing the level of assistance, error, and muscle synergy comparison between the two interaction modes.ResultsIt was found that in free movement error magnitude is higher than with assistance, proving that task complexity required assistance also on healthy controls. Moreover, curvilinear tasks require more assistance than standard radial paths and error is higher. Interestingly, while assistance improved task performance, we found only a slight modification of phasic synergies when comparing assisted and free movement.ConclusionsWe found that on healthy people, the effect of assistance was significant on task performance, but limited on muscle synergies. The findings of this study can find applications for assessing human-robot interaction and to design training to maximize motor recovery

Expression of the neurosecretory process in pc12 cells is governed by rest

The neurosecretory process is acquired during differentiation and can be lost en block by differentiated cells. To investigate the role of REST/NRSF, a transcription repressor, in the maintenance of the process we studied two PC12 clones, one wt and one defective, expressing low and high levels of endogenous RE-1 silencing transcription (factor) (REST), respectively. Stable transfection of constructs demonstrated that REST represses 10 genes coding for proteins of neurosecretory vesicles and their exocytosis, eight including and two lacking the REST-binding sequence, RE-1. Of these genes, those of chromogranins were strongly repressed by fewfold increases of REST, those of VAMP2 and syntaxin1a required much higher levels. Moreover, in wt cells transfected with an active construct the dense-core vesicles, still competent for regulated exocytosis, were much smaller, with lighter cores; in defective cells, the dominant-negative construct induced the rescue of many vesicle/exocytosis genes but not of those of chromogranins. Small dense-core vesicles, exocytized upon stimulation, were rescued when the construct-transfected defective cells were transfected also with chromograninA or treated with trichostatinA, a blocker of histone deacetylases. Our results identify REST, working by direct and indirect mechanisms, as the factor governing the maintenance of the neurosecretory process and the properties of dense-core vesicles in PC12 cells

Safe obstacle avoidance for industrial robot working without fences

Abstract-Until now, the presence of fences is a technological barrier for the adoption of robots in Small Medium Enterprises (SME). The work deals with the definition of an intrinsically safe algorithm to avoid collisions between an industrial manipulator and obstacles in its workspace (Standard ISO 10218-1). The suggested strategy aims to offer an industrial solution to the problem: an off-line analysis of the workspace is performed to have an exhaustive and intrinsically description of the static obstacles and a safe spatial grid of "pass-through points" is calculated; an on-line algorithm, based on an enhanced Artificial Potential Field evaluates the most suitable points to avoid collisions against obstacles and perform a realtime replanning the path of the robot. A Matlab toolbox that elaborates STL CAD files has been developed to obtain a full description of the workcell, and the avoidance algorithm has been designed and implemented in a standard industrial controller. Various experimental results are reported by using a COMAU NS16 arm manipulator

Kinect V2 Performance Assessment in Daily-Life Gestures: Cohort Study on Healthy Subjects for a Reference Database for Automated Instrumental Evaluations on Neurological Patients

Background. The increase of sanitary costs related to poststroke rehabilitation requires new sustainable and cost-effective strategies for promoting autonomous and dehospitalized motor training. In the Riprendo@Home and Future Home for Future Communities research projects, the promising approach of introducing low-cost technologies that promote home rehabilitation is exploited. In order to provide reliable evaluation of patients, a reference database of healthy people’s performances is required and should consider variability related to healthy people performances. Methods. 78 healthy subjects performed several repetitions of daily-life gestures, the reaching movement (RM) and hand-to-mouth (HtMM) movement with both the dominant and nondominant upper limbs. Movements were recorded with a Kinect V2. A synthetic biomechanical protocol based on kinematical, dynamical, and motor control parameters was used to assess motor performance of the healthy people. The investigation was conducted by clustering participants depending on their limb dominancy (right/left), gender (male/female), and age (young/middle/senior) as sources of variability. Results. Results showed that limb dominancy has minor relevance in affecting RM and HtMM; gender has relevance in affecting the HtMM; age has major effect in affecting RM and HtMM. Conclusions. An investigation of healthy subjects’ upper limb performances during daily-life gestures was performed with the Kinect V2 sensor. Findings will be the basis for a database of normative data for neurological patients’ motor evaluation

Intracerebral Injection of Extracellular Vesicles from Mesenchymal Stem Cells Exerts Reduced Aβ Plaque Burden in Early Stages of a Preclinical Model of Alzheimer's Disease

Bone marrow Mesenchymal Stem Cells (BM-MSCs), due to their strong protective and anti-inflammatory abilities, have been widely investigated in the context of several diseases for their possible therapeutic role, based on the release of a highly proactive secretome composed of soluble factors and Extracellular Vesicles (EVs). BM-MSC-EVs, in particular, convey many of the beneficial features of parental cells, including direct and indirect \u3b2-amyloid degrading-activities, immunoregulatory and neurotrophic abilities. Therefore, EVs represent an extremely attractive tool for therapeutic purposes in neurodegenerative diseases, including Alzheimer's disease (AD). We examined the therapeutic potential of BM-MSC-EVs injected intracerebrally into the neocortex of APPswe/PS1dE9 AD mice at 3 and 5 months of age, a time window in which the cognitive behavioral phenotype is not yet detectable or has just started to appear. We demonstrate that BM-MSC-EVs are effective at reducing the A\u3b2 plaque burden and the amount of dystrophic neurites in both the cortex and hippocampus. The presence of Neprilysin on BM-MSC-EVs, opens the possibility of a direct \u3b2-amyloid degrading action. Our results indicate a potential role for BM-MSC-EVs already in the early stages of AD, suggesting the possibility of intervening before overt clinical manifestations