CaloriNet: From silhouettes to calorie estimation in private environments

We propose a novel deep fusion architecture, CaloriNet, for the online estimation of energy expenditure for free living monitoring in private environments, where RGB data is discarded and replaced by silhouettes. Our fused convolutional neural network architecture is trainable end-to-end, to estimate calorie expenditure, using temporal foreground silhouettes alongside accelerometer data. The network is trained and cross-validated on a publicly available dataset, SPHERE_RGBD + Inertial_calorie. Results show state-of-the-art minimum error on the estimation of energy expenditure (calories per minute), outperforming alternative, standard and single-modal techniques.Comment: 11 pages, 7 figure

Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring Applications

The use of visual sensors for monitoring people in their living environments is critical in processing more accurate health measurements, but their use is undermined by the issue of privacy. Silhouettes, generated from RGB video, can help towards alleviating the issue of privacy to some considerable degree. However, the use of silhouettes would make it rather complex to discriminate between different subjects, preventing a subject-tailored analysis of the data within a free-living, multi-occupancy home. This limitation can be overcome with a strategic fusion of sensors that involves wearable accelerometer devices, which can be used in conjunction with the silhouette video data, to match video clips to a specific patient being monitored. The proposed method simultaneously solves the problem of Person ReID using silhouettes and enables home monitoring systems to employ sensor fusion techniques for data analysis. We develop a multimodal deep-learning detection framework that maps short video clips and accelerations into a latent space where the Euclidean distance can be measured to match video and acceleration streams. We train our method on the SPHERE Calorie Dataset, for which we show an average area under the ROC curve of 76.3% and an assignment accuracy of 77.4%. In addition, we propose a novel triplet loss for which we demonstrate improving performances and convergence speed

Meta-Learning with Context-Agnostic Initialisations

Meta-learning approaches have addressed few-shot problems by finding initialisations suited for fine-tuning to target tasks. Often there are additional properties within training data (which we refer to as context), not relevant to the target task, which act as a distractor to meta-learning, particularly when the target task contains examples from a novel context not seen during training. We address this oversight by incorporating a context-adversarial component into the meta-learning process. This produces an initialisation for fine-tuning to target which is both context-agnostic and task-generalised. We evaluate our approach on three commonly used meta-learning algorithms and two problems. We demonstrate our context-agnostic meta-learning improves results in each case. First, we report on Omniglot few-shot character classification, using alphabets as context. An average improvement of 4.3% is observed across methods and tasks when classifying characters from an unseen alphabet. Second, we evaluate on a dataset for personalised energy expenditure predictions from video, using participant knowledge as context. We demonstrate that context-agnostic meta-learning decreases the average mean square error by 30%

Temporal-Relational CrossTransformers for Few-Shot Action Recognition

We propose a novel approach to few-shot action recognition, finding temporally-corresponding frame tuples between the query and videos in the support set. Distinct from previous few-shot works, we construct class prototypes using the CrossTransformer attention mechanism to observe relevant sub-sequences of all support videos, rather than using class averages or single best matches. Video representations are formed from ordered tuples of varying numbers of frames, which allows sub-sequences of actions at different speeds and temporal offsets to be compared. Our proposed Temporal-Relational CrossTransformers (TRX) achieve state-of-the-art results on few-shot splits of Kinetics, Something-Something V2 (SSv2), HMDB51 and UCF101. Importantly, our method outperforms prior work on SSv2 by a wide margin (12%) due to the its ability to model temporal relations. A detailed ablation showcases the importance of matching to multiple support set videos and learning higher-order relational CrossTransformers.Comment: Accepted in CVPR 202

Design of Antibody-Functionalized Polymeric Membranes for the Immunoisolation of Pancreatic Islets

none8noAn immunoencapsulation strategy for pancreatic islets aimed to reduce the risk of rejection in transplanted patients due to the immune response of the host organism is proposed. In this sense, a polyethylene glycol (PEG) hydrogel functionalized with an immunosuppressive antibody (Ab), such as Cytotoxic T-lymphocyte antigen-4 Ig (CTLA4-Ig), would act as both passive and active barrier to the host immune response. To demonstrate the feasibility of this approach, a photopolymerizable-PEG was conjugated to the selected antibody and the PEG-Ab complex was used to coat the islets. Moreover, to preserve the antigen-recognition site of the antibody during the conjugation process, a controlled immobilization method was setup through the attachment of the His-tagged antigen to a solid support. In detail, a gold-coated silicon wafer functionalized with 11-Mercaptoundecanoic acid was used as a substrate for further modification, leading to a nickel(II)-terminated ligand surface. Then, the immobilized antigen was recognized by the corresponding antibody that was conjugated to the PEG. The antibody-PEG complex was detached from the support prior to be photopolymerized around the islets. First, this immobilization method has been demonstrated for the green fluorescent protein (GFP)–anti-green fluorescent protein (Anti-GFP) antigen-antibody pair, as proof of principle. Then, the approach was extended to the immunorelevant B7-1 CTLA-4-Ig antigen-antibody pair, followed by the binding of Acryl-PEG to the immobilized constant region of the antibody. In both cases, after using an elution protocol, only a partial recovery of the antibody-PEG complex was obtained. Nevertheless, the viability and the functional activity of the encapsulated islets, as determined by the glucose-stimulated insulin secretion (GSIS) assay, showed the good compatibility of this approach.openAnna Cavallo; Ugo Masullo; Alessandra Quarta; Alessandro Sannino; Amilcare Barca; Tiziano Verri; Marta Madaghiele; Laura BlasiCavallo, Anna; Masullo, Ugo; Quarta, Alessandra; Sannino, Alessandro; Barca, Amilcare; Verri, Tiziano; Madaghiele, Marta; Blasi, Laur

Improving Protein Crystal Quality by the Without-Oil Microbatch Method: Crystallization and Preliminary X-ray Diffraction Analysis of Glutathione Synthetase from Pseudoalteromonas haloplanktis

Glutathione synthetases catalyze the ATP-dependent synthesis of glutathione from l-γ-glutamyl- l-cysteine and glycine. Although these enzymes have been sequenced and characterized from a variety of biological sources, their exact catalytic mechanism is not fully understood and nothing is known about their adaptation at extremophilic environments. Glutathione synthetase from the Antarctic eubacterium Pseudoalteromonas haloplanktis (PhGshB) has been expressed, purified and successfully crystallized. An overall improvement of the crystal quality has been obtained by adapting the crystal growth conditions found with vapor diffusion experiments to the without-oil microbatch method. The best crystals of PhGshB diffract to 2.34 Å resolution and belong to space group P212121, with unit-cell parameters a = 83.28 Å, b = 119.88 Å, c = 159.82 Å. Refinement of the model, obtained using phases derived from the structure of the same enzyme from Escherichia coli by molecular replacement, is in progress. The structural determination will provide the first structural characterization of a psychrophilic glutathione synthetase reported to date