Human Activity Recognition with Pose-driven Attention to RGB

International audienceWe address human action recognition from multi-modal video data involving articulated pose and RGB frames and propose a two-stream approach. The pose stream is processed with a convolutional model taking as input a 3D tensor holding data from a sub-sequence. A specific joint ordering, which respects the topology of the human body, ensures that different convolutional layers correspond to meaningful levels of abstraction. The raw RGB stream is handled by a spatio-temporal soft-attention mechanism conditioned on features from the pose network. An LSTM network receives input from a set of image locations at each instant. A trainable glimpse sensor extracts features on a set of pre-defined locations specified by the pose stream, namely the 4 hands of the two people involved in the activity. Appearance features give important cues on hand motion and on objects held in each hand. We show that it is of high interest to shift the attention to different hands at different time steps depending on the activity itself. Finally a temporal attention mechanism learns how to fuse LSTM features over time. State-of-the-art results are achieved on the largest dataset for human activity recognition, namely NTU-RGB+D

Evaluation of early-phase [F-18]-florbetaben PET acquisition in clinical routine cases

Objectives: In recent years several [F-18]-labelled amyloid PET tracers have been developed and have obtained clinical approval. There is accumulating evidence that early (post injection) acquisitionswith these tracers are equally informative as conventional blood flow andmetabolismstudies for diagnosis of Alzheimer's disease, but there have been few side-by-side studies. Therefore, we investigated the performance of early acquisitions of [F-18]florbetaben (FBB) PET compared to [F-18]-fluorodeoxyglucose (FDG) PET in a clinical setting. Methods: All subjects were recruited with clinical suspicion of dementia due to neurodegenerative disease. FDG PET was undertaken by conventional methods, and amyloid PET was performed with FBB, with early recordings for the initial 10 min (early-phase FBB), and late recordings at 90-110 min p.i. (late-phase FBB). Regional SUVR with cerebellar and globalmean normalization were calculated for early-phase FBB and FDG PET. Pearson correlation coefficients between FDG and early-phase FBB were calculated for predefined cortical brain regions. Furthermore, a visual interpretation of disease pattern using 3-dimensional stereotactic surface projections (3DSSP) was performed, with assessment of intra-reader agreement. Results: Among a total of 33 patients (mean age 67.5 +/- 11.0 years) included in the study, 18 were visually rated amyloid-positive, and 15 amyloid-negative based on late-phase FBB scans. Correlation coefficients for earlyphase FBB vs. FDG scans displayed excellent agreement in all target brain regions for global mean normalization. Cerebellar normalization gave strong, but significantly lower correlations. 3D representations of early-phase FBB visually resembled the corresponding FDG PET images, irrespective of the amyloid-status of the late FBB scans. Conclusions: Early-phase FBB acquisitions correlate on a relative quantitative and visual level with FDG PET scans, irrespective of the amyloid plaque density assessed in late FBB imaging. Thus, early-phase FBB uptake depicts a metabolism-like image, suggesting it as a valid surrogatemarker for synaptic dysfunction, which could ultimately circumvent the need for additional FDG PET investigation in diagnosis of dementia. (C) 2016 The Author(s). Published by Elsevier Inc

Voxel-wise analysis of dynamic F-18-FET PET: a novel approach for non-invasive glioma characterisation

Background: Glioma grading with dynamic F-18-FET PET (0-40 min p.i.) is typically performed by analysing the mean time-activity curve of the entire tumour or a suspicious area within a heterogeneous tumour. This work aimed to ensure a reader-independent glioma characterisation and identification of aggressive sub-volumes by performing a voxel-based analysis with diagnostically relevant kinetic and static 18F-FET PET parameters. One hundred sixty-two patients with a newly diagnosed glioma classified according to histologic and molecular genetic properties were evaluated. The biological tumour volume (BTV) was segmented in static 20-40 min p.i. F-18-FET PET images using the established threshold of 1.6 x background activity. For each enclosed voxel, the time-to-peak (TTP), the late slope (Slope(15-40)), and the tumour-to-background ratios (TBR5-15, TBR20-40) obtained from 5 to 15 min p.i. and 20 to 40 min p.i. images were determined. The percentage portion of these values within the BTV was evaluated with percentage volume fractions (PVFs) and cumulated percentage volume histograms (PVHs). The ability to differentiate histologic and molecular genetic classes was assessed and compared to volume-of-interest (VOI)-based parameters. Results: Aggressive WHO grades III and IV and IDH-wildtype gliomas were dominated by a high proportion of voxels with an early peak, negative slope, and high TBR, whereby the PVHs with TTP 2 yielded the most significant differences between glioma grades. We found significant differences of the parameters between WHO grades and IDH mutation status, where the effect size was predominantly higher for voxel-based PVHs compared to the corresponding VOI-based parameters. A low overlap of BTV sub-volumes defined by TTP ) (2)- and TBR20-40 (> 2)-defined hotspots was observed. Conclusions: The presented approach applying voxel-wise analysis of dynamic F-18-FET PET enables an enhanced characterisation of gliomas and might potentially provide a fast identification of aggressive sub-volumes within the BTV. Parametric 3D F-18-FET PET information as investigated in this study has the potential to guide individual therapy instrumentation and may be included in future biopsy studies

The Eighteenth Data Release of the Sloan Digital Sky Surveys: Targeting and First Spectra from SDSS-V

The eighteenth data release of the Sloan Digital Sky Surveys (SDSS) is the first one for SDSS-V, the fifth generation of the survey. SDSS-V comprises three primary scientific programs, or "Mappers": Milky Way Mapper (MWM), Black Hole Mapper (BHM), and Local Volume Mapper (LVM). This data release contains extensive targeting information for the two multi-object spectroscopy programs (MWM and BHM), including input catalogs and selection functions for their numerous scientific objectives. We describe the production of the targeting databases and their calibration- and scientifically-focused components. DR18 also includes ~25,000 new SDSS spectra and supplemental information for X-ray sources identified by eROSITA in its eFEDS field. We present updates to some of the SDSS software pipelines and preview changes anticipated for DR19. We also describe three value-added catalogs (VACs) based on SDSS-IV data that have been published since DR17, and one VAC based on the SDSS-V data in the eFEDS field.Comment: Accepted to ApJ

The eighteenth data release of the Sloan Digital Sky Surveys : targeting and first spectra from SDSS-V

The eighteenth data release of the Sloan Digital Sky Surveys (SDSS) is the first one for SDSS-V, the fifth generation of the survey. SDSS-V comprises three primary scientific programs, or "Mappers": Milky Way Mapper (MWM), Black Hole Mapper (BHM), and Local Volume Mapper (LVM). This data release contains extensive targeting information for the two multi-object spectroscopy programs (MWM and BHM), including input catalogs and selection functions for their numerous scientific objectives. We describe the production of the targeting databases and their calibration- and scientifically-focused components. DR18 also includes ~25,000 new SDSS spectra and supplemental information for X-ray sources identified by eROSITA in its eFEDS field. We present updates to some of the SDSS software pipelines and preview changes anticipated for DR19. We also describe three value-added catalogs (VACs) based on SDSS-IV data that have been published since DR17, and one VAC based on the SDSS-V data in the eFEDS field.Publisher PDFPeer reviewe

The Seventeenth Data Release of the Sloan Digital Sky Surveys: Complete Release of MaNGA, MaStar and APOGEE-2 Data

This paper documents the seventeenth data release (DR17) from the Sloan Digital Sky Surveys; the fifth and final release from the fourth phase (SDSS-IV). DR17 contains the complete release of the Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey, which reached its goal of surveying over 10,000 nearby galaxies. The complete release of the MaNGA Stellar Library (MaStar) accompanies this data, providing observations of almost 30,000 stars through the MaNGA instrument during bright time. DR17 also contains the complete release of the Apache Point Observatory Galactic Evolution Experiment 2 (APOGEE-2) survey which publicly releases infra-red spectra of over 650,000 stars. The main sample from the Extended Baryon Oscillation Spectroscopic Survey (eBOSS), as well as the sub-survey Time Domain Spectroscopic Survey (TDSS) data were fully released in DR16. New single-fiber optical spectroscopy released in DR17 is from the SPectroscipic IDentification of ERosita Survey (SPIDERS) sub-survey and the eBOSS-RM program. Along with the primary data sets, DR17 includes 25 new or updated Value Added Catalogs (VACs). This paper concludes the release of SDSS-IV survey data. SDSS continues into its fifth phase with observations already underway for the Milky Way Mapper (MWM), Local Volume Mapper (LVM) and Black Hole Mapper (BHM) surveys

Sloan Digital Sky Survey IV: mapping the Milky Way, nearby galaxies, and the distant universe

We describe the Sloan Digital Sky Survey IV (SDSS-IV), a project encompassing three major spectroscopic programs. The Apache Point Observatory Galactic Evolution Experiment 2 (APOGEE-2) is observing hundreds of thousands of Milky Way stars at high resolution and high signal-to-noise ratios in the near-infrared. The Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey is obtaining spatially resolved spectroscopy for thousands of nearby galaxies (median ). The extended Baryon Oscillation Spectroscopic Survey (eBOSS) is mapping the galaxy, quasar, and neutral gas distributions between and 3.5 to constrain cosmology using baryon acoustic oscillations, redshift space distortions, and the shape of the power spectrum. Within eBOSS, we are conducting two major subprograms: the SPectroscopic IDentification of eROSITA Sources (SPIDERS), investigating X-ray AGNs and galaxies in X-ray clusters, and the Time Domain Spectroscopic Survey (TDSS), obtaining spectra of variable sources. All programs use the 2.5 m Sloan Foundation Telescope at the Apache Point Observatory; observations there began in Summer 2014. APOGEE-2 also operates a second near-infrared spectrograph at the 2.5 m du Pont Telescope at Las Campanas Observatory, with observations beginning in early 2017. Observations at both facilities are scheduled to continue through 2020. In keeping with previous SDSS policy, SDSS-IV provides regularly scheduled public data releases; the first one, Data Release 13, was made available in 2016 July

Sloan Digital Sky Survey IV: Mapping the Milky Way, Nearby Galaxies, and the Distant Universe

We describe the Sloan Digital Sky Survey IV (SDSS-IV), a project encompassing three major spectroscopic programs. The Apache Point Observatory Galactic Evolution Experiment 2 (APOGEE-2) is observing hundreds of thousands of Milky Way stars at high resolution and high signal-to-noise ratios in the near-infrared. The Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey is obtaining spatially resolved spectroscopy for thousands of nearby galaxies (median $z\sim 0.03$). The extended Baryon Oscillation Spectroscopic Survey (eBOSS) is mapping the galaxy, quasar, and neutral gas distributions between $z\sim 0.6$ and 3.5 to constrain cosmology using baryon acoustic oscillations, redshift space distortions, and the shape of the power spectrum. Within eBOSS, we are conducting two major subprograms: the SPectroscopic IDentification of eROSITA Sources (SPIDERS), investigating X-ray AGNs and galaxies in X-ray clusters, and the Time Domain Spectroscopic Survey (TDSS), obtaining spectra of variable sources. All programs use the 2.5 m Sloan Foundation Telescope at the Apache Point Observatory; observations there began in Summer 2014. APOGEE-2 also operates a second near-infrared spectrograph at the 2.5 m du Pont Telescope at Las Campanas Observatory, with observations beginning in early 2017. Observations at both facilities are scheduled to continue through 2020. In keeping with previous SDSS policy, SDSS-IV provides regularly scheduled public data releases; the first one, Data Release 13, was made available in 2016 July