Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentation

Today's success of state of the art methods for semantic segmentation is driven by large datasets. Data is considered an important asset that needs to be protected, as the collection and annotation of such datasets comes at significant efforts and associated costs. In addition, visual data might contain private or sensitive information, that makes it equally unsuited for public release. Unfortunately, recent work on membership inference in the broader area of adversarial machine learning and inference attacks on machine learning models has shown that even black box classifiers leak information on the dataset that they were trained on. We show that such membership inference attacks can be successfully carried out on complex, state of the art models for semantic segmentation. In order to mitigate the associated risks, we also study a series of defenses against such membership inference attacks and find effective counter measures against the existing risks with little effect on the utility of the segmentation method. Finally, we extensively evaluate our attacks and defenses on a range of relevant real-world datasets: Cityscapes, BDD100K, and Mapillary Vistas.Comment: Accepted to ECCV 2020. Code at: https://github.com/SSAW14/segmentation_membership_inferenc

MTR-A: 1st Place Solution for 2022 Waymo Open Dataset Challenge -- Motion Prediction

In this report, we present the 1st place solution for motion prediction trackin 2022 Waymo Open Dataset Challenges. We propose a novel Motion Transformerframework for multimodal motion prediction, which introduces a small set ofnovel motion query pairs for generating better multimodal future trajectoriesby jointly performing the intention localization and iterative motionrefinement. A simple model ensemble strategy with non-maximum-suppression isadopted to further boost the final performance. Our approach achieves the 1stplace on the motion prediction leaderboard of 2022 Waymo Open DatasetChallenges, outperforming other methods with remarkable margins. Code will beavailable at https://github.com/sshaoshuai/MTR.<br

{PoseTrackReID}: {D}ataset Description

Current datasets for video-based person re-identification (re-ID) do not include structural knowledge in form of human pose annotations for the persons of interest. Nonetheless, pose information is very helpful to disentangle useful feature information from background or occlusion noise. Especially real-world scenarios, such as surveillance, contain a lot of occlusions in human crowds or by obstacles. On the other hand, video-based person re-ID can benefit other tasks such as multi-person pose tracking in terms of robust feature matching. For that reason, we present PoseTrackReID, a large-scale dataset for multi-person pose tracking and video-based person re-ID. With PoseTrackReID, we want to bridge the gap between person re-ID and multi-person pose tracking. Additionally, this dataset provides a good benchmark for current state-of-the-art methods on multi-frame person re-ID

{MoCapDeform}: {M}onocular {3D} Human Motion Capture in Deformable Scenes

3D human motion capture from monocular RGB images respecting interactions ofa subject with complex and possibly deformable environments is a verychallenging, ill-posed and under-explored problem. Existing methods address itonly weakly and do not model possible surface deformations often occurring whenhumans interact with scene surfaces. In contrast, this paper proposesMoCapDeform, i.e., a new framework for monocular 3D human motion capture thatis the first to explicitly model non-rigid deformations of a 3D scene forimproved 3D human pose estimation and deformable environment reconstruction.MoCapDeform accepts a monocular RGB video and a 3D scene mesh aligned in thecamera space. It first localises a subject in the input monocular video alongwith dense contact labels using a new raycasting based strategy. Next, ourhuman-environment interaction constraints are leveraged to jointly optimiseglobal 3D human poses and non-rigid surface deformations. MoCapDeform achievessuperior accuracy than competing methods on several datasets, including ournewly recorded one with deforming background scenes.<br