I Bet You Are Wrong: Gambling Adversarial Networks for Structured Semantic Segmentation

Adversarial training has been recently employed for realizing structured semantic segmentation, in which the aim is to preserve higher-level scene structural consistencies in dense predictions. However, as we show, value-based discrimination between the predictions from the segmentation network and ground-truth annotations can hinder the training process from learning to improve structural qualities as well as disabling the network from properly expressing uncertainties. In this paper, we rethink adversarial training for semantic segmentation and propose to formulate the fake/real discrimination framework with a correct/incorrect training objective. More specifically, we replace the discriminator with a "gambler" network that learns to spot and distribute its budget in areas where the predictions are clearly wrong, while the segmenter network tries to leave no clear clues for the gambler where to bet. Empirical evaluation on two road-scene semantic segmentation tasks shows that not only does the proposed method re-enable expressing uncertainties, it also improves pixel-wise and structure-based metrics.Comment: 13 pages, 8 figure

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Accelerated development of cerebral small vessel disease in young stroke patients.

OBJECTIVE: To study the long-term prevalence of small vessel disease after young stroke and to compare this to healthy controls. METHODS: This prospective cohort study comprises 337 patients with an ischemic stroke or TIA, aged 18-50 years, without a history of TIA or stroke. In addition, 90 age- and sex-matched controls were included. At follow-up, lacunes, microbleeds, and white matter hyperintensity (WMH) volume were assessed using MRI. To investigate the relation between risk factors and small vessel disease, logistic and linear regression were used. RESULTS: After mean follow-up of 9.9 (SD 8.1) years, 337 patients were included (227 with an ischemic stroke and 110 with a TIA). Mean age of patients was 49.8 years (SD 10.3) and 45.4% were men; for controls, mean age was 49.4 years (SD 11.9) and 45.6% were men. Compared with controls, patients more often had at least 1 lacune (24.0% vs 4.5%, p < 0.0001). In addition, they had a higher WMH volume (median 1.5 mL [interquartile range (IQR) 0.5-3.7] vs 0.4 mL [IQR 0.0-1.0], p < 0.001). Compared with controls, patients had the same volume WMHs on average 10-20 years earlier. In the patient group, age at stroke (β = 0.03, 95% confidence interval [CI] 0.02-0.04) hypertension (β = 0.22, 95% CI 0.04-0.39), and smoking (β = 0.18, 95% CI 0.01-0.34) at baseline were associated with WMH volume. CONCLUSIONS: Patients with a young stroke have a higher burden of small vessel disease than controls adjusted for confounders. Cerebral aging seems accelerated by 10-20 years in these patients, which may suggest an increased vulnerability to vascular risk factors.This is the final version of the article. It first appeared from Wolters Kluwer via https://doi.org/10.​1212/​WNL.​0000000000003123

Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge

Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. The automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We organized a scientific challenge, in which developers could evaluate their methods on a standardized multi-center/-scanner image dataset, giving an objective comparison: the WMH Segmentation Challenge. Sixty T1 + FLAIR images from three MR scanners were released with the manual WMH segmentations for training. A test set of 110 images from five MR scanners was used for evaluation. The segmentation methods had to be containerized and submitted to the challenge organizers. Five evaluation metrics were used to rank the methods: 1) Dice similarity coefficient; 2) modified Hausdorff distance (95th percentile); 3) absolute log-transformed volume difference; 4) sensitivity for detecting individual lesions; and 5) F1-score for individual lesions. In addition, the methods were ranked on their inter-scanner robustness; 20 participants submitted their methods for evaluation. This paper provides a detailed analysis of the results. In brief, there is a cluster of four methods that rank significantly better than the other methods, with one clear winner. The inter-scanner robustness ranking shows that not all the methods generalize to unseen scanners. The challenge remains open for future submissions and provides a public platform for method evaluation

PVP: Personalized Video Prior for Editable Dynamic Portraits using StyleGAN

Portrait synthesis creates realistic digital avatars which enable users to interact with others in a compelling way. Recent advances in StyleGAN and its extensions have shown promising results in synthesizing photorealistic and accurate reconstruction of human faces. However, previous methods often focus on frontal face synthesis and most methods are not able to handle large head rotations due to the training data distribution of StyleGAN. In this work, our goal is to take as input a monocular video of a face, and create an editable dynamic portrait able to handle extreme head poses. The user can create novel viewpoints, edit the appearance, and animate the face. Our method utilizes pivotal tuning inversion (PTI) to learn a personalized video prior from a monocular video sequence. Then we can input pose and expression coefficients to MLPs and manipulate the latent vectors to synthesize different viewpoints and expressions of the subject. We also propose novel loss functions to further disentangle pose and expression in the latent space. Our algorithm shows much better performance over previous approaches on monocular video datasets, and it is also capable of running in real-time at 54 FPS on an RTX 3080.Comment: Project website: https://cseweb.ucsd.edu//~viscomp/projects/EGSR23PVP