Deep Sketch-Photo Face Recognition Assisted by Facial Attributes

In this paper, we present a deep coupled framework to address the problem of matching sketch image against a gallery of mugshots. Face sketches have the essential in- formation about the spatial topology and geometric details of faces while missing some important facial attributes such as ethnicity, hair, eye, and skin color. We propose a cou- pled deep neural network architecture which utilizes facial attributes in order to improve the sketch-photo recognition performance. The proposed Attribute-Assisted Deep Con- volutional Neural Network (AADCNN) method exploits the facial attributes and leverages the loss functions from the facial attributes identification and face verification tasks in order to learn rich discriminative features in a common em- bedding subspace. The facial attribute identification task increases the inter-personal variations by pushing apart the embedded features extracted from individuals with differ- ent facial attributes, while the verification task reduces the intra-personal variations by pulling together all the fea- tures that are related to one person. The learned discrim- inative features can be well generalized to new identities not seen in the training data. The proposed architecture is able to make full use of the sketch and complementary fa- cial attribute information to train a deep model compared to the conventional sketch-photo recognition methods. Exten- sive experiments are performed on composite (E-PRIP) and semi-forensic (IIIT-D semi-forensic) datasets. The results show the superiority of our method compared to the state- of-the-art models in sketch-photo recognition algorithm

Adaptive Parameter Selection for Deep Brain Stimulation in Parkinson’s Disease

Each year, around 60,000 people are diagnosed with Parkinson’s disease (PD) and the economic burden of PD is at least $14.4 billion a year in the United States. Pharmaceutical costs for a Parkinson’s patient can be reduced from$12,000 to $6,000 per year with the addition of neuromodulation therapies such as Deep Brain Stimulation (DBS), transcranial Direct Current Stimulation (tDCS), Transcranial Magnetic Stimulation (TMS), etc. In neurodegenerative disorders such as PD, deep brain stimulation (DBS) is a desirable approach when the medication is less effective for treating the symptoms. DBS incorporates transferring electrical pulses to a specific tissue of the central nervous system and obtaining therapeutic results by modulating the neuronal activity of that region. The hyperkinetic symptoms of PD are associated with the ensembles of interacting oscillators that cause excess or abnormal synchronous behavior within the Basal Ganglia (BG) circuitry. Delayed feedback stimulation is a closed loop technique shown to suppress this synchronous oscillatory activity. Deep Brain Stimulation via delayed feedback is known to destabilize the complex intermittent synchronous states. Computational models of the BG network are often introduced to investigate the effect of delayed feedback high frequency stimulation on partially synchronized dynamics. In this work, we developed several computational models of four interacting nuclei of the BG as well as considering the Thalamo-Cortical local effects on the oscillatory dynamics. These models are able to capture the emergence of 34 Hz beta band oscillations seen in the Local Field Potential (LFP) recordings of the PD state. Traditional High Frequency Stimulations (HFS) has shown deficiencies such as strengthening the synchronization in case of highly fluctuating neuronal activities, increasing the energy consumed as well as the incapability of activating all neurons in a large-scale network. To overcome these drawbacks, we investigated the effects of the stimulation waveform and interphase delays on the overall efficiency and efficacy of DBS. We also propose a new feedback control variable based on the filtered and linearly delayed LFP recordings. The proposed control variable is then used to modulate the frequency of the stimulation signal rather than its amplitude. In strongly coupled networks, oscillations reoccur as soon as the amplitude of the stimulus signal declines. Therefore, we show that maintaining a fixed amplitude and modulating the frequency might ameliorate the desynchronization process, increase the battery lifespan and activate substantial regions of the administered DBS electrode. The charge balanced stimulus pulse itself is embedded with a delay period between its charges to grant robust desynchronization with lower amplitude needed. The efficiency and efficacy of the proposed Frequency Adjustment Stimulation (FAS) protocol in a delayed feedback method might contribute to further investigation of DBS modulations aspired to address a wide range of abnormal oscillatory behaviors observed in neurological disorders. Adaptive stimulation can open doors towards simultaneous stimulation with MRI recordings. We additionally propose a new pipeline to investigate the effect of Transcranial Magnetic Stimulation (TMS) on patient specific models. The pipeline allows us to generate a full head segmentation based on each individual MRI data. In the next step, the neurosurgeon can adaptively choose the proper location of stimulation and transmit accurate magnetic field with this pipeline

Domain Adaptation and Privileged Information for Visual Recognition

The automatic identification of entities like objects, people or their actions in visual data, such as images or video, has significantly improved, and is now being deployed in access control, social media, online retail, autonomous vehicles, and several other applications. This visual recognition capability leverages supervised learning techniques, which require large amounts of labeled training data from the target distribution representative of the particular task at hand. However, collecting such training data might be expensive, require too much time, or even be impossible. In this work, we introduce several novel approaches aiming at compensating for the lack of target training data. Rather than leveraging prior knowledge for building task-specific models, typically easier to train, we focus on developing general visual recognition techniques, where the notion of prior knowledge is better identified by additional information, available during training. Depending on the nature of such information, the learning problem may turn into domain adaptation (DA), domain generalization (DG), leaning using privileged information (LUPI), or domain adaptation with privileged information (DAPI).;When some target data samples are available and additional information in the form of labeled data from a different source is also available, the learning problem becomes domain adaptation. Unlike previous DA work, we introduce two novel approaches for the few-shot learning scenario, which require only very few labeled target samples, and even one can be very effective. The first method exploits a Siamese deep neural network architecture for learning an embedding where visual categories from the source and target distributions are semantically aligned and yet maximally separated. The second approach instead, extends adversarial learning to simultaneously maximize the confusion between source and target domains while achieving semantic alignment.;In complete absence of target data, several cheaply available source datasets related to the target distribution can be leveraged as additional information for learning a task. This is the domain generalization setting. We introduce the first deep learning approach to address the DG problem, by extending a Siamese network architecture for learning a representation of visual categories that is invariant with respect to the sources, while imposing semantic alignment and class separation to maximize generalization performance on unseen target domains.;There are situations in which target data for training might come equipped with additional information that can be modeled as an auxiliary view of the data, and that unfortunately is not available during testing. This is the LUPI scenario. We introduce a novel framework based on the information bottleneck that leverages the auxiliary view to improve the performance of visual classifiers. We do so by introducing a formulation that is general, in the sense that can be used with any visual classifier.;Finally, when the available target data is unlabeled, and there is closely related labeled source data, which is also equipped with an auxiliary view as additional information, we pose the question of how to leverage the source data views to train visual classifiers for unseen target data. This is the DAPI scenario. We extend the LUPI framework based on the information bottleneck to learn visual classifiers in DAPI settings and show that privileged information can be leveraged to improve the learning on new domains. Also, the novel DAPI framework is general and can be used with any visual classifier.;Every use of auxiliary information has been validated extensively using publicly available benchmark datasets, and several new state-of-the-art accuracy performance values have been set. Examples of application domains include visual object recognition from RGB images and from depth data, handwritten digit recognition, and gesture recognition from video