Generalized Gumbel-Softmax Gradient Estimator for Various Discrete Random Variables

Estimating the gradients of stochastic nodes is one of the crucial research questions in the deep generative modeling community, which enables the gradient descent optimization on neural network parameters. This estimation problem becomes further complex when we regard the stochastic nodes to be discrete because pathwise derivative techniques cannot be applied. Hence, the stochastic gradient estimation of discrete distributions requires either a score function method or continuous relaxation of the discrete random variables. This paper proposes a general version of the Gumbel-Softmax estimator with continuous relaxation, and this estimator is able to relax the discreteness of probability distributions including more diverse types, other than categorical and Bernoulli. In detail, we utilize the truncation of discrete random variables and the Gumbel-Softmax trick with a linear transformation for the relaxed reparameterization. The proposed approach enables the relaxed discrete random variable to be reparameterized and to backpropagated through a large scale stochastic computational graph. Our experiments consist of (1) synthetic data analyses, which show the efficacy of our methods; and (2) applications on VAE and topic model, which demonstrate the value of the proposed estimation in practices

Improvisation of classification performance based on feature optimization for differentiation of Parkinson’s disease from other neurological diseases using gait characteristics

Most neurological disorders that include Parkinson’s disease (PD) as well as other neurological diseases such as Amyotrophic Lateral Sclerosis (ALS) and Huntington’s disease (HD) have some common abnormalities regarding the movement, vocal, and cognitive behaviors of sufferers. Variations in the manifestation of these types of abnormality help distinguish one disorder from another. In this study, differentiation was performed based on the gait characteristics of patients afflicted by different neurological disorders. In the recent past, many researchers have applied different machine learning and feature selection techniques to the classification of different groups of patients based on common abnormalities. However, in an era of modernization where the focus is on timely low-cost automatization and pattern recognition, such techniques require improvisation to provide high performance. We attempted to improve the performance of such techniques using different feature optimization methods, such as a genetic algorithm (GA) and principal component analysis (PCA), and applying different classification approaches, i.e., linear, nonlinear, and probabilistic classifiers. In this study, gait dynamics data of patients suffering with PD, ALS, and HD were collated from a public database, and a binary classification approach was used by taking PD as one group and adopting ALS+HD as another group. Performance comparison was achieved using different classification techniques that incorporated optimized feature sets obtained from GA and PCA. In comparison with other classifiers using different feature sets, the highest accuracy (97.87%) was obtained using random forest combined with GA-based feature sets. The results provide evidence that could assist medical practitioners in differentiating PD from other neurological diseases using gait characteristics

Sequential Recommendation with Relation-Aware Kernelized Self-Attention

Recent studies identified that sequential Recommendation is improved by the attention mechanism. By following this development, we propose Relation-Aware Kernelized Self-Attention (RKSA) adopting a self-attention mechanism of the Transformer with augmentation of a probabilistic model. The original self-attention of Transformer is a deterministic measure without relation-awareness. Therefore, we introduce a latent space to the self-attention, and the latent space models the recommendation context from relation as a multivariate skew-normal distribution with a kernelized covariance matrix from co-occurrences, item characteristics, and user information. This work merges the self-attention of the Transformer and the sequential recommendation by adding a probabilistic model of the recommendation task specifics. We experimented RKSA over the benchmark datasets, and RKSA shows significant improvements compared to the recent baseline models. Also, RKSA were able to produce a latent space model that answers the reasons for recommendation.Comment: 8 pages, 5 figures, AAA

Utilizing ECG Waveform Features as New Biometric Authentication Method

In this study, we are proposing a practical way for human identification based on a new biometric method. The new method is built on the use of the electrocardiogram (ECG) signal waveform features, which are produced from the process of acquiring electrical activities of the heart by using electrodes placed on the body. This process is launched over a period of time by using a recording device to read and store the ECG signal. On the contrary of other biometrics method like voice, fingerprint and iris scan, ECG signal cannot be copied or manipulated. The first operation for our system is to record a portion of 30 seconds out of whole ECG signal of a certain user in order to register it as user template in the system. Then the system will take 7 to 9 seconds in authenticating the template using template matching techniques. 44 subjects‟ raw ECG data were downloaded from Physionet website repository. We used a template matching technique for the authentication process and Linear SVM algorithm for the classification task. The accuracy rate was 97.2% for the authentication process and 98.6% for the classification task; with false acceptance rate 1.21%

Transcanal Endoscopic Ear Surgery for Congenital Cholesteatoma

Objectives As endoscopic instrumentation, techniques and knowledges have significantly improved recently, endoscopic ear surgery has become increasingly popular. Transcanal endoscopic ear surgery (TEES) can provide better visualization of hidden areas in the middle ear cavity during congenital cholesteatoma removal. We aimed to describe outcomes for TEES for congenital cholesteatoma in a pediatric population. Methods Twenty-five children (age, 17 months to 9 years) with congenital cholesteatoma confined to the middle ear underwent TEES by an experienced surgeon; 13 children had been classified as Potsic stage I, seven as stage II, and five as stage III. The mean follow-up period was 24 months. Recurrence of congenital cholesteatoma and surgical complication was observed. Results Congenital cholesteatoma can be removed successfully via transcanal endoscopic approach in all patients, and no surgical complications occurred; only one patient with a stage II cholesteatoma showed recurrence during the follow-up visit, and the patient underwent revision surgery. The other patients underwent one-stage operations and showed no cholesteatoma recurrence at their last visits. Two patients underwent second-stage ossicular reconstruction. Conclusion Although the follow-up period and number of patients were limited, pediatric congenital cholesteatoma limited to the middle ear cavity could be safely and effectively removed using TEES

Bidirectional two colored light emission from stress-activated ZnS-microparticles-embedded polydimethylsiloxane elastomer films

Bidirectional two-colored mechanoluminescent light emission has been demonstrated by unifying two polydimethylsiloxane elastomer layers functionalized with zinc sulfide doped with Cu (ZnS:Cu) or Cu and Mn (ZnS:Cu,Mn). The bilayered composite films are simply fabricated by dispensing uncured ZnS:Cu,Mn + PDMS onto previously spin-coated and ardened ZnS:Cu + PDMS film. The robust PDMS-PDMS bonding yields a ilm which can simultaneously emit light with color coordinates of (0.25, 0.56) and (0.50, 0.48), similar to the intrinsic colors of ZnS:Cu and ZnS:Cu,Mn, respectively. Composite films can emit light in upper and lower directions without fracture when it is stretched. © 2013 Optical Society of America.1

Make Prompts Adaptable: Bayesian Modeling for Vision-Language Prompt Learning with Data-Dependent Prior

Recent Vision-Language Pretrained (VLP) models have become the backbone for many downstream tasks, but they are utilized as frozen model without learning. Prompt learning is a method to improve the pre-trained VLP model by adding a learnable context vector to the inputs of the text encoder. In a few-shot learning scenario of the downstream task, MLE training can lead the context vector to over-fit dominant image features in the training data. This overfitting can potentially harm the generalization ability, especially in the presence of a distribution shift between the training and test dataset. This paper presents a Bayesian-based framework of prompt learning, which could alleviate the overfitting issues on few-shot learning application and increase the adaptability of prompts on unseen instances. Specifically, modeling data-dependent prior enhances the adaptability of text features for both seen and unseen image features without the trade-off of performance between them. Based on the Bayesian framework, we utilize the Wasserstein Gradient Flow in the estimation of our target posterior distribution, which enables our prompt to be flexible in capturing the complex modes of image features. We demonstrate the effectiveness of our method on benchmark datasets for several experiments by showing statistically significant improvements on performance compared to existing methods. The code is available at https://github.com/youngjae-cho/APP.Comment: Accepted to AAAI-202