Personalizing gesture recognition using hierarchical bayesian neural networks

Building robust classifiers trained on data susceptible to group or subject-specific variations is a challenging pattern recognition problem. We develop hierarchical Bayesian neural networks to capture subject-specific variations and share statistical strength across subjects. Leveraging recent work on learning Bayesian neural networks, we build fast, scalable algorithms for inferring the posterior distribution over all network weights in the hierarchy. We also develop methods for adapting our model to new subjects when a small number of subject-specific personalization data is available. Finally, we investigate active learning algorithms for interactively labeling personalization data in resource-constrained scenarios. Focusing on the problem of gesture recognition where inter-subject variations are commonplace, we demonstrate the effectiveness of our proposed techniques. We test our framework on three widely used gesture recognition datasets, achieving personalization performance competitive with the state-of-the-art.http://openaccess.thecvf.com/content_cvpr_2017/html/Joshi_Personalizing_Gesture_Recognition_CVPR_2017_paper.htmlhttp://openaccess.thecvf.com/content_cvpr_2017/html/Joshi_Personalizing_Gesture_Recognition_CVPR_2017_paper.htmlhttp://openaccess.thecvf.com/content_cvpr_2017/html/Joshi_Personalizing_Gesture_Recognition_CVPR_2017_paper.htmlPublished versio

Attend to You: Personalized Image Captioning with Context Sequence Memory Networks

We address personalization issues of image captioning, which have not been discussed yet in previous research. For a query image, we aim to generate a descriptive sentence, accounting for prior knowledge such as the user's active vocabularies in previous documents. As applications of personalized image captioning, we tackle two post automation tasks: hashtag prediction and post generation, on our newly collected Instagram dataset, consisting of 1.1M posts from 6.3K users. We propose a novel captioning model named Context Sequence Memory Network (CSMN). Its unique updates over previous memory network models include (i) exploiting memory as a repository for multiple types of context information, (ii) appending previously generated words into memory to capture long-term information without suffering from the vanishing gradient problem, and (iii) adopting CNN memory structure to jointly represent nearby ordered memory slots for better context understanding. With quantitative evaluation and user studies via Amazon Mechanical Turk, we show the effectiveness of the three novel features of CSMN and its performance enhancement for personalized image captioning over state-of-the-art captioning models.Comment: Accepted paper at CVPR 201

Implementasi dan analisis Biased Regularization Support Vector Machine (BRSVM) pada kasus Pengenalan Huruf Tulisan Tangan

ABSTRAKSI: Berkembangnya teknologi komputerisasi membuat metode penulisan konvensional bergeseser, namun cara ini akan terus bertahan selama menulis menggunakan pena dan kertas lebih memberikan kenyamanan dan kesenangan dibanding menggunakan keyboard dan PC table. Untuk menjembatani hal tersebut dengan perkembangan teknologi, maka diperlukan suatu sistem pengenalan tulisan tangan yang mampu mengubah tulisan tangan kertas menjadi format teks komputer. Hal inilah yang mendorong munculnya berbagai penelitian mengenai pengenalan tulisan tangan. Dan hingga saat ini tingkat kesempurnaan dari penelitian tersebut belum ada yang mampu menggantikan kemampuan semantik manusia secara sempurna. Penggunaan SVM sebagai metode pembelajaran dalam sistem tersebut dapat membantu penentuan hyperplane terbaik yang memisahkan kelas-kelas karakter huruf tulisan tangan. Permasalahannya tingkat akurasi klasifikasi data tersebut menjadi kurang optimal ketika data tulisan tangan yang diuji berbeda jauh dengan sistem yang ada. Dan dengan menggunakan Biased Regularization pada SVM (BRSVM), tingkat akurasi dari pengujian terhadap data tersebut dapat ditingkatkan dengan cara membiaskan resiko umum yang muncul dari pengenalan tulisan tangan SVM. Kata Kunci : Hand Writing Recognition , BRSVM, SVM, Biased RegularizationABSTRACT: Computerization technology development takes over the conventional writing method, but this way will survive as long as paper writing is more comfortable and gives the happiness than keyboard writing or PC table writing. For bridging this case with technology development, we need a system to recognize hand writing which is able to change this hand writing become a computer text format. This case encourages a lot of research about hand writing recognition. For this moment, none of those researches are able to replace the perfection rate of human semantic ability. Using SVM as learning method in the system will helps determination of the best hyperplane which is separating word character classes of hand writing. The problem is the accuracy rate of data classification become not optimal when the testing hand writing data is very different from the existing system. With using Biased Regularization on SVM (BRSVM), accuracy rate from testing for the data can be increased with refracting the general risk that is appear in hand writing recognition SVM.Keyword: Hand Writing Recognition, BRSVM, SVM, Biased Regularizatio

Heterogeneous Federated Learning: State-of-the-art and Research Challenges

Federated learning (FL) has drawn increasing attention owing to its potential use in large-scale industrial applications. Existing federated learning works mainly focus on model homogeneous settings. However, practical federated learning typically faces the heterogeneity of data distributions, model architectures, network environments, and hardware devices among participant clients. Heterogeneous Federated Learning (HFL) is much more challenging, and corresponding solutions are diverse and complex. Therefore, a systematic survey on this topic about the research challenges and state-of-the-art is essential. In this survey, we firstly summarize the various research challenges in HFL from five aspects: statistical heterogeneity, model heterogeneity, communication heterogeneity, device heterogeneity, and additional challenges. In addition, recent advances in HFL are reviewed and a new taxonomy of existing HFL methods is proposed with an in-depth analysis of their pros and cons. We classify existing methods from three different levels according to the HFL procedure: data-level, model-level, and server-level. Finally, several critical and promising future research directions in HFL are discussed, which may facilitate further developments in this field. A periodically updated collection on HFL is available at https://github.com/marswhu/HFL_Survey.Comment: 42 pages, 11 figures, and 4 table

Mitigating Group Bias in Federated Learning for Heterogeneous Devices

Federated Learning is emerging as a privacy-preserving model training approach in distributed edge applications. As such, most edge deployments are heterogeneous in nature i.e., their sensing capabilities and environments vary across deployments. This edge heterogeneity violates the independence and identical distribution (IID) property of local data across clients and produces biased global models i.e. models that contribute to unfair decision-making and discrimination against a particular community or a group. Existing bias mitigation techniques only focus on bias generated from label heterogeneity in non-IID data without accounting for domain variations due to feature heterogeneity and do not address global group-fairness property. Our work proposes a group-fair FL framework that minimizes group-bias while preserving privacy and without resource utilization overhead. Our main idea is to leverage average conditional probabilities to compute a cross-domain group \textit{importance weights} derived from heterogeneous training data to optimize the performance of the worst-performing group using a modified multiplicative weights update method. Additionally, we propose regularization techniques to minimize the difference between the worst and best-performing groups while making sure through our thresholding mechanism to strike a balance between bias reduction and group performance degradation. Our evaluation of human emotion recognition and image classification benchmarks assesses the fair decision-making of our framework in real-world heterogeneous settings

On predicting stopping time of human sequential decision-making using discounted satisficing heuristic

“Human sequential decision-making involves two essential questions: (i) what to choose next? , and (ii) when to stop? . Assuming that the human agents choose an alternative according to their preference order, our goal is to model and learn how human agents choose their stopping time while making sequential decisions. In contrary to traditional assumptions in the literature regarding how humans exhibit satisficing behavior on instantaneous utilities, we assume that humans employ a discounted satisficing heuristic to compute their stopping time, i.e., the human agent stops working if the total accumulated utility goes beyond a dynamic threshold that gets discounted with time. In this thesis, we model the stopping time in 3 scenarios where the payoff of the human worker is assumed as (i) single-attribute utility, (ii) multi-attribute utility with known weights, and (iii) multi-attribute utility with unknown weights. We propose algorithms to estimate the model parameters followed by predicting the stopping time in all three scenarios and present the simulation results to demonstrate the error performance. Simulation results are presented to demonstrate the convergence of prediction error of stopping time, in spite of the fact that model parameters converge to biased estimates. This observation is later justified using an illustrative example to show that there are multiple discounted satisficing models that explain the same stopping time decision. A novel web application is also developed to emulate a crowd-sourcing platform in our lab to capture multi-attribute information regarding the task in order to perform validations of the proposed algorithms on real data”--Abstract, page iii