Bayes-optimal Learning of Deep Random Networks of Extensive-width

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width are proportionally large. We propose a closed-form expression for the Bayes-optimal test error, for regression and classification tasks. We further compute closed-form expressions for the test errors of ridge regression, kernel and random features regression. We find, in particular, that optimally regularized ridge regression, as well as kernel regression, achieve Bayes-optimal performances, while the logistic loss yields a near-optimal test error for classification. We further show numerically that when the number of samples grows faster than the dimension, ridge and kernel methods become suboptimal, while neural networks achieve test error close to zero from quadratically many samples

Penerapan Combine Undersampling Pada Klasifikasi Data Imbalanced Biner (Studi Kasus : Desa Tertinggal Di Jawa Timur Tahun 2014)

Regresi Logistik memiliki beberapa kelebihan dibandingkan metode klasifikasi lainnya yaitu sebagai classifier dengan akurasi yang cukup tinggi dan penggunaan algoritma yang tepat akan mampu menghasilkan waktu perhitungan yang lebih cepat khususnya pada data besar. Adanya permasalahan data yang tidak seimbang akan berpengaruh pada hasil ketepatan klasifikasi. Pada penelitian ini, metode resampling yang digunakan adalah Combine Undersampling dan metode classifier yang digunakan adalah Regresi Logistik, Regresi Logistik Ridge, dan Analisis Diskriminan Kernel. Data yang diteliti adalah data status desa tertinggal di Jawa Timur tahun 2014 sebanyak 7.721 desa. Penerapan Combine Undersampling mampu meningkatkan ketepatan klasifikasi pada rata-rata sensitivititas secara siginifikan khususnya pada klasifikasi Regresi Logistik Ridge sebesar 42,4 kali dengan menggunakan semua variabel. Selain itu, hasil ketepatan klasifikasi terbaik menunjukkan nilai akurasi total dan AUC yang sama ketika menerapkan metode Combine Undersampling pada Klasifikasi Analisis Diskriminan Kernel yaitu 78.0 % sedangkan pada variabel signifikan metode Regresi Logistik Ridge menghasilkan ketepatan klasifikasi lebih baik dari metode lainnya yang memiliki nilai AUC sebesar 73,5% . ========================================================= Logistic regression has several advantages over other classification methods that is as a classifier with a fairly high accuracy and the use of appropriate algorithms will be able to produce faster calculation times, especially on large data. The existence of imbalanced data problems will affect the results of classification accuracy. In this research, the resampling method used is Combine Undersampling and the classifier method used are Logistic Regression, Ridge Logistic Regression, and Kernel Discriminant Analysis. The data studied is the status data of underdevelop villages in East Java in 2014 as many as 7,721 villages. The application of Combine Undersampling is able to increase the classification accuracy on the average sensitivity significantly in the Ridge Logistic Regression classification by 42.4 times using all the variables. In addition, the best classification accuracy results show the same total accuracy and AUC value when applying Combine Undersampling method in the Kernel Discriminant Classification Classification is 78.0% whereas in the significant variables the Ridge Logistic Regression method produces better classification accuracy than other methods which have AUC value of 73 , 5%

Tensor-Based Algorithms for Image Classification

Interest in machine learning with tensor networks has been growing rapidly in recent years. We show that tensor-based methods developed for learning the governing equations of dynamical systems from data can, in the same way, be used for supervised learning problems and propose two novel approaches for image classification. One is a kernel-based reformulation of the previously introduced multidimensional approximation of nonlinear dynamics (MANDy), the other an alternating ridge regression in the tensor train format. We apply both methods to the MNIST and fashion MNIST data set and show that the approaches are competitive with state-of-the-art neural network-based classifiers