Dimension Reduction in Big Data Environment-A Survey

Relational database management system is able to tackle data set which is structured in some way and by means of querying to the system user gets certain answer. But if the data set itself does not lie under any sort of structure, it is generally very tedious job for user to get answer to certain query. This is the new challenge coming out for the last decade to the scientists, researchers, industrialists and this new form of data is termed as big data. Parallel computation not only from the concept of hardware, but different application dependent software is now being developed to tackle this new data set for solving the challenges generally attached with large data set such as data curation, search, querying, storage etc. Information sensing devices, RFID readers, cloud storage now days are making data set to grow in an increasing manner. The goal of big data analytics is to help industry and organizations to take intelligent decisions by analyzing huge number of transactions that remain untouched till today by conventional business intelligent systems. As the size of dataset grows large also with redundancy, software and people need to analyze only useful information for particular application and this newly reduced dataset are useful compare to noisy and large data

PENERAPAN ALGORITMA BACKPROPAGATION DAN PRINCIPAL COMPONENT ANALYSIS UNTUK PENGENALAN WAJAH

Perkembangan teknologi komputer yang terus meningkat sekarang ini telah mampu mengenali wajah manusia. Jaringan syaraf tiruan telah berhasil diterapkan untuk berbagai masalah klasifikasi pola. Salah satu algoritma Jaringan Saraf Tiruan yang banyak digunakan dalam pengenalan wajah adalah Backpropagation. Backpropagation adalah metode umum mengajar jaringan syaraf tiruan bagaimana melakukan tugas yang diberikan. Sedangkan Principal Component Analysis (PCA) adalah salah satu metode yang populer digunakan untuk ekstraksi ciri dan representasi data. Dengan mengkombinasikan kedua metode antara Jaringan Saraf Tiruan Backpropagation dengan Principal Component Analysis akan menghasilkan suatu pengenalan wajah yang lebih cepat. Penelitian ini mencoba mengimplementasikan kedua metode ke dalam suatu aplikasi pengenalan wajah. Dan hasilnya sistem lebih cepat melakukan pengenalan wajah

СНИЖЕНИЕ РАЗМЕРНОСТИ ОБУЧАЮЩИХ ВЫБОРОК ПРИ РАСПОЗНАВАНИИ ОБРАЗОВ НА КОСМИЧЕСКИХ ИЗОБРАЖЕНИЯХ С ПОМОЩЬЮ МЕТОДА ГЛАВНЫХ КОМПОНЕНТ

The essence of principal components analysis and the problem of dimension reduction are described. A method of principal components calculation is presented, which is based on the covariance matrix eigenvalues determination. Practical implementations of principal components analysis are described, which are based on QR-algorithm. Application of principal components analysis in space images classification for the reduction of training samples dimension is discussed.Описываются сущность метода главных компонент и задача снижения размерности в про-цессе статистической обработки. Приводится способ вычисления главных компонент на основе оп-ределения собственных значений ковариационной матрицы. Описываются алгоритмы практической реализации метода главных компонент на основе QR-алгоритма. Проводится анализ возможности использования метода главных компонент при классификации космических изображений с целью снижения размерности обучающих выборок

FACE CLASSIFICATION FOR AUTHENTICATION APPROACH BY USING WAVELET TRANSFORM AND STATISTICAL FEATURES SELECTION

This thesis consists of three parts: face localization, features selection and classification process. Three methods were proposed to locate the face region in the input image. Two of them based on pattern (template) Matching Approach, and the other based on clustering approach. Five datasets of faces namely: YALE database, MIT-CBCL database, Indian database, BioID database and Caltech database were used to evaluate the proposed methods. For the first method, the template image is prepared previously by using a set of faces. Later, the input image is enhanced by applying n-means kernel to decrease the image noise. Then Normalized Correlation (NC) is used to measure the correlation coefficients between the template image and the input image regions. For the second method, instead of using n-means kernel, an optimized metrics are used to measure the difference between the template image and the input image regions. In the last method, the Modified K-Means Algorithm was used to remove the non-face regions in the input image. The above-mentioned three methods showed accuracy of localization between 98% and 100% comparing with the existed methods. In the second part of the thesis, Discrete Wavelet Transform (DWT) utilized to transform the input image into number of wavelet coefficients. Then, the coefficients of weak statistical energy less than certain threshold were removed, and resulted in decreasing the primary wavelet coefficients number up to 98% out of the total coefficients. Later, only 40% statistical features were extracted from the hight energy features by using the variance modified metric. During the experimental (ORL) Dataset was used to test the proposed statistical method. Finally, Cluster-K-Nearest Neighbor (C-K-NN) was proposed to classify the input face based on the training faces images. The results showed a significant improvement of 99.39% in the ORL dataset and 100% in the Face94 dataset classification accuracy. Moreover, a new metrics were introduced to quantify the exactness of classification and some errors of the classification can be corrected. All the above experiments were implemented in MATLAB environment

FACE CLASSIFICATION FOR AUTHENTICATION APPROACH BY USING WAVELET TRANSFORM AND STATISTICAL FEATURES SELECTION

This thesis consists of three parts: face localization, features selection and classification process. Three methods were proposed to locate the face region in the input image. Two of them based on pattern (template) Matching Approach, and the other based on clustering approach. Five datasets of faces namely: YALE database, MIT-CBCL database, Indian database, BioID database and Caltech database were used to evaluate the proposed methods. For the first method, the template image is prepared previously by using a set of faces. Later, the input image is enhanced by applying n-means kernel to decrease the image noise. Then Normalized Correlation (NC) is used to measure the correlation coefficients between the template image and the input image regions. For the second method, instead of using n-means kernel, an optimized metrics are used to measure the difference between the template image and the input image regions. In the last method, the Modified K-Means Algorithm was used to remove the non-face regions in the input image. The above-mentioned three methods showed accuracy of localization between 98% and 100% comparing with the existed methods. In the second part of the thesis, Discrete Wavelet Transform (DWT) utilized to transform the input image into number of wavelet coefficients. Then, the coefficients of weak statistical energy less than certain threshold were removed, and resulted in decreasing the primary wavelet coefficients number up to 98% out of the total coefficients. Later, only 40% statistical features were extracted from the hight energy features by using the variance modified metric. During the experimental (ORL) Dataset was used to test the proposed statistical method. Finally, Cluster-K-Nearest Neighbor (C-K-NN) was proposed to classify the input face based on the training faces images. The results showed a significant improvement of 99.39% in the ORL dataset and 100% in the Face94 dataset classification accuracy. Moreover, a new metrics were introduced to quantify the exactness of classification and some errors of the classification can be corrected. All the above experiments were implemented in MATLAB environment

Novel methods of object recognition and fault detection applied to non-destructive testing of rail’s surface during production

A series of rail image inspection algorithms have been developed for Tata Steels Scunthorpe rail production line. The following thesis describes the contributions made by the author in the design and application of these algorithms. A fully automated rail inspection system that has never been implemented before in any such company or setup has been developed. An industrial computer vision system (JLI) already exists for the image acquisition of rails during production at a rail manufacturing plant in Scunthorpe. An automated inspection system using the same JLI vision system has been developed for the detection of rail‟s surface defects during manufacturing process. This is to complement the human factor by developing a fully automated image processing based system to recognize the faults with an improved efficiency and to allow an exhaustive detection on the entire rail in production. A set of bespoke algorithms has been developed from a plethora of available image processing techniques to extract and identify components in an image of rail in order to detect abnormalities. This has been achieved through offline processing of the rail images using the blended use of different object recognition and image processing techniques, in particular, variation of standard image processing techniques. Several edge detection methods as well as adapted well known Artificial Neural Network and Principal Component Analysis techniques for fault detection on rail have been developed. A combination of customised existing image algorithms and newly developed algorithms have been put together to perform the efficient defect detection. The developed system is fast, reliable and efficient for detection of unique artefacts occurring on the rail surface during production followed by fault classification on the rail imaging system. Extensive testing shows that the defect detection techniques developed for automated rail inspection is capable of detecting more than 90% of the defects present in the available data set of rail images, which has more than 100,000 images under investigation. This demonstrates the efficiency and accuracy of the algorithms developed in this work