Handwriting recognition using webcam for data entry

The Handwriting Recognition using Webcam for Data Entry project has its primary purpose to develop a system or algorithm that is robust enough to recognize numerical handwritings. A web camera is to be utilized to capture images of handwritten scores and question numbers on the examination score sheet in real time. It is then preprocessed and all the features are being fed into a neural network that is already been trained by various test samples. The outcome of the project should be able to obtain a system that is able to recognize handwritten numerical data with the lowest overshoot and errors. Several distinctive feature from each character is extracted using a few feature extraction methods, in which a comparison between three types of feature extraction modules were used. The first test was done with a neural network trained with only the Character Vector Module as its feature extraction method. A result that is far below the set point of the recognition accuracy was achieved, a mere average of 64.67% accuracy. However, the testing were later enhanced with another feature extraction module, which consists of the combination of Character Vector Module, Kirsch Edge Detection Module, Alphabet Profile Feature Extraction Module, Modified Character Module and Image Compression Module. The modules have its distinct characteristics which is trained using the Back-Propagation algorithm to cluster the pattern recognition capabilities among different samples of handwriting. Several untrained samples of numerical handwritten data were obtained at random from various people to be tested with the program. The second tests shows far greater results compared to the first test, have yielded an average of 84.52% accuracy. As the recognition results have not reached the target of 90%, further feature extraction modules are being recommended and an additional feature extraction module was added for the third test, which successfully yields 90.67%. With the timeframe target achieved, a robust data entry system was developed using web camera together with a user-friendly GUI (Graphical User Interface)

Pattern detection and recognition using over-complete and sparse representations

Recent research in harmonic analysis and mammalian vision systems has revealed that over-complete and sparse representations play an important role in visual information processing. The research on applying such representations to pattern recognition and detection problems has become an interesting field of study. The main contribution of this thesis is to propose two feature extraction strategies - the global strategy and the local strategy - to make use of these representations. In the global strategy, over-complete and sparse transformations are applied to the input pattern as a whole and features are extracted in the transformed domain. This strategy has been applied to the problems of rotation invariant texture classification and script identification, using the Ridgelet transform. Experimental results have shown that better performance has been achieved when compared with Gabor multi-channel filtering method and Wavelet based methods. The local strategy is divided into two stages. The first one is to analyze the local over-complete and sparse structure, where the input 2-D patterns are divided into patches and the local over-complete and sparse structure is learned from these patches using sparse approximation techniques. The second stage concerns the application of the local over-complete and sparse structure. For an object detection problem, we propose a sparsity testing technique, where a local over-complete and sparse structure is built to give sparse representations to the text patterns and non-sparse representations to other patterns. Object detection is achieved by identifying patterns that can be sparsely represented by the learned. structure. This technique has been applied. to detect texts in scene images with a recall rate of 75.23% (about 6% improvement compared with other works) and a precision rate of 67.64% (about 12% improvement). For applications like character or shape recognition, the learned over-complete and sparse structure is combined. with a Convolutional Neural Network (CNN). A second text detection method is proposed based on such a combination to further improve (about 11% higher compared with our first method based on sparsity testing) the accuracy of text detection in scene images. Finally, this method has been applied to handwritten Farsi numeral recognition, which has obtained a 99.22% recognition rate on the CENPARMI Database and a 99.5% recognition rate on the HODA Database. Meanwhile, a SVM with gradient features achieves recognition rates of 98.98% and 99.22% on these databases respectivel

Off-line Arabic Handwriting Recognition System Using Fast Wavelet Transform

In this research, off-line handwriting recognition system for Arabic alphabet is introduced. The system contains three main stages: preprocessing, segmentation and recognition stage. In the preprocessing stage, Radon transform was used in the design of algorithms for page, line and word skew correction as well as for word slant correction. In the segmentation stage, Hough transform approach was used for line extraction. For line to words and word to characters segmentation, a statistical method using mathematic representation of the lines and words binary image was used. Unlike most of current handwriting recognition system, our system simulates the human mechanism for image recognition, where images are encoded and saved in memory as groups according to their similarity to each other. Characters are decomposed into a coefficient vectors, using fast wavelet transform, then, vectors, that represent a character in different possible shapes, are saved as groups with one representative for each group. The recognition is achieved by comparing a vector of the character to be recognized with group representatives. Experiments showed that the proposed system is able to achieve the recognition task with 90.26% of accuracy. The system needs only 3.41 seconds a most to recognize a single character in a text of 15 lines where each line has 10 words on average

A novel approach to handwritten character recognition

A number of new techniques and approaches for off-line handwritten character recognition are presented which individually make significant advancements in the field. First. an outline-based vectorization algorithm is described which gives improved accuracy in producing vector representations of the pen strokes used to draw characters. Later. Vectorization and other types of preprocessing are criticized and an approach to recognition is suggested which avoids separate preprocessing stages by incorporating them into later stages. Apart from the increased speed of this approach. it allows more effective alteration of the character images since more is known about them at the later stages. It also allows the possibility of alterations being corrected if they are initially detrimental to recognition. A new feature measurement. the Radial Distance/Sector Area feature. is presented which is highly robust. tolerant to noise. distortion and style variation. and gives high accuracy results when used for training and testing in a statistical or neural classifier. A very powerful classifier is therefore obtained for recognizing correctly segmented characters. The segmentation task is explored in a simple system of integrated over-segmentation. Character classification and approximate dictionary checking. This can be extended to a full system for handprinted word recognition. In addition to the advancements made by these methods. a powerful new approach to handwritten character recognition is proposed as a direction for future research. This proposal combines the ideas and techniques developed in this thesis in a hierarchical network of classifier modules to achieve context-sensitive. off-line recognition of handwritten text. A new type of "intelligent" feedback is used to direct the search to contextually sensible classifications. A powerful adaptive segmentation system is proposed which. when used as the bottom layer in the hierarchical network. allows initially incorrect segmentations to be adjusted according to the hypotheses of the higher level context modules

Off-line Arabic Handwriting Recognition System Using Fast Wavelet Transform

In this research, off-line handwriting recognition system for Arabic alphabet is introduced. The system contains three main stages: preprocessing, segmentation and recognition stage. In the preprocessing stage, Radon transform was used in the design of algorithms for page, line and word skew correction as well as for word slant correction. In the segmentation stage, Hough transform approach was used for line extraction. For line to words and word to characters segmentation, a statistical method using mathematic representation of the lines and words binary image was used. Unlike most of current handwriting recognition system, our system simulates the human mechanism for image recognition, where images are encoded and saved in memory as groups according to their similarity to each other. Characters are decomposed into a coefficient vectors, using fast wavelet transform, then, vectors, that represent a character in different possible shapes, are saved as groups with one representative for each group. The recognition is achieved by comparing a vector of the character to be recognized with group representatives. Experiments showed that the proposed system is able to achieve the recognition task with 90.26% of accuracy. The system needs only 3.41 seconds a most to recognize a single character in a text of 15 lines where each line has 10 words on average

Machine Learning and Pedometers: An Integration-Based Convolutional Neural Network for Step Counting and Detection

This thesis explores a machine learning-based approach to step detection and counting for a pedometer. Our novelty is to analyze a window of time containing an arbitrary number of steps, and integrate the detected count using a sliding window technique. We compare the effectiveness of this approach against classic deterministic algorithms. While classic algorithms perform well during regular gait (e.g. walking or running), they can perform significantly worse during semi-regular and irregular gaits that still contribute to a person’s overall step count. These non-regular gaits can make up a significant portion of the daily step count for people, and an improvement to measuring these gaits can drastically improve the performance of the overall pedometer. Using data collected by 30 participants performing 3 different activities to simulate regular, semi-regular, and irregular gaits, a training and testing strategy was implemented using a sliding window algorithm of pedometer accelerometer data. Data was cut in rows representative of the sliding window, normalized according to the minimum and maximum values of the corresponding sensor-axis combination, and finally collated in specific training and holdout groups for validation purposes. Nine models were trained to predict a continuous count of steps within a given window, for each fold of our five-fold validation process. These nine models correspond to each gait and sensor combination from the collected data set. Once models are trained, they are evaluated against the holdout validation set to test for both run count accuracy (RCA), a measure of the pedometers detected step to actual step count, and step detection accuracy (SDA), a measure of how well the algorithm can predict the time of an actual step. These are obtained through an additional post-processing step that integrates the predicted steps per window over time in order to find the total count of steps within a given training data set. Additionally, an algorithm estimates the times when predicted steps occur by using the running count of total steps. Once testing is performed on all nine models, the process is repeated across all five folds to verify model architecture consistency throughout the entire data set. A window size test was implemented to vary the window size of the sliding window algorithm between 1 and 10 seconds to discover the effect of the sliding window size on the convolutional neural network\u27s step count and detection performance. Again, these tests were run across five different folds to ensure an accurate average measure of each model\u27s performance. By comparing the metrics of RCA and SDA between the machine-learning approach and other algorithms, we see that the method introduced in this thesis performs similarly or better than both a consumer pedometer device, as well as the three classic algorithms of peak detection, thresholding, and autocorrelation. It was found that with a window size of two seconds, this novel approach can detect steps with an overall average RCA of 0.99 and SDA of 0.88, better than any individual classic algorithm

A novel approach to handwritten character recognition

A number of new techniques and approaches for off-line handwritten character recognition are presented which individually make significant advancements in the field. First. an outline-based vectorization algorithm is described which gives improved accuracy in producing vector representations of the pen strokes used to draw characters. Later. Vectorization and other types of preprocessing are criticized and an approach to recognition is suggested which avoids separate preprocessing stages by incorporating them into later stages. Apart from the increased speed of this approach. it allows more effective alteration of the character images since more is known about them at the later stages. It also allows the possibility of alterations being corrected if they are initially detrimental to recognition. A new feature measurement. the Radial Distance/Sector Area feature. is presented which is highly robust. tolerant to noise. distortion and style variation. and gives high accuracy results when used for training and testing in a statistical or neural classifier. A very powerful classifier is therefore obtained for recognizing correctly segmented characters. The segmentation task is explored in a simple system of integrated over-segmentation. Character classification and approximate dictionary checking. This can be extended to a full system for handprinted word recognition. In addition to the advancements made by these methods. a powerful new approach to handwritten character recognition is proposed as a direction for future research. This proposal combines the ideas and techniques developed in this thesis in a hierarchical network of classifier modules to achieve context-sensitive. off-line recognition of handwritten text. A new type of "intelligent" feedback is used to direct the search to contextually sensible classifications. A powerful adaptive segmentation system is proposed which. when used as the bottom layer in the hierarchical network. allows initially incorrect segmentations to be adjusted according to the hypotheses of the higher level context modules

Weakly-Supervised Anomaly Detection in Surveillance Videos Based on Two-Stream I3D Convolution Network

The widespread adoption of city surveillance systems has led to an increase in the use of surveillance videos for maintaining public safety and security. This thesis tackles the problem of detecting anomalous events in surveillance videos. The goal is to automatically identify abnormal events by learning from both normal and abnormal videos. Most of previous works consider any deviation from learned normal patterns as an anomaly, which may not always be valid since the same activity could be normal or abnormal under different circumstances. To address this issue, the thesis utilizes the Two-Stream Inflated 3D (I3D) Convolutional Networks to extract spatial and temporal video features and demonstrates how it outperforms the 3D Convolutional Network (C3D) used in prior work as feature extractor. To avoid annotating abnormal activities in training videos, a weakly supervised anomaly detection model is implemented based on the Multiple Instance Learning (MIL) framework. The model considers normal and abnormal videos as bags and video clips as instances, learns a ranking model to predict high anomaly scores for video clips containing anomalies. The thesis further shows that the choice of features input, such as concatenating RGB and flow features, and careful choice of optimization settings, such as optimizer, can significantly improve the performance of the anomaly detection model on some evaluation metrics