Content Recognition and Context Modeling for Document Analysis and Retrieval

The nature and scope of available documents are changing significantly in many areas of document analysis and retrieval as complex, heterogeneous collections become accessible to virtually everyone via the web. The increasing level of diversity presents a great challenge for document image content categorization, indexing, and retrieval. Meanwhile, the processing of documents with unconstrained layouts and complex formatting often requires effective leveraging of broad contextual knowledge. In this dissertation, we first present a novel approach for document image content categorization, using a lexicon of shape features. Each lexical word corresponds to a scale and rotation invariant local shape feature that is generic enough to be detected repeatably and is segmentation free. A concise, structurally indexed shape lexicon is learned by clustering and partitioning feature types through graph cuts. Our idea finds successful application in several challenging tasks, including content recognition of diverse web images and language identification on documents composed of mixed machine printed text and handwriting. Second, we address two fundamental problems in signature-based document image retrieval. Facing continually increasing volumes of documents, detecting and recognizing unique, evidentiary visual entities (\eg, signatures and logos) provides a practical and reliable supplement to the OCR recognition of printed text. We propose a novel multi-scale framework to detect and segment signatures jointly from document images, based on the structural saliency under a signature production model. We formulate the problem of signature retrieval in the unconstrained setting of geometry-invariant deformable shape matching and demonstrate state-of-the-art performance in signature matching and verification. Third, we present a model-based approach for extracting relevant named entities from unstructured documents. In a wide range of applications that require structured information from diverse, unstructured document images, processing OCR text does not give satisfactory results due to the absence of linguistic context. Our approach enables learning of inference rules collectively based on contextual information from both page layout and text features. Finally, we demonstrate the importance of mining general web user behavior data for improving document ranking and other web search experience. The context of web user activities reveals their preferences and intents, and we emphasize the analysis of individual user sessions for creating aggregate models. We introduce a novel algorithm for estimating web page and web site importance, and discuss its theoretical foundation based on an intentional surfer model. We demonstrate that our approach significantly improves large-scale document retrieval performance

Video content analysis for automated detection and tracking of humans in CCTV surveillance applications

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The problems of achieving high detection rate with low false alarm rate for human detection and tracking in video sequence, performance scalability, and improving response time are addressed in this thesis. The underlying causes are the effect of scene complexity, human-to-human interactions, scale changes, and scene background-human interactions. A two-stage processing solution, namely, human detection, and human tracking with two novel pattern classifiers is presented. Scale independent human detection is achieved by processing in the wavelet domain using square wavelet features. These features used to characterise human silhouettes at different scales are similar to rectangular features used in [Viola 2001]. At the detection stage two detectors are combined to improve detection rate. The first detector is based on shape-outline of humans extracted from the scene using a reduced complexity outline extraction algorithm. A Shape mismatch measure is used to differentiate between the human and the background class. The second detector uses rectangular features as primitives for silhouette description in the wavelet domain. The marginal distribution of features collocated at a particular position on a candidate human (a patch of the image) is used to describe statistically the silhouette. Two similarity measures are computed between a candidate human and the model histograms of human and non human classes. The similarity measure is used to discriminate between the human and the non human class. At the tracking stage, a tracker based on joint probabilistic data association filter (JPDAF) for data association, and motion correspondence is presented. Track clustering is used to reduce hypothesis enumeration complexity. Towards improving response time with increase in frame dimension, scene complexity, and number of channels; a scalable algorithmic architecture and operating accuracy prediction technique is presented. A scheduling strategy for improving the response time and throughput by parallel processing is also presented

Progame:event-based machine learning approach for in-game marketing

Abstract. There’s been a significant growth in the gaming industry, which has lead to an increased number of collected player and usage data, including game events, player interactions, the connections between players and individual preferences. Such big data has many use cases such as the identification of gaming bottlenecks, detection and prediction of anomalies and suspicious usage patterns for security, and real time offer specification via fine-grained user profiling based on their interest profiles. Offering personalized offer timing could reduce product cannibalization, and ethical methods increase the trust of customers. The goal of this thesis is to predict the value and time of the next in-game purchase in a mobile game. Using data aggregation, event-based purchase data, daily in-game behaviour metrics and session data are combined into a single data table, from which samples of 50 000 data points are taken. The features are analyzed for linear correlation with the labels, and their combinations are used as input for three machine learning algorithms: Random Forest, Support Vector Machine and Multi-Layer Perceptron. Both purchase value and purchase time are correlated with features related to previous purchase behaviour. Multi-Layer Perceptron showed the lowest error in predicting both labels, showing an improvement of 22,0% for value in USD and 20,7% for days until purchase compared to a trivial baseline predictor. For ethical customer behaviour prediction, sharing of research knowledge and customer involvement in the data analysis process is suggested to build awareness.Progame : tapahtumapohjainen koneoppimisjärjestelmä pelinsisäiseen markkinointiin. Tiivistelmä. Peliteollisuuden kasvu on johtanut kerättävän pelaaja- ja käyttödatan määrään nousuun, koostuen mm. pelitapahtumista, interaktiodatasta, pelaajien välisistä yhteyksistä ja henkilökohtaisista mieltymyksistä. Tällaisella massadatalla on monia käyttötarkoituksia kuten tietoliikenteen teknisten rajoitusten tunnistaminen pelikäytössä, käyttäjien tavallisuudesta poikkeavan käytöksen tunnistaminen ja ennustaminen tietoturvatarkoituksiin, sekä reaaliaikainen tarjousten määrittäminen hienovaraisella käyttäjien mieltymysten profiloinnilla. Ostotarjousten henkilökohtaistaminen voi vähentää uusien tuotteiden aiheuttamaa vanhojen tuotteiden myynnin laskua, ja eettiset menetelmät parantavat asiakkaiden luottamusta. Tässä työssä ennustetaan asiakkaan seuraavan pelinsisäisen oston arvoa ja aikaa mobiilipelissä. Tapahtumapohjainen ostodata, päivittäiset pelin sisäiset metriikat ja sessiodata yhdistetään yhdeksi datataulukoksi, josta otetaan kerrallaan 50 000:n datarivin näytteitä. Jokaisen selittävän muuttujan lineaarinen korrelaatio ennustettavan muuttujan kanssa analysoidaan, ja niiden yhdistelmiä käytetään syötteenä kolmelle eri koneoppimismallille: satunnainen metsä (Random Forest), tukivektorikone (Support Vector Machine) ja monikerroksinen perseptroniverkko (Multi-Layer Perceptron). Tutkimuksessa havaittiin, että sekä tulevan oston arvo että ajankohta korreloivat aiemman ostokäyttäytymisen kanssa. Monikerroksisella perseptroniverkolla oli pienin virhe molemmille ennustettaville muuttujille, ja verrattuna triviaaliin vertailuennustimeen, se vähensi virhettä 22,0% arvon ennustamisessa ja 20,7% seuraavaan ostoon jäljellä olevien päivien ennustamisessa. Eettisen asiakkaiden käyttäytymisen ennustamisen varmistamiseksi ja tietoisuuden lisäämiseksi ehdotetaan tutkimustiedon jakamista ja asiakkaan ottamista mukaan analyysin tekemiseen

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table