Understanding egocentric human actions with temporal decision forests

Understanding human actions is a fundamental task in computer vision with a wide range of applications including pervasive health-care, robotics and game control. This thesis focuses on the problem of egocentric action recognition from RGB-D data, wherein the world is viewed through the eyes of the actor whose hands describe the actions. The main contributions of this work are its findings regarding egocentric actions as described by hands in two application scenarios and a proposal of a new technique that is based on temporal decision forests. The thesis first introduces a novel framework to recognise fingertip writing in mid-air in the context of human-computer interaction. This framework detects whether the user is writing and tracks the fingertip over time to generate spatio-temporal trajectories that are recognised by using a Hough forest variant that encourages temporal consistency in prediction. A problem with using such forest approach for action recognition is that the learning of temporal dynamics is limited to hand-crafted temporal features and temporal regression, which may break the temporal continuity and lead to inconsistent predictions. To overcome this limitation, the thesis proposes transition forests. Besides any temporal information that is encoded in the feature space, the forest automatically learns the temporal dynamics during training, and it is exploited in inference in an online and efficient manner achieving state-of-the-art results. The last contribution of this thesis is its introduction of the first RGB-D benchmark to allow for the study of egocentric hand-object actions with both hand and object pose annotations. This study conducts an extensive evaluation of different baselines, state-of-the art approaches and temporal decision forest models using colour, depth and hand pose features. Furthermore, it extends the transition forest model to incorporate data from different modalities and demonstrates the benefit of using hand pose features to recognise egocentric human actions. The thesis concludes by discussing and analysing the contributions and proposing a few ideas for future work.Open Acces

Coding of object parts, view, orientation and size in the temporal cortex of the macaque

The study examined the importance of (1) component parts, (2) view, (3) orientation and (4) size in the neural encoding of the sight of a complex object in the temporal cortex of the macaque. Studies focused on cells selectively responsive to the sight of the head/body but unresponsive to control stimuli. (1) Cells responsive to the static whole body were tested with two component parts of the body. 44% (29/66) of cells responded to the whole body and to one of the two body regions tested: 23 to the head; 6 to the body. 36% (24/66) responded independently to both regions of the body when tested in isolation. The remaining cells were selective for the entire body and unresponsive to component parts. Similar selectivity for component parts was observed amongst cells responsive to moving heads/bodies (18 cells tested). (2) 90% (66/73) of cells (selectively responsive to static or moving head/bodies) tested were sensitive to perspective view (viewer-centred). Comparable levels of view sensitivity were found for responses to the whole body and its parts. Contrary to some influential models of object recognition these results indicate view-specific processing for both the appearance of separate object components and for integration of information across components. (3) The majority of cells tested (18/25, 72%) were selectively responsive to a particular orientation in the picture plane of the static whole body stimulus. 7 cells generalised across all orientations (4 cell with pure generalisation; 3 cells with superimposed orientation tuning). Of all cells sensitive to orientation, the majority (15/21, 71%) were tuned to the upright image. (4) The majority of cells tested (81%, 13/16) were selective for a particular stimulus size. The remaining cells (3/16) showed generalisation across a 4 fold decrease in size from life-sized. Interestingly, all size sensitive cells were tuned to life-sized stimuli. These results do not support previous suggestions that cells responsive to the head and body are selective to the view but generalise across orientation and size. Here, extensive selectivity for size and orientation is reported. It is suggested that object part, view, orientation and size specific responses might be pooled to obtain generalising responses. Experience appears to affect neuronal coding in two ways: a) Cells become selective for multiple object components due to spatial and temporal association between parts; and b) more cells become tuned to views, orientations and image sizes commonly experienced

Automated Deformable Mapping Methods to Relate Corresponding Lesions in 3D X-ray and 3D Ultrasound Breast Images

Mammography is the current standard imaging method for detecting breast cancer by using x-rays to produce 2D images of the breast. However, with mammography alone there is difficulty determining whether a lesion is benign or malignant and reduced sensitivity to detecting lesions in dense breasts. Ultrasound imaging used in conjunction with mammography has shown valuable contributions for lesion characterization by differentiating between solid and cystic lesions. Conventional breast ultrasound has high false positive rates; however, it has shown improved abilities to detect lesions in dense breasts. Breast ultrasound is typically performed freehand to produce anterior-to-posterior 2D images in a different geometry (supine) than mammography (upright). This difference in geometries is likely responsible for the finding that at least 10% of the time lesions found in the ultrasound images do not correspond with lesions found in mammograms. To solve this problem additional imaging techniques must be investigated to aid a radiologist in identifying corresponding lesions in the two modalities to ensure early detection of a potential cancer. This dissertation describes and validates automated deformable mapping methods to register and relate corresponding lesions between multi-modality images acquired using 3D mammography (Digital Breast Tomosynthesis (DBT) and dedicated breast Computed Tomography (bCT)) and 3D ultrasound (Automated Breast Ultrasound (ABUS)). The methodology involves the use of finite element modeling and analysis to simulate the differences in compression and breast orientation to better align lesions acquired from images from these modalities. Preliminary studies were performed using several multimodality compressible breast phantoms to determine breast lesion registrations between: i) cranio-caudal (CC) and mediolateral oblique (MLO) DBT views and ABUS, ii) simulated bCT and DBT (CC and MLO views), and iii) simulated bCT and ABUS. Distances between the centers of masses, dCOM, of corresponding lesions were used to assess the deformable mapping method. These phantom studies showed the potential to apply this technique for real breast lesions with mean dCOM registration values as low as 4.9 ± 2.4 mm for DBT (CC view) mapped to ABUS, 9.3 ± 2.8 mm for DBT (MLO view) mapped to ABUS, 4.8 ± 2.4 mm for bCT mapped to ABUS, 5.0 ± 2.2 mm for bCT mapped to DBT (CC view), and 4.7 ± 2.5 mm for bCT mapped to DBT (MLO view). All of the phantom studies showed that using external fiducial markers helped improve the registration capability of the deformable mapping algorithm. An IRB-approved proof-of-concept study was performed with patient volunteers to validate the deformable registration method on 5 patient datasets with a total of up to 7 lesions for DBT (CC and MLO views) to ABUS registration. Resulting dCOM’s using the deformable method showed statistically significant improvements over rigid registration techniques with a mean dCOM of 11.6 ± 5.3 mm for DBT (CC view) mapped to ABUS and a mean dCOM of 12.3 ± 4.8 mm for DBT (MLO view) mapped to ABUS. The present work demonstrates the potential for using deformable registration techniques to relate corresponding lesions in 3D x-ray and 3D ultrasound images. This methodology should improve a radiologists’ characterization of breast lesions which can reduce patient callbacks, misdiagnoses, additional patient dose and unnecessary biopsies. Additionally, this technique can save a radiologist time in navigating 3D image volumes and the one-to-one lesion correspondence between modalities can aid in the early detection of breast malignancies.PHDNuclear Engineering & Radiological SciencesUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/150042/1/canngree_1.pd

Neural mechanisms underlying the perception of socially relevant stimuli in the macaque monkey

Present knowledge indicates the importance of one region of monkey temporal association cortex, the superior temporal sulcus (STS), in predominantly high level analysis of 'biologically' important objects. To clarify and elaborate on the function of the monkey STS, the following questions are addressed: (1) what kind of tactile processing occurs in the polymodal STS and does it compare with the complex visual processing observed; (2) does behavioural sensitivity to face and body information parallel neural sensitivity (of STS cells) to the same stimulus dimensions; (3) does monkey STS ablation result in a behavioural indication of impairments in the perception of socially relevant stimuli; and (4) are visual cells in the STS sensitive to social communicational elements of facial or postural expression? Single-unit recording studies of the macaque STS (using standard techniques in awake, behaving animals) reveal a population of somatosensory neurones, with large receptive fields, sensitive only to unexpected (unpredictable) tactile stimulation. Complex tactual-visual interactions observed stress the importance of this dimension of processing. A separate population of visual cells exhibit sensitivity to compound facial expressions and head/body postures important in primate social communication. A behavioural study of monkeys' socio-emotional responses to configurational aspects of faces, the posture of the head and the interaction of form and motion, reveal their ability to discriminate salient cues in the context of social communication/interaction. It is tentatively shown that monkeys with the STS ablated are unable to make such discriminations, so reacting inappropriately to the stimuli (a symptom of Kluver-Bucy syndrome). The combined findings show that the STS performs a multimodal perceptual analysis of socially relevant stimuli, and suggest that the STS provides a sensory input to a limbic structure, such as the amygdala, through which it mediates appropriate emotional reactive behaviour

Machine Learning Methods with Noisy, Incomplete or Small Datasets

In many machine learning applications, available datasets are sometimes incomplete, noisy or affected by artifacts. In supervised scenarios, it could happen that label information has low quality, which might include unbalanced training sets, noisy labels and other problems. Moreover, in practice, it is very common that available data samples are not enough to derive useful supervised or unsupervised classifiers. All these issues are commonly referred to as the low-quality data problem. This book collects novel contributions on machine learning methods for low-quality datasets, to contribute to the dissemination of new ideas to solve this challenging problem, and to provide clear examples of application in real scenarios

Message from the President of the United States to the two Houses of Congress at the commencement of the First Session of the Twenty-Fourth Congress, 1835

Annual Message to Congress with Documents; Pres. Jackson. 7 Dec. SD 1, 24-1, v1, 428p. [279] or HD 2, 24-1, v1, 428p. [286] Removal of Indians; reports of Sec. of War, Indian Office, etc. [Vol. 286 also includes 24th Congress 2nd Session

[¹⁸F]fluorination of biorelevant arylboronic acid pinacol ester scaffolds synthesized by convergence techniques

Aim: The development of small molecules through convergent multicomponent reactions (MCR) has been boosted during the last decade due to the ability to synthesize, virtually without any side-products, numerous small drug-like molecules with several degrees of structural diversity.(1) The association of positron emission tomography (PET) labeling techniques in line with the “one-pot” development of biologically active compounds has the potential to become relevant not only for the evaluation and characterization of those MCR products through molecular imaging, but also to increase the library of radiotracers available. Therefore, since the [18F]fluorination of arylboronic acid pinacol ester derivatives tolerates electron-poor and electro-rich arenes and various functional groups,(2) the main goal of this research work was to achieve the 18F-radiolabeling of several different molecules synthesized through MCR. Materials and Methods: [18F]Fluorination of boronic acid pinacol esters was first extensively optimized using a benzaldehyde derivative in relation to the ideal amount of Cu(II) catalyst and precursor to be used, as well as the reaction solvent. Radiochemical conversion (RCC) yields were assessed by TLC-SG. The optimized radiolabeling conditions were subsequently applied to several structurally different MCR scaffolds comprising biologically relevant pharmacophores (e.g. β-lactam, morpholine, tetrazole, oxazole) that were synthesized to specifically contain a boronic acid pinacol ester group. Results: Radiolabeling with fluorine-18 was achieved with volumes (800 μl) and activities (≤ 2 GBq) compatible with most radiochemistry techniques and modules. In summary, an increase in the quantities of precursor or Cu(II) catalyst lead to higher conversion yields. An optimal amount of precursor (0.06 mmol) and Cu(OTf)2(py)4 (0.04 mmol) was defined for further reactions, with DMA being a preferential solvent over DMF. RCC yields from 15% to 76%, depending on the scaffold, were reproducibly achieved. Interestingly, it was noticed that the structure of the scaffolds, beyond the arylboronic acid, exerts some influence in the final RCC, with electron-withdrawing groups in the para position apparently enhancing the radiolabeling yield. Conclusion: The developed method with high RCC and reproducibility has the potential to be applied in line with MCR and also has a possibility to be incorporated in a later stage of this convergent “one-pot” synthesis strategy. Further studies are currently ongoing to apply this radiolabeling concept to fluorine-containing approved drugs whose boronic acid pinacol ester precursors can be synthesized through MCR (e.g. atorvastatin)

Baseball and Antitrust: The Legislative History of the Curt Flood Act of 1998

https://scholarship.law.nd.edu/law_books/1185/thumbnail.jp