Learning Co-Sparse Analysis Operators with Separable Structures

In the co-sparse analysis model a set of filters is applied to a signal out of the signal class of interest yielding sparse filter responses. As such, it may serve as a prior in inverse problems, or for structural analysis of signals that are known to belong to the signal class. The more the model is adapted to the class, the more reliable it is for these purposes. The task of learning such operators for a given class is therefore a crucial problem. In many applications, it is also required that the filter responses are obtained in a timely manner, which can be achieved by filters with a separable structure. Not only can operators of this sort be efficiently used for computing the filter responses, but they also have the advantage that less training samples are required to obtain a reliable estimate of the operator. The first contribution of this work is to give theoretical evidence for this claim by providing an upper bound for the sample complexity of the learning process. The second is a stochastic gradient descent (SGD) method designed to learn an analysis operator with separable structures, which includes a novel and efficient step size selection rule. Numerical experiments are provided that link the sample complexity to the convergence speed of the SGD algorithm.Comment: 11 pages double column, 4 figures, 3 table

Separable Cosparse Analysis Operator Learning

The ability of having a sparse representation for a certain class of signals has many applications in data analysis, image processing, and other research fields. Among sparse representations, the cosparse analysis model has recently gained increasing interest. Many signals exhibit a multidimensional structure, e.g. images or three-dimensional MRI scans. Most data analysis and learning algorithms use vectorized signals and thereby do not account for this underlying structure. The drawback of not taking the inherent structure into account is a dramatic increase in computational cost. We propose an algorithm for learning a cosparse Analysis Operator that adheres to the preexisting structure of the data, and thus allows for a very efficient implementation. This is achieved by enforcing a separable structure on the learned operator. Our learning algorithm is able to deal with multidimensional data of arbitrary order. We evaluate our method on volumetric data at the example of three-dimensional MRI scans.Comment: 5 pages, 3 figures, accepted at EUSIPCO 201

Joint Spatial-Angular Sparse Coding, Compressed Sensing, and Dictionary Learning for Diffusion MRI

Neuroimaging provides a window into the inner workings of the human brain to diagnose and prevent neurological diseases and understand biological brain function, anatomy, and psychology. Diffusion Magnetic Resonance Imaging (dMRI) is an emerging medical imaging modality used to study the anatomical network of neurons in the brain, which form cohesive bundles, or fiber tracts, that connect various parts of the brain. Since about 73% of the brain is water, measuring the flow, or diffusion of water molecules in the presence of fiber bundles, allows researchers to estimate the orientation of fiber tracts and reconstruct the internal wiring of the brain, in vivo. Diffusion MRI signals can be modeled within two domains: the spatial domain consisting of voxels in a brain volume and the diffusion or angular domain, where fiber orientation is estimated in each voxel. Researchers aim to estimate the probability distribution of fiber orientation in every voxel of a brain volume in order to trace paths of fiber tracts from voxel to voxel over the entire brain. Therefore, the traditional framework for dMRI processing and analysis has been from a voxel-wise vantage point with added spatial regularization considered post-hoc. In contrast, we propose a new joint spatial-angular representation of dMRI data which pairs signals in each voxel with the global spatial environment, jointly. This has the ability to improve many aspects of dMRI processing and analysis and re-envision the core representation of dMRI data from a local perspective to a global one. In this thesis, we propose three main contributions which take advantage of such joint spatial-angular representations to improve major machine learning tasks applied to dMRI: sparse coding, compressed sensing, and dictionary learning. First, we will show that we can achieve sparser representations of dMRI by utilizing a global spatial-angular dictionary instead of a purely voxel-wise angular dictionary. As dMRI data is very large in size, we provide a number of novel extensions to popular spare coding algorithms that perform efficient optimization on a global-scale by exploiting the separability of our dictionaries over the spatial and angular domains. Next, compressed sensing is used to accelerate signal acquisition based on an underlying sparse representation of the data. We will show that our proposed representation has the potential to push the limits of the current state of scanner acceleration within a new compressed sensing model for dMRI. Finally, sparsity can be further increased by learning dictionaries directly from datasets of interest. Prior dictionary learning for dMRI learn angular dictionaries alone. Our third contribution is to learn spatial-angular dictionaries jointly from dMRI data directly to better represent the global structure. Traditionally, the problem of dictionary learning is non-convex with no guarantees of finding a globally optimal solution. We derive the first theoretical results of global optimality for this class of dictionary learning problems. We hope the core foundation of a joint spatial-angular representation will open a new perspective on dMRI with respect to many other processing tasks and analyses. In addition, our contributions are applicable to any general signal types that can benefit from separable dictionaries. We hope the contributions in this thesis may be adopted in the larger signal processing, computer vision, and machine learning communities. dMRI signals can be modeled within two domains: the spatial domain consisting of voxels in a brain volume and the diffusion or angular domain, where fiber orientation is estimated in each voxel. Computationally speaking, researchers aim to estimate the probability distribution of fiber orientation in every voxel of a brain volume in order to trace paths of fiber tracts from voxel to voxel over the entire brain. Therefore, the traditional framework for dMRI processing and analysis is from a voxel-wise, or angular, vantage point with post-hoc consideration of their local spatial neighborhoods. In contrast, we propose a new global spatial-angular representation of dMRI data which pairs signals in each voxel with the global spatial environment, jointly, to improve many aspects of dMRI processing and analysis, including the important need for accelerating the otherwise time-consuming acquisition of advanced dMRI protocols. In this thesis, we propose three main contributions which utilize our joint spatial-angular representation to improve major machine learning tasks applied to dMRI: sparse coding, compressed sensing, and dictionary learning. We will show that sparser codes are possible by utilizing a global dictionary instead of a voxel-wise angular dictionary. This allows for a reduction of the number of measurements needed to reconstruct a dMRI signal to increase acceleration using compressed sensing. Finally, instead of learning angular dictionaries alone, we learn spatial-angular dictionaries jointly from dMRI data directly to better represent the global structure. In addition, this problem is non-convex and so we derive the first theories to guarantee convergence to a global minimum. As dMRI data is very large in size, we provide a number of novel extensions to popular algorithms that perform efficient optimization on a global-scale by exploiting the separability of our global dictionaries over the spatial and angular domains. We hope the core foundation of a joint spatial-angular representation will open a new perspective on dMRI with respect to many other processing tasks and analyses. In addition, our contributions are applicable to any separable dictionary setting which we hope may be adopted in the larger image processing, computer vision, and machine learning communities

Tense and aspect in the Vetālapañcaviṅśati, a work of late classical Sanskrit

The loss of many of the finite verbal forms of Sanskrit between the Vedic period (1200- 700 B.C.) and the Classical period (400-1700 A.D.) has been well documented (Burrow 1955; Taraporewala 1967; Pap 1990; Masica 1991). By the Classical period, the three finite past tenses, the aorist, perfect and imperfect, had "fallen together" and were being used interchangeably to relate past events (Taraporewala 1967:76; Misra 1968:62; Whitney 1889/1967:201). -- This thesis investigated verbal forms of a text of late Classical Sanskrit, the Vetālapañcaviṅśati, ‘Twenty Five Tales of a Demon', with a view to discovering some of the directions taken in the post-Vedic process of "rebuilding" the verbal system. -- Jambhaladatta's version of the Vetālapañcaviṅśati was found to contain two systems used to indicate past events: the 'archaic' (including the aorist, imperfect and perfect) and the 'innovative' (including the P-oriented participle -ta and the A-oriented participle -tavant). The three 'old' tenses showed no significant semantic differences, consistent with their acknowledged "collapse", although the perfect did show a discourse function of indicating "finality". The P-oriented -ta participle was used three times as often as the A-oriented -tavant participle, indicating that in the 'new’ system, the syntactic shift from A- to P-orientation (as in Hock 1986) was well underway. -- The Historical Present, consistent with its function in all periods of the language, was heavily used for the "lively" narration of past events, both as a main verb, and as the auxiliary component of analytic forms. -- The 'new’ system also showed numerous analytic aspectual forms. To indicate imperfective aspect, constructions involving the present participle plus auxiliaries ✓sthā 'to stand’, ✓ās 'to stay,sit', and vidyate<✓vid ‘to find' were used extensively. The auxiliary vidyate was noted to be a recategorized middle voice verb, with features of both A- and P- orientation. Retrospective aspect was regularly indicated by constructions involving the PPP combined with the auxiliary ✓as 'to be; this construction was especially common in direct speech. -- The increase in analytic forms, clearly marked for imperfective aspect, was considered to be a strategy to redress an imbalance in the 'old' system of preterite tenses where imperfective aspect was under represented

Learning Theory and Approximation

The main goal of this workshop – the third one of this type at the MFO – has been to blend mathematical results from statistical learning theory and approximation theory to strengthen both disciplines and use synergistic effects to work on current research questions. Learning theory aims at modeling unknown function relations and data structures from samples in an automatic manner. Approximation theory is naturally used for the advancement and closely connected to the further development of learning theory, in particular for the exploration of new useful algorithms, and for the theoretical understanding of existing methods. Conversely, the study of learning theory also gives rise to interesting theoretical problems for approximation theory such as the approximation and sparse representation of functions or the construction of rich kernel reproducing Hilbert spaces on general metric spaces. This workshop has concentrated on the following recent topics: Pitchfork bifurcation of dynamical systems arising from mathematical foundations of cell development; regularized kernel based learning in the Big Data situation; deep learning; convergence rates of learning and online learning algorithms; numerical refinement algorithms to learning; statistical robustness of regularized kernel based learning

Rangga ngenandayin, lingbe berranben-nging-ngerri – Possession in Miriwoong, a non-Pama-Nyungan language of north-west Australia

In her PhD thesis Christina Ringel shows that factors such as animacy, semantic criteria and semantic domains, negation and the structural make-up of the sentence influence Miriwoong speaker’s choice among a variety of linguistic expressions of possessive relationships. Following ethnographic information and an analysis of language attitudes, she describes how she used language games as her main method to elicit data. Based on a discussion of the cultural concept of possession, typological predictions and data from neighbouring languages, she demonstrates that the Miriwoong data complies with some predictions concerning possessive constructions but not with others: With Dixon (1980) but contra McGregor (1990), Miriwoong data indicate that inalienability plays a role: The game data yields correlations between alienable possession and the Benefactive enclitic (BEN) and inalienable possession and the Indirect Object enclitic (IO). There is a tendency for the use of IO with respect to the body-parts and part-whole domains, but not the kinship domain. This distribution of marking over domains is in line with claims about Australian languages (Dixon 1980, Heine 1997) but not Stassen's (2009) typology for alienable vs. inalienable possession. In the latter, both body part and kinship relations are analysed as being defined by +Permanent Contact and -Control, i.e. inalienable possession. Miriwoong speakers are argued to make use of two out of four attributive possessive construction types and one out of four predicative verbless nominal clause types described by McGregor (2004) and Dixon (1980, 2002, 2009): One attributive type (a possessive pronoun indicating the possessor (PR) and a nominal specifying the possessee (PE)) is used extensively, another (juxtaposition of PR and PE NPs to express part-whole relations) is possible. One predicative type (a have-construction formed by a comitative-marked PE) is used widely, whereas the other three are not part of Miriwoong grammar

International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book

The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions. This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more