Speaker Identification by BYY Automatic Local Factor Analysis based Three-Level Voting Combination

Local Factor Analysis (LFA) is known as more general and powerful than Gaussian Mixture Model (GMM) in unsupervised learning with local subspace structure analysis. In the literature of text-independent speaker identification, GMM has been widely used and investigated, with some preprocessing or postprocessing approaches, while there still lacks efforts on LFA for this task. In pursuit of fast implementation for LFA modeling, this paper focuses on the Bayesian Ying-Yang automatic learning with data smoothing based regularization (BYY-A), which makes automatic model selection during parameter learning. Furthermore for sequence classification, based on trained LFA models, we design and analyze a three-level combination, namely sequence, classifier and committee, respectively. Different combination approaches are designed with variant sequential topologies and voting schemes. Experimental results on the KING speech corpus demonstrate the proposed approaches' effectiveness and potentials

Speaker Identification by BYY Automatic Local Factor Analysis based Three-Level Voting Combination

Local Factor Analysis (LFA) is known as more general and powerful than Gaussian Mixture Model (GMM) in unsupervised learning with local subspace structure analysis. In the literature of text-independent speaker identification, GMM has been widely used and investigated, with some preprocessing or postprocessing approaches, while there still lacks efforts on LFA for this task. In pursuit of fast implementation for LFA modeling, this paper focuses on the Bayesian Ying-Yang automatic learning with data smoothing based regularization (BYY-A), which makes automatic model selection during parameter learning. Furthermore for sequence classification, based on trained LFA models, we design and analyze a three-level combination, namely sequence, classifier and committee, respectively. Different combination approaches are designed with variant sequential topologies and voting schemes. Experimental results on the KING speech corpus demonstrate the proposed approaches' effectiveness and potentials

Shaping the learning landscape in neural networks around wide flat minima

Learning in Deep Neural Networks (DNN) takes place by minimizing a non-convex high-dimensional loss function, typically by a stochastic gradient descent (SGD) strategy. The learning process is observed to be able to find good minimizers without getting stuck in local critical points, and that such minimizers are often satisfactory at avoiding overfitting. How these two features can be kept under control in nonlinear devices composed of millions of tunable connections is a profound and far reaching open question. In this paper we study basic non-convex one- and two-layer neural network models which learn random patterns, and derive a number of basic geometrical and algorithmic features which suggest some answers. We first show that the error loss function presents few extremely wide flat minima (WFM) which coexist with narrower minima and critical points. We then show that the minimizers of the cross-entropy loss function overlap with the WFM of the error loss. We also show examples of learning devices for which WFM do not exist. From the algorithmic perspective we derive entropy driven greedy and message passing algorithms which focus their search on wide flat regions of minimizers. In the case of SGD and cross-entropy loss, we show that a slow reduction of the norm of the weights along the learning process also leads to WFM. We corroborate the results by a numerical study of the correlations between the volumes of the minimizers, their Hessian and their generalization performance on real data.Comment: 37 pages (16 main text), 10 figures (7 main text

Graph Signal Processing: Overview, Challenges and Applications

Research in Graph Signal Processing (GSP) aims to develop tools for processing data defined on irregular graph domains. In this paper we first provide an overview of core ideas in GSP and their connection to conventional digital signal processing. We then summarize recent developments in developing basic GSP tools, including methods for sampling, filtering or graph learning. Next, we review progress in several application areas using GSP, including processing and analysis of sensor network data, biological data, and applications to image processing and machine learning. We finish by providing a brief historical perspective to highlight how concepts recently developed in GSP build on top of prior research in other areas.Comment: To appear, Proceedings of the IEE

Channeling Change: Making Collective Impact Work

Large-scale social change requires broad cross-sector coordination, yet the social sector remains focused on the isolated intervention of individual organizations. Substantially greater progress could be made in alleviating many of our most serious and complex social problems if nonprofits, governments, businesses, and the public were brought together around a common agenda to create collective impact. Published in the Stanford Social Innovation Review, Winter 2011

Access to basic education in Ghana: politics, policies and progress

This monograph examines the history and politics of educational reform in Ghana, focusing on the issue of access to basic education in the post-colonial period. The monograph employs data from a series of interviews conducted with senior policy-makers, implementers and researchers, as well as drawing on documentary sources, to explore the drivers and inhibitors of change at the political, bureaucratic and grass-roots levels. It describes the patterns of change in relation to enrolment and outlines the key policies adopted through from the British colonial administration to the various independent regimes, authoritarian and democratic. Progress in universalising access has been substantial and basic education indicators in Ghana, both in early post-colonial times and today, stand out positively when compared to most countries in sub-Saharan Africa. The study explores the nature of the domestic political and administrative machinery which has enabled comparative success in enrolment growth in Ghana, attending also to the importance of political will as well as to shifting patterns of international and donor influence. The study draws out key tensions in education policy making, including tensions between the goals of access, equity, quality and relevance; those between academic and vocational orientations; those between elite and popular interests and those between political and technical imperatives. The processes of reform begun by the Kwapong and Dzobo committees and continued through to the fCUBE policy are examined in detail and the underlying aims and objectives of these processes are shown to share a number of common although sometimes mutually conflicting features. Interview data allow a nuanced interpretation of both impetus and resistance to policy formulation and implementation. The reforms of 1987 are shown to be critical in the development of the universal basic education policies that emerged subsequently and those later policies are considered partly as responses to unrealised objectives from 1987. Following the restoration of democratic government in Ghana, the establishment of a constitutional commitment to universal basic education in 1992 provided a lasting and binding responsibility for the state, which was followed by a comprehensive policy in fCUBE. Subsequently education policy has played an important role in political manifesto pledges. The monograph concludes by considering the election pledges of the 2008 Ghana Government, their provenance and initial indications of their implementation and finally summarises its findings on progress and on the importance of policy, regime, political will, and the drivers and inhibitors of reform implementation in relation to the pursuit of basic education for all in historical perspective