QCS: A System for Querying, Clustering and Summarizing Documents

Information retrieval systems consist of many complicated components. Research and development of such systems is often hampered by the difficulty in evaluating how each particular component would behave across multiple systems. We present a novel integrated information retrieval system—the Query, Cluster, Summarize (QCS) system—which is portable, modular, and permits experimentation with different instantiations of each of the constituent text analysis components. Most importantly, the combination of the three types of methods in the QCS design improves retrievals by providing users more focused information organized by topic. We demonstrate the improved performance by a series of experiments using standard test sets from the Document Understanding Conferences (DUC) as measured by the best known automatic metric for summarization system evaluation,ROUGE. Although the DUC data and evaluations were originally designed to test multidocument summarization, we developed a framework to extend it to the task of evaluation for each of the three components: query, clustering, and summarization. Under this framework, we then demonstrate that the QCS system (end-to-end) achieves performance as good as or better than the best summarization engines. Given a query, QCS retrieves relevant documents, separates the retrieved documents into topic clusters, and creates a single summary for each cluster. In the current implementation, Latent Semantic Indexing is used for retrieval

Diversity - a new mode of incorporation?

Faist T. Diversity - a new mode of incorporation? Ethnic and Racial Studies. 2009;32(1):171-190.Lately, cultural diversity in Western societies has, in terms of religions, languages, ethnic we-groups, transnational ties, and countries of origin, once more undergone immense growth. Modes of migrant incorporation reflect endeavours to respond to this change. While some approaches such as assimilation and multiculturalism emphasize the social integration of migrants in the host societies, the vague term 'diversity' harbours innovative measures in two respects. First, diversity addresses not only the incorporation of migrants, but also how societies and particularly their organizations deal with cultural pluralism. Second, diversity can then be understood both as an individual competence of migrants as members of organizations and the civil sphere, and as a set of programmes which organizations adopt to address cultural pluralism. Also, novel forms of diversity have emerged, such as transnationality. Yet in the absence of a rights-based foundation the question arises of how social inequality can be dealt with