Initial specification of the evaluation tasks "Use cases to bridge validation and benchmarking" PROMISE Deliverable 2.1

Evaluation of multimedia and multilingual information access systems needs to be performed from a usage oriented perspective. This document outlines use cases from the three use case domains of the PROMISE project and gives some initial pointers to how their respective characteristics can be extrapolated to determine and guide evaluation activities, both with respect to benchmarking and to validation of the usage hypotheses. The use cases will be developed further during the course of the evaluation activities and workshops projected to occur in coming CLEF conferences

Use Case Oriented Medical Visual Information Retrieval & System Evaluation

Large amounts of medical visual data are produced daily in hospitals, while new imaging techniques continue to emerge. In addition, many images are made available continuously via publications in the scientific literature and can also be valuable for clinical routine, research and education. Information retrieval systems are useful tools to provide access to the biomedical literature and fulfil the information needs of medical professionals. The tools developed in this thesis can potentially help clinicians make decisions about difficult diagnoses via a case-based retrieval system based on a use case associated with a specific evaluation task. This system retrieves articles from the biomedical literature when querying with a case description and attached images. This thesis proposes a multimodal approach for medical case-based retrieval with focus on the integration of visual information connected to text. Furthermore, the ImageCLEFmed evaluation campaign was organised during this thesis promoting medical retrieval system evaluation

Continuous evaluation of digital libraries: a software for automatically assessing performances

Al fine di garantire che i sistemi di reperimento di informazioni abbiano prestazioni adeguate, si eseguono delle attività di valutazione. Spesso queste attività vengono eseguite all’interno di campagne di valutazione; un celebre esempio di campagna di valutazione è la CLEF Initiative. A partire dal 2005, essa si appoggia a DIRECT, piattaforma sviluppata dal Dipartimento di Ingegneria dell’Informazione dell’Università di Padova e che fornisce un sistema centrale per l’accesso e la gestione dei dati prodotti nel corso della campagna. DIRECT ha recentemente implementato la Rete di Eccellenza PROMISE, che punta all’automazione di molti dei processi svolti nel corso di una campagna di valutazione. Nel corso della tesi è stato sviluppato un componente che espanderà le funzionalità offerte da PROMISE: in particolare, il componente sviluppato permette di automatizzare lo svolgimento degli esperiment

Cultural Heritage on line

The 2nd International Conference "Cultural Heritage online – Empowering users: an active role for user communities" was held in Florence on 15-16 December 2009. It was organised by the Fondazione Rinascimento Digitale, the Italian Ministry for Cultural Heritage and Activities and the Library of Congress, through the National Digital Information Infrastructure and Preservation Program - NDIIP partners. The conference topics were related to digital libraries, digital preservation and the changing paradigms, focussing on user needs and expectations, analysing how to involve users and the cultural heritage community in creating and sharing digital resources. The sessions investigated also new organisational issues and roles, and cultural and economic limits from an international perspective

Investigating the tripartite aspects of transactional distance in a blended multimedia adult literacy programme

This study investigates the role of transactional distance in an Adult Basic Education and Training Level One (ABET L1) multimedia, blended learning programme. While empirical research acknowledges that dialogue assumes an important role in mediating the communicative chasm between adult learner and facilitator, how this is mediated through a blended multimedia methodology at ABET L1 , is not clear. This study attempts to investigate some of the complexities in this mediation. The study focused primarily on the interplay of dialogue in relation to programme structure, learner autonomy, and how these variables influenced adult learning. The study consisted of a cohort sample of 20 ABET Level 1 learners on a blended learning programme at a food production plant in Cape Town. A mixed methods approach incorporating both quantitative and qualitative instruments was used to answer the research questions. A range of data collection instruments, namely a survey, interviews, and observations, was utilized to gather and analyze the data. With a particular focus on dialogue, this approach allowed the study to investigate the influence and interaction of the tripartite variables of Transactional Distance Theory on adult learning at ABET L1 and the scope of transactional distance created by these interactions. Both quantitative and qualitative findings suggest that in the absence of constructive, meaningful dialogue and scaffolded learning practices, barriers to learning are accentuated in this blended learning environment and results in increased transactional distance. Findings from the data suggest that improved regular face-to-face facilitator contact throughout the programme could avert learner frustration and demotivation and thereby decrease transactional distance. General consensus is that dialogue plays a critical role in mediating adult learning. While multimedia approaches to blended adult learning in South Africa appear to provide appropriate and effective learning methodologies to address adult literacy education, the findings of this research study suggest that the complexity of literacy acquisition, particularly at ABET Level 1, requires much more than technological methods if literacy is to be seen in broader social and communicative terms. The findings of this study nevertheless remain anticipative that a synergistic and productive codependence can be negotiated between these loci. This particular learning environment could be improved by reassessing the interplay of dialogue, programme structure, and learner autonomy, hereby potentially reducing transactional distance by being cognizant that adult learning is not a homogenous enterprise unmindful of the nature of adult learners and the critical contexts of adult learning

ECLAP 2012 Conference on Information Technologies for Performing Arts, Media Access and Entertainment

It has been a long history of Information Technology innovations within the Cultural Heritage areas. The Performing arts has also been enforced with a number of new innovations which unveil a range of synergies and possibilities. Most of the technologies and innovations produced for digital libraries, media entertainment and education can be exploited in the field of performing arts, with adaptation and repurposing. Performing arts offer many interesting challenges and opportunities for research and innovations and exploitation of cutting edge research results from interdisciplinary areas. For these reasons, the ECLAP 2012 can be regarded as a continuation of past conferences such as AXMEDIS and WEDELMUSIC (both pressed by IEEE and FUP). ECLAP is an European Commission project to create a social network and media access service for performing arts institutions in Europe, to create the e-library of performing arts, exploiting innovative solutions coming from the ICT

Organisations as complex adaptive systems : implications for the design of information systems

Today a paradigm shift in the field of organisation and management theories is no longer disputed and the need to switch from the Command-and-Control to the Leaming Organisation Paradigm (LOP) in the area of organisational theory is well understood. However, it is less well appreciated that learning organisations cannot operate effectively if supported by centralised databases and tailor-made application programs. LOP emphasises adaptability, flexibility, participation and learning. It is important to understand that the changes in organisational and management strategies will not on their own be able to produce the desired effects unless they are supported by appropriate changes in organisational culture, and by effective information systems. This research demonstrates that conventional information system strategies and development methods are no longer adequate. Information system strategies must respond to these needs of the LOP and incorporate new information systems that are capable of evolving, adapting and responding to the constantly changing business environment. The desired adaptability, flexibility and agility in information systems for LOP can be achieved by exploiting the technologies of the Internet, World Wide Web, intelligent agents and intranets. This research establishes that there is a need for synergy between organisational structures and organisational information systems. To obtain this desired synergy it is essential that new information systems be designed as an integral part of the learning organisational structure itself. Complexity theory provides a new set of metaphors and a host of concepts for the understanding of organisations as complex adaptive systems. This research introduces the principles of Complex Adaptive Systems and draws on their significance for designing the information systems needed to support the new generation of learning organisations. The search for new models of information system strategies for today's dynamic world of business points to the 'swarm models' observed in Nature

Ewaluacja skuteczności systemów wyszukiwania informacji. Wyniki eksperymentu Polish Task realizowanego w ramach Conference and Labs of the Evaluation Forum (CLEF) 2012

W niniejszym artykule prezentujemy realizację laboratorium ewaluacyjnego CLEF (Conference and Labs of the Evaluation Forum) ze specjalnym uwzględnieniem kampanii CHiC (Cultural Heritage in CLEF). Opisujemy realizację oraz wyniki zadania Polish Task in ChiC. W artykule zaprezentowano wnioski z realzacji zadania. Zostały omówione wyniki uzyskane przez uczestników zadania przy użyciu różnych strategii indeksowania oraz wyszukiwania zasobów. Porównaliśmy efektywność metod tf-idf, OKAPI, DFR oraz data fusion.The article presents the design of CLEF (Conference and Labs of the Evaluation Forum) evaluation labs with special attention paid to CHiC (Cultural Heritage in CLEF). We describe design of Polish Task in CHiClab and discuss conclusions from lab realisation. We discuss results achieved by different participants using different indexing and matching approaches. Efficiency of tf-idf, OKAPI, DFR and data fusion was compared and analysed