Master of Science

thesisConcept maps have been shown to have positive effects for students on recall. This is because, among other things, they are designed to show learners the relationships between concepts in a visuospatial way. However, it remains to be seen how concept maps affect deeper forms of learning, or whether it is the attention to the relationship between the concepts in the map or the concepts themselves that support the learning. This research examined the impact of the spatial organization of graphical search interfaces on deep learning as well as the impact of focusing student attention on the conceptual relationships between map nodes in this graphical search interface by asking them to generate information about those relationships. Results showed a nonsignificant trend suggesting that participants who were asked to generate information about the relationship between concepts showed greater recall when not learning from a concept map

Search Personalization: Knowledge-Based Recommendation in Digital Libraries

Recommendation engines have made great strides in understanding and implementing search personalization techniques to provide interesting and relevant documents to users. The latest research effort advances a new type of recommendation technique, Knowledge Based (KB) engines, that strive to understand the context of the user’s current information need and then filter information accordingly. The KB engine proposed in this paper requires less effort from the user in representing the search task and is the first of its kind implemented in a digital library setting. The KB engine performance was compared with Content Based (CB) and Collaborative Filtering (CF) recommendation techniques and the text search engine Lucene by asking sixty subjects to perform two different tasks to find relevant documents in a database of 212,000 documents from 22 National Science Digital Library (NSDL) collections. Our KB engine design outperforms CB, CF, and text search techniques in nearly all areas of evaluation

Contexts and Contributions: Building the Distributed Library

This report updates and expands on A Survey of Digital Library Aggregation Services, originally commissioned by the DLF as an internal report in summer 2003, and released to the public later that year. It highlights major developments affecting the ecosystem of scholarly communications and digital libraries since the last survey and provides an analysis of OAI implementation demographics, based on a comparative review of repository registries and cross-archive search services. Secondly, it reviews the state-of-practice for a cohort of digital library aggregation services, grouping them in the context of the problem space to which they most closely adhere. Based in part on responses collected in fall 2005 from an online survey distributed to the original core services, the report investigates the purpose, function and challenges of next-generation aggregation services. On a case-by-case basis, the advances in each service are of interest in isolation from each other, but the report also attempts to situate these services in a larger context and to understand how they fit into a multi-dimensional and interdependent ecosystem supporting the worldwide community of scholars. Finally, the report summarizes the contributions of these services thus far and identifies obstacles requiring further attention to realize the goal of an open, distributed digital library system

A Survey of Digital Library Aggregation Services

This report provides an overview of a diverse set of more than thirty digital library aggregation services, organizes them into functional clusters, and then evaluates them more fully from the perspective of an informed user. Most of the services under review rely wholly or partially on the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH), although some of them predate its inception and a few use predominantly Z39.50 protocols. In the opening section of this report, each service is annotated with its organizational affiliation, subject coverage, function, audience, status, and size. Critical issues surrounding each of these elements are presented in order to provide the reader with an appreciation of the nuances inherent in seemingly straightforward factual information, such as audience or size. Each service is then grouped into one of five functional clusters: • open access e-print archives and servers; • cross-archive search services and aggregators; • from digital collections to digital library environments; • from peer-reviewed referratories to portal services; • specialized search engines

Quand le renard raconte ses histoires au monde. La naissance du portail du patrimoine oral, catalogue collectif d'archives sonores et audiovisuelles

Quatre ans après la ratification par la France de la convention de l'Unesco pour la sauvegarde du patrimoine culturel immatériel, ce premier bilan s'appuie sur des études théoriques et des exemples pratiques pour mieux comprendre cette nouvelle notion du patrimoine. L'ouvrage rassemble des contributions de chercheurs, de responsables des politiques culturelles, de personnalités de l'Unesco. Il s'appuie sur des études théoriques autant que sur des exemples pratiques, pour une compréhension en profondeur de ce nouveau domaine du patrimoine. L'article présente le projet collectif du portail du patrimoine oral : http://www.portaildupatrimoineoral.or

ENRICHING CRITICAL THINKING AND LANGUAGE LEARNING WITH EDUCATIONAL DIGITAL LIBRARIES

As the amount of information available in online digital libraries increases exponentially, questions arise concerning the most productive way to use that information to advance learning. Applying the earlier information seeking theories advocated by Kelly (1963), Taylor (1968), and Belkin (1980) to the digital libraries experience, Carol Kuhlthau created the inquiry-based information search process (ISP) model. This ISP model describes thoughts, actions and feelings in six stages of inquiry: initiation, selection, exploration, formulation, collection, and presentation. This study investigated the value of an organized educational digital library in supporting and improving English Foreign Language (EFL) student's critical thinking skills. The study also considered if critical thinking skills and English language skills can be improved simultaneously in the appropriate learning environment. A quasi-experimental pretest/posttest design was utilized. Participants were 98 Taiwanese freshmen majoring in Applied English. Two groups were compared in their ability to cultivate critical thinking. One approach used traditional open access to information plus training in critical thinking. The other used a structured approach to accessing and organizing information from an online digital library as well as training in critical thinking. A One-Way ANCOVA and an Independent-Samples t-test were used to examine the two groups on their 1) critical thinking skills, 2) English reading comprehension, and 3) attitudes in EFL classrooms. Bivariate correlation was employed to evaluate the relation between critical thinking and English reading comprehension. Results indicated that the experimental digital library group (M=11.69) significantly outperformed the traditional group (M=10.61) in critical thinking; F (1, 95) = 4.10, p<.05. The digital library group (M=11.69) also outperformed the traditional group (M=10.23) on English reading comprehension; F (1, 95) =14.72, p<.05. There was a positive relationship between critical thinking and English reading comprehension (r=.212), p<.05. Also, students in the digital library group (M=38.57), had better learning attitudes toward the intervention training program than did the control group (M=35.59); t (96) =2.48, p<.05. Students who used structured search strategies with digital libraries had higher critical thinking performance and more positive attitudes toward their learning experience. Educators should adopt training strategies that engage learners in every stage of inquiry process, from identifying a topic and selecting what to investigate, to formulating a focused perspective and presenting their final product. Further studies are needed to determine if the benefits of structured search strategies with digital libraries extends to other settings, cultures and grade levels. Collecting and analyzing examples of student projects may provide additional insights into the development of critical thinking skills

Integrating Technology, Curriculum, and Online Resources: A Multilevel Model Study of Impacts on Science Teachers and Students

This scale-up study investigated the impact of a teacher technology tool (Curriculum Customization Service, CCS), curriculum, and online resources on earth science teachers’ attitudes, beliefs, and practices and on students’ achievement and engagement with science learning. Participants included 73 teachers and over 2,000 ninth-grade students within five public school districts in the western U.S. To assess the impact on teachers, changes between pre- and postsurveys were examined. Results suggest that the CCS tool appeared to significantly increase both teachers’ awareness of other earth science teachers’ practices and teachers’ frequency of using interactive resources in their lesson planning and classroom teaching. A standard multiple regression model was developed. In addition to “District,” “Training condition”(whether or not teachers received CCS training) appeared to predict teachers’ attitudes, beliefs, and practices. Teachers who received CCS training tended to have lower postsurvey scores than their peers who had no CCS training. Overall, usage of the CCS tool tended to be low, and there were differences among school districts. To assess the impact on students, changes were examined between pre- and postsurveys of (1) knowledge assessment and (2) students’ engagement with science learning. Students showed pre- to postsurvey improvements in knowledge assessment, with small to medium effect sizes. A nesting effect (students clustered within teachers) in the Earth’s Dynamic Geosphere (EDG) knowledge assessment was identified and addressed by fitting a two-level hierarchical linear model (HLM). In addition, significant school district differences existed for student post-knowledge assessment scores. On the student engagement questionnaire, students tended to be neutral or to slightly disagree that science learning was important in terms of using science in daily life, stimulating their thinking, discovering science concepts, and satisfying their own curiosity. Students did not appear to change their self-reported engagement level after the intervention. Additionally, three multiple regression models were developed. Factors from the district, teacher, and student levels were identified to predict student post-knowledge assessments and their engagement with science learning. The results provide information to both the research community and practitioners

SVMAUD: Using textual information to predict the audience level of written works using support vector machines

Information retrieval systems should seek to match resources with the reading ability of the individual user; similarly, an author must choose vocabulary and sentence structures appropriate for his or her audience. Traditional readability formulas, including the popular Flesch-Kincaid Reading Age and the Dale-Chall Reading Ease Score, rely on numerical representations of text characteristics, including syllable counts and sentence lengths, to suggest audience level of resources. However, the author’s chosen vocabulary, sentence structure, and even the page formatting can alter the predicted audience level by several levels, especially in the case of digital library resources. For these reasons, the performance of readability formulas when predicting the audience level of digital library resources is very low. Rather than relying on these inputs, machine learning methods, including cosine, Naïve Bayes, and Support Vector Machines (SVM), can suggest the grade level of an essay based on the vocabulary chosen by the author. The audience level prediction and essay grading problems share the same inputs, expert-labeled documents, and outputs, a numerical score representing quality or audience level. After a human expert labels a representative sample of resources with audience level, the proposed SVM-based audience level prediction program, SVMAUD, constructs a vocabulary for each audience level; then, the text in an unlabeled resource is compared with this predefined vocabulary to suggest the most appropriate audience level. Two readability formulas and four machine learning programs are evaluated with respect to predicting human-expert entered audience levels based on the text contained in an unlabeled resource. In a collection containing 10,238 expert-labeled HTML-based digital library resources, the Flesch-Kincaid Reading Age and the Dale-Chall Reading Ease Score predict the specific audience level with F-measures of 0.10 and 0.05, respectively. Conversely, cosine, Naïve Bayes, the Collins-Thompson and Callan model, and SVMAUD improve these F-measures to 0.57, 0.61, 0.68, and 0.78, respectively. When a term’s weight is adjusted based on the HTML tag in which it occurs, the specific audience level prediction performance of cosine, Naïve Bayes, the Collins-Thompson and Callan method, and SVMAUD improves to 0.68, 0.70, 0.75, and 0.84, respectively. When title, keyword, and abstract metadata is used for training, cosine, Naïve Bayes, the Collins-Thompson and Callan model, and SVMAUD specific audience level prediction F-measures are found to be 0.61, 0.68, 0.75, and 0.86, respectively. When cosine, Naïve Bayes, the Collins-Thompson and Callan method, and SVMAUD are trained and tested using resources from a single subject category, the specific audience level prediction F- measure performance improves to 0.63, 0.70, 0.77, and 0.87, respectively. SVMAUD experiences the highest audience level prediction performance among all methods under evaluation in this study. After SVMAUD is properly trained, it can be used to predict the audience level of any written work

Full Report: Use and Users of Digital Resources: A Focus on Undergraduate Education in the Humanities and Social Sciences

A "build it and they will come" approach to many university digitization initiatives has precluded systematic investigations of the demand for these resources. Those who fund and develop digital resources have identified the general lack of knowledge about the level and quality of their use in educational settings as pressing concerns. This full report describes our research and the complete results of a large 2-year study. The purpose of our research was (1) to map the universe of digital resources available to a subset of undergraduate educators in the humanities and social sciences, and (2) to investigate how and if available digital resources are actually being used in undergraduate teaching environments. We employed multiple methods, including surveys and focus groups. Our definition of digital resources was intentionally broad and included rich media objects (e.g., maps, video, images, etc.) as well as text

Supporting the Discoverability of Open Educational Resources: on the Scent of a Hidden Treasury

Open Educational Resources (OERs), now available in large numbers, have a considerable potential to improve many aspects of society, yet one of the factors limiting this positive impact is the difficulty to discover them. This thesis investigates and proposes strategies to better support educators in discovering OERs. The literature suggests that the effectiveness of existing search systems, including for OER discovery, could be improved by supporting users, such as teachers, in carrying out more exploratory search activities closer to their existing methods of working. Hence, a preliminary taxonomy of OER-related search tasks was produced, based on an analysis of the literature, interpreted through Information Foraging Theory. This taxonomy was empirically evaluated to preliminarily identify a set of search tasks that involve finding other OERs similar to one that has already been identified, a process that is generally referred to as Query By Example (QBE). Following the Design Science Research methodology, three prototypes to support as well as to refine those tasks were iteratively designed, implemented, and evaluated involving an increasing number of educators in usability oriented studies. The resulting high-level and domain-oriented blended search/recommendation strategy transparently replicates Google searches in specialized networks, and identifies similar resources with a QBE strategy. It makes use of a domain-oriented similarity metric based on shared alignments to educational standards, and clusters results in expandable classes of comparable degrees of similarity. The summative evaluation shows that educators do appreciate this strategy because it is exploratory and – balancing similarity and diversity – it supports their high-level tasks, such as lesson planning and personalization of education. Finally, potential barriers and opportunities for the uptake of OER discovery tools were investigated in a structured interview study with experts from the OER field. Identified issues included how to work across multiple OER portals, variability in the use of metadata and how to align with the working practices of teachers. The findings of the thesis can be used to inform the research and development of methods and tools for OER discovery as well as their deployment to serve the needs of educators