A Methodology for Simulated Experiments in Interactive Search

Interactive information retrieval has received much attention in recent years, e.g. [7]. Furthermore, increased activity in developing interactive features in search systems used across existing popular Web search engines suggests that interactive systems are being recognised as a promising next step in assisting information search. One of the most challenging problems with interactive systems however remains evaluation. We describe the general specifications of a methodology for conducting controlled and reproducible experiments in the context of interactive search. It was developed in the AutoAdapt project1 focusing on search in intranets, but the methodology is more generic than that and can be applied to interactive Web search as well. The goal of this methodology is to evaluate the ability of different algorithms to produce domain models that provide accurate suggestions for query modifications. The AutoAdapt project investigates the application of automatically constructed adaptive domain models for providing suggestions for query modifications to the users of an intranet search engine. This goes beyond static models such as the one employed to guide users who search the Web site of the University of Essex which is based on a domain model that has been built in advance using the documents’ markup structure

A Personalized System for Conversational Recommendations

Searching for and making decisions about information is becoming increasingly difficult as the amount of information and number of choices increases. Recommendation systems help users find items of interest of a particular type, such as movies or restaurants, but are still somewhat awkward to use. Our solution is to take advantage of the complementary strengths of personalized recommendation systems and dialogue systems, creating personalized aides. We present a system -- the Adaptive Place Advisor -- that treats item selection as an interactive, conversational process, with the program inquiring about item attributes and the user responding. Individual, long-term user preferences are unobtrusively obtained in the course of normal recommendation dialogues and used to direct future conversations with the same user. We present a novel user model that influences both item search and the questions asked during a conversation. We demonstrate the effectiveness of our system in significantly reducing the time and number of interactions required to find a satisfactory item, as compared to a control group of users interacting with a non-adaptive version of the system

Teacher Evaluator Training & Certification: Lessons Learned From the Measures of Effective Teaching Project

Makes recommendations for the design and implementation of programs to train and certify principals in conducting teacher evaluations, including content, format, and length of training, scoring practice, and criteria for certification tests

Customer perception of switch-feel in luxury sports utility vehicles

Successful new product introduction requires that product characteristics relate to the customer on functional, emotional, aesthetic and cultural levels. As a part of research into automotive human machine interfaces (HMI), this paper describes holistic customer research carried out to investigate how the haptics of switches in luxury sports utility vehicles (SUVs) are perceived by customers. The application of these techniques, including an initial proposal for objective specifications, is addressed within the broader new product introduction context, and benefits described. One-hundred and one customers of SUVs assessed the feel of automotive push switches, completing the tasks both in, and out of vehicles to investigate the effect of context. Using the semantic differential technique, hedonic testing, and content analysis of customers’ verbatim comments, a holistic picture has been built up of what influences the haptic experience. It was found that customers were able to partially discriminate differences in switch-feel, alongside considerations of visual appearance, image, and usability. Three factors named ‘Affective’, ‘Robustness and Precision’, and ‘Silkiness’ explained 61% of the variance in a principle components analysis. Correlations of the factors with acceptance scores were 0.505, 0.371, and 0.168, respectively

Optimising ITS behaviour with Bayesian networks and decision theory

We propose and demonstrate a methodology for building tractable normative intelligent tutoring systems (ITSs). A normative ITS uses a Bayesian network for long-term student modelling and decision theory to select the next tutorial action. Because normative theories are a general framework for rational behaviour, they can be used to both define and apply learning theories in a rational, and therefore optimal, way. This contrasts to the more traditional approach of using an ad-hoc scheme to implement the learning theory. A key step of the methodology is the induction and the continual adaptation of the Bayesian network student model from student performance data, a step that is distinct from other recent Bayesian net approaches in which the network structure and probabilities are either chosen beforehand by an expert, or by efficiency considerations. The methodology is demonstrated by a description and evaluation of CAPIT, a normative constraint-based tutor for English capitalisation and punctuation. Our evaluation results show that a class using the full normative version of CAPIT learned the domain rules at a faster rate than the class that used a non-normative version of the same system

Towards an Expert Network in Open Standards and Open Source Software: Research, Expertise and Synergy for Open and Libres Standards and Software (RESOLL) - Version 2.0

CIRANO and its partners are proposing the creation of an Expert Network in Open Standards and Open Source Software (Research, Expertise and Synergy for Open and Libres Standards and Software-RESOLL), which would be a partnership between information technology research centres, government and private user organizations, and businesses working in the field. The network will conduct studies and pilot projects that integrate computer solutions based on open standards and open source software, mainly in e-government fields such as health, education, and scientific research, as well as municipal and quasi-public services and business processes for SMEs. The knowledge, expertise and tools thus developed will be disseminated in a number of ways in order to Quebec and Canadian expertise in the field. RESOLL will also have economic and strategic benefits in that it will put the new economic model to the test in terms of open standards and open source software as well as the reuse of software components by organizations. Background The development of on-line government services and e-business is a priority for governments and businesses of all sizes. It requires considerable spending and significant strategic and organizational changes. Of the many information technology solutions available, the use of open standards and open source software is often brought up by those in the know. Although the Internet and many world-renowned software programs were developed largely from open standards and open source software, there is still a need to study, and above all prove the advantages of this approach for public and quasi-public organizations as well as small to medium-sized businesses. It is essential to identify the needs of these organizations, document best practices, experiment with open source software solutions, evaluate the performance of the software and share the knowledge and know-how of Quebec and Canadian research centres and businesses. RESOLL Goals he main goal of the Expert Network on Open Standards and Open Source Software (RESOLL) is to give people an understanding of the benefits of open standards and open source software and suggest an intelligent and advantageous use of them for public and quasi-public organizations and SMEs. More specifically, the goals are as follows: Document and share government and industry policies, strategies, and practices with respect to the use and development of adaptive software and open source software, defining open standards, open source software, adaptive software, and proprietary software; Adapt these practices and share the different methods with partners and the IT management and development communities in government and business; Establish innovative prototypes and pilot projects in order to test and demonstrate the advantages and features of this approach; Develop the expertise of Quebec and Canadian organizations in the field and create synergy between them and their users; P ublish and share the findings of the work, contribute to the enrichment of a collective software asset base available to public and quasi-public organizations and SMEs while explaining the legal issues involved in the various types of licences and electronic services. Process ESOLL is a multilateral partnership founded on the excellence of partners in their respective field. The RESOLL process will be based on the needs of its partners and users. Once these needs have been identified, research will be conducted to identify available solutions, adapt them through an integration process and alpha test them. This would be followed by a pilot project as required by the organizations and businesses. The pilot project will be implemented and evaluated in order to learn from it and ensure that necessary adjustments are made. Solutions thus obtained will be implemented as electronic services either by the client organization’s IT department or by a business partner. It is up to each organization to select their service provider. RESOLL will encourage the transfer of developed tools and services to partners for complete autonomy. Each project will have its own budget, funded by client partners. RESOLL will use part of its operating budget to start projects and develop a start-up asset base for its activities. Expectations and Deliverables The expectations of RESOLL partners and the team can be expressed by the achievement of their goals. RESOLL’s actions will quickly lead to concrete results. The deliverables will be: Policy and position papers to help partners make clear and informed decisions; Needs analyses and suggested solutions; Software solutions based on open standards and open source software integrated into experimental electronic services; Pilot project experiments that combine strategies, plans, software solutions, project support, evaluation and recommendations; Studies and interpretation documents for different types of licences and software; Collaborative Web site for sharing documents and open source software developed in the context of RESOLL projects or available on the Internet, with comments and explanations; Information and knowledge sharing activities for RESOLL and its partners (conferences, workshops, training, etc.). Partners RESOLL is a multilateral partnership. The partners that have been asked to become involved are: CIRANO, CRIM, RISQ, the governments of Quebec and Canada, Industry Canada, university researchers, Canadian and Quebec software and information technology companies, and not-for-profit user organizations from the software and information technology fields. Budget ESOLL’s master infrastructure budget will make it possible to establish a small coordination team involving part-time resources seconded from their parent organizations. We plan to obtain general financing from government and the businesses involved. The individual projects will provide their own financing. Other Benefits RESOLL will contribute to Quebec’s and Canada’s world leadership by sharing the results of its work. It will contribute to the eventual creation of resources that will enable partner companies to commercialize services based on open source software.Open standard, free software, open software, FOSS, e-government, business process, small and medium enterprises,