Evaluation campaigns and TRECVid

The TREC Video Retrieval Evaluation (TRECVid) is an international benchmarking activity to encourage research in video information retrieval by providing a large test collection, uniform scoring procedures, and a forum for organizations interested in comparing their results. TRECVid completed its fifth annual cycle at the end of 2005 and in 2006 TRECVid will involve almost 70 research organizations, universities and other consortia. Throughout its existence, TRECVid has benchmarked both interactive and automatic/manual searching for shots from within a video corpus, automatic detection of a variety of semantic and low-level video features, shot boundary detection and the detection of story boundaries in broadcast TV news. This paper will give an introduction to information retrieval (IR) evaluation from both a user and a system perspective, highlighting that system evaluation is by far the most prevalent type of evaluation carried out. We also include a summary of TRECVid as an example of a system evaluation benchmarking campaign and this allows us to discuss whether such campaigns are a good thing or a bad thing. There are arguments for and against these campaigns and we present some of them in the paper concluding that on balance they have had a very positive impact on research progress

Voices for Change: A Taxonomy of Public Communications Campaigns and Their Evaluation Challenges

Makes the case that communications campaigns cover a broad range of different types and characteristics, and can be differentiated along the axes of purpose, scope, and maturity. Examines what communications campaigns that fall on different areas of these three axes look like, and how where they fall may affect the evaluation approach used and lead to distinct evaluation challenges and needs

Mobilizing Public Will For Social Change

Examines the theory and strategies of "public will" campaigns and offers tangible criteria for their evaluation. It provides a rich inventory of strategies for use in mobilizing the public will through an integration of models of agenda building, social problem construction, issues management, social movements, media advocacy, and social capital. In addition, the paper provides cases and examples of public will campaigns directed at various social problems, along with criteria for evaluating these campaigns at various stages of a social problem's life cycle

Survey of prevention and information campaigns on psychoactive drugs and driving

Objectives: One of the objectives of the DRUID project is to develop guidelines for spreading information regarding driving under the influence of drugs and medicines, and this aiming at different target groups, including young drivers, patients, physicians, pharmacists,… We reviewed the state-of-the-art of existing information campaigns regarding psychoactive drugs and driving, as well as the documented effectiveness of those campaigns. Methods: In total 55 institutes were contacted, of which 17 (31%) responded and 12 (22%) were able to give information. Information was also gathered through the Internet by means of websites of relevant organizations, Google and YouTube. Results: A total of 75 campaigns were found, from 13 different countries. Twenty-four were launched to inform the general public on the risks associated with driving under the influence of drugs, medicines and/or alcohol. Twenty-nine campaigns were launched to inform young people and 16 campaigns to inform physicians and/or pharmacists. Nine campaigns were designed for teachers, seven campaigns for patients and five campaigns for other target populations. Some campaigns were designed for more than one target group. Information on impact evaluation was available only for seven campaigns. All evaluations showed a positive impact of the campaigns. Conclusions: The majority of the retrieved campaigns concerning driving under the influence of drugs were aiming at young people. The type of medium that is used the most is brochures. Most campaigns are organized by governmental organizations and road safety organizations. As only a few evaluations were found, and these campaigns and their evaluations were performed in many different ways, it is not possible to draw conclusions concerning the association between the design of the campaigns and their effectiveness

Sweatshop Watch Newsletter

Winter 2005 newsletter of Sweatshop Watch covering the evaluation of California’s landmark anti-sweatshop law, the major campaigns of 2005, and the “Shop with a Conscience” initiative

Evaluating Communication Campaigns

Summarizes presentations from a September 2007 conference on evaluating communication campaigns. Discusses the mechanism of effecting change through communication; the principles of advocacy evaluation; the design, methods, and tools; and lessons learned

Evaluating Direct Marketing Campaigns: recent findings and future research topics

This paper contains a survey of the recent literature on the evaluation of direct marketing campaigns. We give an outline of the various stages included in such a campaign. Next, we review the statistical methods most frequently used and we review the general findings from using these methods.direct marketing;target selection;evaluation;quantitative models

Towards more Challenging Problems for Ontology Matching Tools

We motivate the need for challenging problems in the evaluation of ontology matching tools. To address this need, we propose mapping sets between well-known biomedical ontologies that are based on the UMLS Metathesaurus. These mappings could be used as a basis for a new track in future OAEI campaigns (http://oaei.ontologymatching.org/).
&#xa

Assert Yourself! evaluating the performance of an HIV prevention intervention

Duration: November 1996 - June 2012 CHAPS is a collaborative programme of HIV health promotion for gay men and bisexual men living in England. It is co-ordinated by Terrence Higgins Trust on behalf of a range of project partners, including Sigma Research. The CHAPS programme includes a national HIV prevention mass media campaign each year, which usually include adverts, leaflets, magazines and other small media. Data from the Gay Men's Sex Survey is used to inform the targets and aims of these interventions and their development is supported by pre-testing with the target audience via focus groups. After the launch of the interventions, Sigma generates performance data about their performance. This includes annual coverage surveys where gay men and bisexual men are asked if they recognise the adverts, magazines and leaflets. This data is used to look in greater detail at the targeting of the campaigns. Using data from these surveys, we make recommendations on the most efficient media spend for advertising. In addition, we undertake end-user focus groups in which gay men and bisexual men are asked about their opinions, reactions and responses to the campaigns. This work follows on from the pre-testing groups, and looks at the relevance of the campaigns in situ. We also evaluate the sector development or facilitation interventions, undertaken within the CHAPS partnership including training events, Symposiums and the annual CHAPS conference. Three CHAPS summary final evaluation reports are available alongside many of the interim reports of CHAPS evaluation activities on which they are based. Click here for more details. CHAPS evaluation is funded by Terrence Higgins Trust as part of CHAPS, a national HIV prevention initiative funded by the Department of Health