TREC video retrieval evaluation: a case study and status report

The TREC Video Retrieval Evaluation is a multiyear, international effort, funded by the US Advanced Research and Development Agency (ARDA) and the National Institute of Standards and Technology (NIST) to promote progress in content-based retrieval from digital video via open, metrics-based evaluation. Now beginning its fourth year, it aims over time to develop both a better understanding of how systems can effectively accomplish such retrieval and how one can reliably benchmark their performance. This paper can be seen as a case study in the development of video retrieval systems and their evaluation as well as a report on their status to-date. After an introduction to the evolution of the evaluation over the past three years, the paper reports on the most recent evaluation TRECVID 2003: the evaluation framework — the 4 tasks (shot boundary determination, high-level feature extraction, story segmentation and typing, search), 133 hours of US television news data, and measures —, the results, and the approaches taken by the 24 participating groups

TRECVID: evaluating the effectiveness of information retrieval tasks on digital video

TRECVID is an annual exercise which encourages research in information retrieval from digital video by providing a large video test collection, uniform scoring procedures, and a forum for organizations interested in comparing their results. TRECVID benchmarking covers both interactive and manual searching by end users, as well as the benchmarking of some supporting technologies including shot boundary detection, extraction of some semantic features, and the automatic segmentation of TV news broadcasts into non-overlapping news stories. TRECVID has a broad range of over 40 participating groups from across the world and as it is now (2004) in its 4th annual cycle it is opportune to stand back and look at the lessons we have learned from the cumulative activity. In this paper we shall present a brief and high-level overview of the TRECVID activity covering the data, the benchmarked tasks, the overall results obtained by groups to date and an overview of the approaches taken by selective groups in some tasks. While progress from one year to the next cannot be measured directly because of the changing nature of the video data we have been using, we shall present a summary of the lessons we have learned from TRECVID and include some pointers on what we feel are the most important of these lessons

Evaluation campaigns and TRECVid

The TREC Video Retrieval Evaluation (TRECVid) is an international benchmarking activity to encourage research in video information retrieval by providing a large test collection, uniform scoring procedures, and a forum for organizations interested in comparing their results. TRECVid completed its fifth annual cycle at the end of 2005 and in 2006 TRECVid will involve almost 70 research organizations, universities and other consortia. Throughout its existence, TRECVid has benchmarked both interactive and automatic/manual searching for shots from within a video corpus, automatic detection of a variety of semantic and low-level video features, shot boundary detection and the detection of story boundaries in broadcast TV news. This paper will give an introduction to information retrieval (IR) evaluation from both a user and a system perspective, highlighting that system evaluation is by far the most prevalent type of evaluation carried out. We also include a summary of TRECVid as an example of a system evaluation benchmarking campaign and this allows us to discuss whether such campaigns are a good thing or a bad thing. There are arguments for and against these campaigns and we present some of them in the paper concluding that on balance they have had a very positive impact on research progress

The GMT/MATLAB Toolbox

The GMT/MATLAB toolbox is a basic interface between MATLAB (R) (or Octave) and GMT, the Generic Mapping Tools, which allows MATLAB users full access to all GMT modules. Data may be passed between the two programs using intermediate MATLAB structures that organize the metadata needed; these are produced when GMT modules are run. In addition, standard MATLAB matrix data can be used directly as input to GMT modules. The toolbox improves interoperability between two widely used tools in the geosciences and extends the capability of both tools: GMT gains access to the powerful computational capabilities of MATLAB while the latter gains the ability to access specialized gridding algorithms and can produce publication-quality PostScript-based illustrations. The toolbox is available on all platforms and may be downloaded from the GMT website.US National Science Foundation [OCE-1029874

Bathymetry of the Pacific plate and its implications for thermal evolution of lithosphere and mantle dynamics

A long-standing question in geodynamics is the cause of deviations of ocean depth or seafloor topography from the prediction of a cooling half-space model (HSC). Are the deviations caused entirely by mantle plumes or lithospheric reheating associated with sublithospheric small-scale convection or some other mechanisms? In this study we analyzed the age and geographical dependences of ocean depth for the Pacific plate, and we removed the effects of sediments, seamounts, and large igneous provinces (LIPs), using recently available data sets of high-resolution bathymetry, sediments, seamounts, and LIPs. We found that the removal of seamounts and LIPs results in nearly uniform standard deviations in ocean depth of ∼300 m for all ages. The ocean depth for the Pacific plate with seamounts, LIPs, the Hawaiian swell, and South Pacific super-swell excluded can be fit well with a HSC model till ∼80–85 Ma and a plate model for older seafloor, particularly, with the HSC-Plate depth-age relation recently developed by Hillier and Watts (2005) with an entirely different approach for the North Pacific Ocean. A similar ocean depth-age relation is also observed for the northern region of our study area with no major known mantle plumes. Residual topography with respect to Hillier and Watts' HSC-Plate model shows two distinct topographic highs: the Hawaiian swell and South Pacific super-swell. However, in this residual topography map, the Darwin Rise does not display anomalously high topography except the area with seamounts and LIPs. We also found that the topography estimated from the seismic model of the Pacific lithosphere of Ritzwoller et al. (2004) generally agrees with the observed topography, including the reduced topography at relatively old seafloor. Our analyses show that while mantle plumes may be important in producing the Hawaiian swell and South Pacific super-swell, they cannot be the only cause for the topographic deviations. Other mechanisms, particularly lithospheric reheating associated with “trapped” heat below old lithosphere (Huang and Zhong, 2005), play an essential role in causing the deviations in topography from the HSC model prediction

TRECVID 2008 - goals, tasks, data, evaluation mechanisms and metrics

The TREC Video Retrieval Evaluation (TRECVID) 2008 is a TREC-style video analysis and retrieval evaluation, the goal of which remains to promote progress in content-based exploitation of digital video via open, metrics-based evaluation. Over the last 7 years this effort has yielded a better understanding of how systems can effectively accomplish such processing and how one can reliably benchmark their performance. In 2008, 77 teams (see Table 1) from various research organizations --- 24 from Asia, 39 from Europe, 13 from North America, and 1 from Australia --- participated in one or more of five tasks: high-level feature extraction, search (fully automatic, manually assisted, or interactive), pre-production video (rushes) summarization, copy detection, or surveillance event detection. The copy detection and surveillance event detection tasks are being run for the first time in TRECVID. This paper presents an overview of TRECVid in 2008