Analysis of Visualisation and Interaction Tools Authors

This document provides an in-depth analysis of visualization and interaction tools employed in the context of Virtual Museum. This analysis is required to identify and design the tools and the different components that will be part of the Common Implementation Framework (CIF). The CIF will be the base of the web-based services and tools to support the development of Virtual Museums with particular attention to online Virtual Museum.The main goal is to provide to the stakeholders and developers an useful platform to support and help them in the development of their projects, despite the nature of the project itself. The design of the Common Implementation Framework (CIF) is based on an analysis of the typical workflow ofthe V-MUST partners and their perceived limitations of current technologies. This document is based also on the results of the V-MUST technical questionnaire (presented in the Deliverable 4.1). Based on these two source of information, we have selected some important tools (mainly visualization tools) and services and we elaborate some first guidelines and ideas for the design and development of the CIF, that shall provide a technological foundation for the V-MUST Platform, together with the V-MUST repository/repositories and the additional services defined in the WP4. Two state of the art reports, one about user interface design and another one about visualization technologies have been also provided in this document

Semantic Web Technologies Meet BIM for Accessing and Understanding Cultural Heritage

Within the EU funded project INCEPTION – Inclusive Cultural Heritage in Europe through 3D semantic modelling, the key-targeted achievement is the development of a specific cloud based platform, in order to accomplish the main objectives of accessing, understanding and strengthening European Cultural Heritage by means of enriched 3D models. The whole INCEPTION project is based on the close connection between state-of-the-art architectural modeling technologies (BIM, Building Information Modeling) and the latest cutting-edge web technologies. The platform is grounded on semantic web technologies and makes extensive use of WebGL and RESTful APIs, in order to enrich heritage 3D models by using Semantic Web standards. The INCEPTION platform will be a space for interchange of information and for the dialogue among professionals, students, scholars, curators, non-expert users, etc. Furthermore, the Semantic Web structure interlinks the platform with external Cultural Heritage available linked data and makes it gradually enhanced by specific flexible data structures provided as project specific ontologies. The paper will describe solutions based on the match between BIM, Cloud and Semantic Web

Estudio de los repositorios y plataformas de patrimonio digital en 3D

[EN] Despite the increasing number of three-dimensional (3D) model portals and online repositories catering for digital heritage scholars, students and interested members of the general public, there are very few recent academic publications that offer a critical analysis when reviewing the relative potential of these portals and online repositories. Solid reviews of the features and functions they offer are insufficient; there is also a lack of explanations as to how these assets and their related functionality can further the digital heritage (and virtual heritage) field, and help in the preservation, maintenance, and promotion of real-world 3D heritage sites and assets. What features do they offer? How could their feature list better cater for the needs of the GLAM (galleries, libraries, archives and museums) sector? This article’s priority is to examine the useful features of 8 institutional and 11 commercial repositories designed specifically to host 3D digital models. The available features of their associated 3D viewers, where applicable, are also analysed, connecting recommendations for future-proofing with the need to address current gaps and weaknesses in the scholarly field of 3D digital heritage. Many projects do not address the requirements stipulated by charters, such as access, reusability, and preservation. The lack of preservation strategies and examples highlights the oxymoronic nature of virtual heritage (oxymoronic in the sense that the virtual heritage projects themselves are seldom preserved). To study these concerns, six criteria for gauging the usefulness of the 3D repositories to host 3D digital models and related digital assets are suggested. The authors also provide 13 features that would be useful additions for their 3D viewers.[ES] A pesar del creciente número de portales de modelos tridimensionales (3D) y repositorios en línea que atienden a los estudiosos del patrimonio digital, a los estudiantes y al público en general, hay muy pocas publicaciones académicas recientes que analizan de forma crítica el potencial relativo de esos portales y repositorios en línea. Tampoco hay suficientes revisiones críticas de las características y funciones que ofrecen, ni muchas explicaciones sobre la forma en que estos activos y su funcionalidad pueden impulsar en el campo del patrimonio digital (y el patrimonio virtual), y ayudar a preservar, mantener y promocionar los sitios y activos del patrimonio 3D del mundo real. ¿Qué características ofrecen? ¿Cómo podría su lista de características satisfacer mejor las necesidades del sector GLAM (galerías, bibliotecas, archivos y museos)? La prioridad de este artículo es examinar las características útiles de 8 depósitos institucionales y 11 comerciales diseñados específicamente para albergar modelos digitales en 3D. También son examinadas las características disponibles de su visores 3D asociados, cuando sea aplicable, y ello conecta con lo recomendado sobre las necesidades futuras y mejoradas para abordar las lagunas y debilidades en el campo académico del patrimonio digital 3D. Muchos proyectos no estudian los requisitos estipulados en las cartas, como son los factores de acceso, la reutilización y la preservación. La escasez de estrategias y ejemplos de preservación pone de relieve el carácter oximorónico del patrimonio virtual (oximorónico en el sentido de que los propios proyectos de patrimonio virtual se preservan con muy poca frecuencia). Para hacer frente a estas preocupaciones, se sugieren seis criterios para calibrar la utilidad de los repositorios 3D para albergar modelos digitales 3D y activos digitales relacionados. Los autores también proporcionan 13 características adicionales que serían útiles en los visores 3D.Champion, E.; Rahaman, H. (2020). Survey of 3D digital heritage repositories and platforms. Virtual Archaeology Review. 11(23):1-15. https://doi.org/10.4995/var.2020.13226OJS1151123Aalbersberg, I. J., Cos Alvarez, P., Jomier, J., Marion, C., & Zudilova-Seinstra, E. (2014). Bringing 3D visualization into the online research article. Information Services & Use, 34(1-2), 27-37. https://doi.org/10.3233/ISU-140721Addison, A. C. (2000). Emerging trends in virtual heritage. IEEE Multimedia, 7(2), 22-25. https://doi.org/10.1109/93.848421Alliez, P., Bergerot, L., Bernard, J.-F., Boust, C., Bruseker, G., Carboni, N., Chayani, M., Dellepiane, M., Dell'unto, N., & Dutailly, B. (2017). Digital 3D objects in art and humanities: Challenges of creation, interoperability and preservation. In White paper: A result of the PARTHENOS Workshop held in Bordeaux at Maison des Sciences de l'Homme d'Aquitaine and at Archeovision Lab. (France) (pp. 71). France.Beacham, R., Hugh, D., & Niccolucci, F. (2009). The London Charter. In For computer-based visualization of cultural heritage (Vol. Draft 2.1).Bernard, Y., Barreau, J.-B., Bizien-Jaglin, C., Quesnel, L., Langouët, L., & Daire, M.-Y. (2017). 3D model as a dynamic compilation of knowledge: Interim results on the city of Alet. Virtual Archaeology Review, 8(16). https://doi.org/10.4995/var.2017.5862Boutsi, A.-M., Ioannidis, C., & Soile, S. (2019). An integrated approach to 3D web visualization of cultural heritage heterogeneous datasets. Remote Sensing, 11(21). https://doi.org/10.3390/rs11212508Calin, M., Damian, G., Popescu, T., Manea, R., Erghelegiu, B., & Salagean, T. (2015). 3D modeling for digital preservation of Romanian heritage monuments. Agriculture and Agricultural Science Procedia, 6, 421-428. https://doi.org/10.1016/j.aaspro.2015.08.111Champion, E. (2018). The role of 3D models in virtual heritage intrastructures. In A. Benardou, E. Champion, C. Dallas, & L. M. Hughes (Eds.), Cultural Heritage Infrastructures in Digital Humanities (pp. 172). Abingdon, Oxon New York: NY Routledge. https://doi.org/10.4324/9781315575278Champion, E. (2019). From historical models to virtual heritage simulations. In P. Kuroczyński, M. Pfarr-Harfst, & S. Münster (Eds.), Der Modelle Tugend 2.0 Digitale 3d-Rekonstruktion Als Virtueller Raum Der Architekturhistorischen Forschung Computing in Art and Architecture (Vol. 2, pp. 338-351). Heidelberg, Germany: arthistoricum.net. https://doi.org/10.11588/arthistoricum.515Champion, E., & Rahaman, H. (2019). 3D digital heritage models as sustainable scholarly resources. Sustainability, 11(8), 1-8. https://doi.org/10.3390/su11082425Clarke, M. (2015). The digital dilemma: preservation and the digital archaeological record. Advances in Archaeological Practice, 3(4), 313-330. https://doi.org/10.7183/2326-3768.3.4.313Cots, I., Vilà, J., Diloli, J., Ferré, R., & Bricio, L. (2018). La arqueología virtual: de la excavación arqueológica a la gestión y socialización del patrimonio. Les cases de la Catedral (Tortosa) y el yacimiento protohistórico de La Cella(Salou), Tarragona. Virtual Archaeology Review, 9(19). https://doi.org/10.4995/var.2018.9754Di Giuseppantonio Di Franco, P. , Galeazzi, F., & Vassallo, V. (Eds.). (2018). Authenticity and cultural heritage in the age of 3D digital reproductions. Cambridge, UK: McDonald Institute for Archaeological Research. http://doi.org/10.17863/CAM.27029Doyle, J., Viktor, H., & Paquet, E. (2009). Long-term digital preservation: preserving authenticity and usability of 3-D data. International Journal on Digital Libraries, 10(1), 33-47. https://doi.org/10.1007/s00799-009-0051-7Flynn, T. (2019). What happens when you share 3D models online (In 3D)? In J. Grayburn, Z. Lischer-Katz, K. Golubiewski-Davis, & V. Ikeshoji-Orlati (Eds.), 3D/VR in the Academic Library: Emerging Practices and Trends (pp. 73-86). Arlington, USA: Council on Library and Information Resources.Galeazzi, F., Baker, F., Champion, E., Gartski, K., Jeffrey, S., & Kuzminsky, S. (2018). Commentary on 3-D virtual replicas and simulations of the past : "real" or "fake" representations? Current Anthropology, 59(3), 268-286. http://doi.org/10.1086/697489Galeazzi, F., & Franco, P. D. G. D. (2017). Theorising 3D visualisation systems in archaeology: Towards more effective design, evaluations and life cycles. Internet Archaeology(44). http://doi.org/10.11141/ia.44.5Greenop, K., & Barton, J. (2014). Scan, save, and archive: how to protect our digital cultural heritage. The Conversation, 1. https://theconversation.com/scan-save-and-archive-how-to-protect-our-digital-cultural-heritage-22160.Guidazzoli, A., Liguori, M. C., Chiavarini, B., Verri, L., Imboden, S., De Luca, D., & Ponti, F. D. (2017, 31 Oct-4 Nov). From 3D Web to VR historical scenarios: A cross-media digital heritage application for audience development. In 2017 23rd International Conference on Virtual System & Multimedia (VSMM), (pp. 1-8) Dublin, Ireland. https://doi.org/10.1109/VSMM.2017.8346273Huk, T. (2006). Who benefits from learning with 3D models? the case of spatial ability. Journal of Computer Assisted Learning, 22(6), 392-404. https://doi.org/10.1111/j.1365-2729.2006.00180.xIoannides, M., & Quak, E. (Eds.). (2014). 3D research challenges in cultural heritage : A roadmap in digital heritage preservation. NewYork, Dordrecht, London: Springer-Verlag Berlin Heidelberg. https://doi.org/10.1007/978-3-662-44630-0Khronos, G. (2009). OpenGL ES for the web. WebGL Overview. Retrieved 4 March, 2020, from https://www.khronos.org/webgl/Kiourt, C., Koutsoudis, A., Markantonatou, S., & Pavlidis, G. (2016). The 'synthesis' virtual museum. Mediterranean Archaeology and Archaeometry, 16(5), 1-9. http://doi.org/10.5281/zenodo.204961Koller, D., Frischer, B., & Humphreys, G. (2009). Research challenges for digital archives of 3D cultural heritage models. Journal on Computing and Cultural Heritage, 2(3), 1-17. https://doi.org/10.1145/1658346.1658347Koutsabasis, P. (2017). Empirical evaluations of interactive systems in cultural heritage: A review. International Journal of Computational Methods in Heritage Science, 1(1), 100-122. https://doi.org/10.4018/IJCMHS.2017010107Kuroczynski, P. (2017). Virtual research environment for digital 3D reconstructions : Standards, thresholds and prospects. Studies in Digital Heritage, 1(2), 456-476. https://doi.org/10.14434/sdh.v1i2.23330Lloyd, J. (2016). Contextualizing 3D cultural heritage. In M. Ioannides, E. Fink, R. Brumana, P. Patias, A. Doulamis, J. Martins, & M. Wallace (Eds.), Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection (Vol. 1, pp. 859-868). Nicosia, Cyprus: Springer International Publishing. https://doi.org/10.1007/978-3-319-48496-9_69Maiwald, F., Bruschke, J., Lehmann, C., & Niebling, F. (2019). A 4D information system for the exploration of multitemporal images and maps using photogrammetry, web technologies and VR/AR. Virtual Archaeology Review, 10(21). https://doi.org/10.4995/var.2019.11867McHenry, K., & Bajcsy, P. (2008). An overview of 3d data content, file formats and viewers. Retrieved from Urbana, IL: https://www.archives.gov/files/applied-research/ncsa/8-an-overview-of-3d-data-content-file-formats-and-viewers.pdf.Muñoz Morcillo, J., Schaaf, F., Schneider, R. H., & Robertson-von Trotha, C. Y. (2017). Authenticity through VR-based documentation of cultural heritage. A theoretical approach based on conservation and documentation practices. Virtual Archaeology Review, 8(16). https://doi.org/10.4995/var.2017.5932Munster, S. (2018, 26-29 June). Digital 3D modelling in the humanities. In Digital Heritage 2018, (pp. 627-629) Mexico.Münster, S., Pfarr-Harfst, M., Kuroczyński, P., & Ioannides, M. (Eds.). (2016). 3D research challenges in cultural heritage II : How to manage data and knowledge related to interpretative digital 3D reconstructions of cultural heritage. Cham, Switzerland: Springer International Publishing. https://doi.org/10.1007/978-3-319-47647-6Newe, A., Brandner, J., Aichinger, W., & Becker, L. (2018). An open source tool for creating model files for virtual volume rendering in PDF documents. In Bildverarbeitung für die Medizin 2018, (pp. 133-138) Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-56537-7_97Niven, K., & Richards, J. D. (2017). The storage and long-term preservation of 3D data. In D. Errickson & T. Thompson (Eds.), Human Remains: Another Dimension (pp. 175-184): Academic Press. https://doi.org/10.1016/B978-0-12-804602-9.00013-8Pauwels, P., Verstraeten, R., De Meyer, R., & Van Campenhout, J. (2008). Architectural Information Modelling for Virtual Heritage Application. In Digital Heritage-Proceedings of the 14th International Conference on Virtual Systems and Multimedia, (pp. 18-23).Pavlidis, G., Koutsoudis, A., Arnaoutoglou, F., Tsioukas, V., & Chamzas, C. (2007). Methods for 3D digitization of cultural heritage. Journal of Cultural Heritage, 8(1), 93-98. https://doi.org/10.1016/j.culher.2006.10.007Pletinckx, D., & Nolle, D. (2015). 3D-ICONS: D5.1-Report on 3D publication formats suitable for Europeana. Retrieved from https://zenodo.org/record/1311590#.Xt34Zy97G50. https://doi.org/10.5281/zenodo.1311589Potenziani, M., Callieri, M., Dellepiane, M., Corsini, M., Ponchio, F., & Scopigno, R. (2015). 3DHOP: 3D heritage online presenter. Computers & Graphics, 52, 129-141. http://doi.org/10.1016/j.cag.2015.07.001Rabinowitz, A., Esteva, M., & Trelogan, J. (2013, 26-28 September). Ensuring a future for the past. In Proceedings of The Memory of the World in the Digital Age: Digitization and Preservation, (pp. 940-954) Vancouver, British Columbia, Canada.Rahaman, H., & Champion, E. (2019, 15-18 April). The scholarly rewards and tragic irony of 3D models in virtual heritage discourse. In 24th Annual Conference of the Association for Computer-Aided Architectural Design Research in Asia (CAADRIA 2019), (pp. 695-704) Wellington, New Zealand.Roussou, M. (2002, 24-25 November). Virtual heritage : From the research lab to the broad public. In VAST Euroconference, (pp. 93-100) Arezzo, Italy.Scopigno, R., Callieri, M., Dellepiane, M., Ponchio, F., & Potenziani, M. (2017). Delivering and using 3D models on the web: are we ready? Virtual Archaeology Review, 8(17), 1-9. http://dx.doi.org/10.4995/var.2017.6405Snyder, L. M. (2014). VSim : Scholarly annotations in real-time 3D environments. Paper presented at the DH-CASE II: Collaborative Annotations on Shared Environments: metadata, tools and techniques in the Digital Humanities - DH-CASE '14, (pp. 1-8.) Fort Collins, CA, USA. http://dx.doi.org/10.1145/2657480.2657483Statham, N. (2019). Scientific rigour of online platforms for 3D visualisation of heritage. Virtual Archaeology Review, 10(20), 1-16. https://doi.org/10.4995/var.2019.9715Sullivan, E. (2016). Potential pasts: Taking a humanistic approach to computer visualization of ancient landscapes. Bulletin of the Institute of Classical Studies, 59(2), 71-88. https://doi.org/10.1111/j.2041-5370.2016.12039.xSullivan, E., Nieves, A. D., & Snyder, L. M. (2017). Making the model: Scholarship and rhetoric in 3-D historical reconstructions. In J. Sayers (Ed.), Making Things and Drawing Boundaries : Experiments in the Digital Humanities. Minneapolis, MN: University of Minnesota Press. https://doi.org/10.5749/j.ctt1pwt6wq.38Sullivan, E. A., & Snyder, L. M. (2017). Digital Karnak. Journal of the Society of Architectural Historians, 76(4), 464-482. https://doi.org/10.1525/jsah.2017.76.4.464Thwaites, H. (2013). Digital heritage : What happens when we digitize everything? In E. Ch'ng, V. Gaffney, & H. Chapman (Eds.), Visual heritage in the digital age (pp. 327-348). London, UK: Springer-Verlag. https://doi.org/10.1007/978-1-4471-5535-5Tsiafaki, D., & Michailidou, N. (2015). Benefits and problems through the application of 3D technologies in archaeology: Recording, visualisation, representation and reconstruction. Scientific Culture, 1(3), 37-45. http://doi.org/10.5281/zenodo.18448Tucci, G., Bonora, V., Conti, A., & Fiorini, L. (2017). High-quality 3d models and their use in a cultural heritage conservation project. In The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences : 26th International CIPA Symposium 2017, (pp. 687) Ottawa, Canada. http://doi.org/10.5194/isprs-archives-XLII-2-W5-687-2017Übel, M. V. (2019). Free 3D models - Best download sites & 3D archives of 2019. Get the Best for Free. Retrieved 12 February, 2019, from https://all3dp.com/1/free-3d-models-download-best-sites-3d-archive-3dUNESCO. (2003, 29 Sept-15Oct 2003). Charter on the preservation of the digital heritage. In 32nd Session: The General Conference of the United Nations Educational, Scientific and Cultural Organization, (pp. 74-76) Paris.Zhang, N., Li, Q., Jia, H., Zhang, M., & Liu, J. (2017). U3D file format analyzing and 3DPDF generating method. In IGTA: Chinese Conference on Image and Graphics Technologies : Advances in Image and Graphics Technologies, (pp. 136-146) Singapore. https://doi.org/10.1007/978-981-10-7389-2_1

Influenza research database: an integrated bioinformatics resource for influenza research and surveillance.

BackgroundThe recent emergence of the 2009 pandemic influenza A/H1N1 virus has highlighted the value of free and open access to influenza virus genome sequence data integrated with information about other important virus characteristics.DesignThe Influenza Research Database (IRD, http://www.fludb.org) is a free, open, publicly-accessible resource funded by the U.S. National Institute of Allergy and Infectious Diseases through the Bioinformatics Resource Centers program. IRD provides a comprehensive, integrated database and analysis resource for influenza sequence, surveillance, and research data, including user-friendly interfaces for data retrieval, visualization and comparative genomics analysis, together with personal log in-protected 'workbench' spaces for saving data sets and analysis results. IRD integrates genomic, proteomic, immune epitope, and surveillance data from a variety of sources, including public databases, computational algorithms, external research groups, and the scientific literature.ResultsTo demonstrate the utility of the data and analysis tools available in IRD, two scientific use cases are presented. A comparison of hemagglutinin sequence conservation and epitope coverage information revealed highly conserved protein regions that can be recognized by the human adaptive immune system as possible targets for inducing cross-protective immunity. Phylogenetic and geospatial analysis of sequences from wild bird surveillance samples revealed a possible evolutionary connection between influenza virus from Delaware Bay shorebirds and Alberta ducks.ConclusionsThe IRD provides a wealth of integrated data and information about influenza virus to support research of the genetic determinants dictating virus pathogenicity, host range restriction and transmission, and to facilitate development of vaccines, diagnostics, and therapeutics

PROTEUS: an immersive tool for exploring the world of cultural heritage across space and time scales

AbstractIn the field of digital humanities, it is increasingly necessary to develop and validate virtual reality tools that are capable of combining various scientific data in a virtualized context providing also access and user friendly consultation of online repositories. This paper reports the main aspects of the implementation of a virtual reality tool integrated with an online repository for storing 3D models, metadata and chemical analyses related to different sectors of digital humanities. The virtual reality software, developed for the Oculus Quest 2 hardware, is called PROTEUS and allows for seamless transition from the macroscopic world of digital humanities to the microscopic world of molecular sciences. The paper illustrates, by means of some case studies, the performances of this innovative tool that permits the researcher to understand and manipulate objects, to test hypotheses and to seek meaningful results, visualising the metadata while changing the parameters of the simulation in a dynamic and interactive way. This represents also a significant step forward in the democratisation of science, thanks to an user-friendly and immersive access to advanced scientific algorithms, which allow the natural perception of structural and topological features of the underlying molecular and supra-molecular systems. Graphical Abstrac

Mapping the Current Landscape of Research Library Engagement with Emerging Technologies in Research and Learning: Final Report

The generation, dissemination, and analysis of digital information is a significant driver, and consequence, of technological change. As data and information stewards in physical and virtual space, research libraries are thoroughly entangled in the challenges presented by the Fourth Industrial Revolution:1 a societal shift powered not by steam or electricity, but by data, and characterized by a fusion of the physical and digital worlds.2 Organizing, structuring, preserving, and providing access to growing volumes of the digital data generated and required by research and industry will become a critically important function. As partners with the community of researchers and scholars, research libraries are also recognizing and adapting to the consequences of technological change in the practices of scholarship and scholarly communication. Technologies that have emerged or become ubiquitous within the last decade have accelerated information production and have catalyzed profound changes in the ways scholars, students, and the general public create and engage with information. The production of an unprecedented volume and diversity of digital artifacts, the proliferation of machine learning (ML) technologies,3 and the emergence of data as the “world’s most valuable resource,”4 among other trends, present compelling opportunities for research libraries to contribute in new and significant ways to the research and learning enterprise. Librarians are all too familiar with predictions of the research library’s demise in an era when researchers have so much information at their fingertips. A growing body of evidence provides a resounding counterpoint: that the skills, experience, and values of librarians, and the persistence of libraries as an institution, will become more important than ever as researchers contend with the data deluge and the ephemerality and fragility of much digital content. This report identifies strategic opportunities for research libraries to adopt and engage with emerging technologies,5 with a roughly fiveyear time horizon. It considers the ways in which research library values and professional expertise inform and shape this engagement, the ways library and library worker roles will be reconceptualized, and the implication of a range of technologies on how the library fulfills its mission. The report builds on a literature review covering the last five years of published scholarship, primarily North American information science literature, and interviews with a dozen library field experts, completed in fall 2019. It begins with a discussion of four cross-cutting opportunities that permeate many or all aspects of research library services. Next, specific opportunities are identified in each of five core research library service areas: facilitating information discovery, stewarding the scholarly and cultural record, advancing digital scholarship, furthering student learning and success, and creating learning and collaboration spaces. Each section identifies key technologies shaping user behaviors and library services, and highlights exemplary initiatives. Underlying much of the discussion in this report is the idea that “digital transformation is increasingly about change management”6 —that adoption of or engagement with emerging technologies must be part of a broader strategy for organizational change, for “moving emerging work from the periphery to the core,”7 and a broader shift in conceptualizing the research library and its services. Above all, libraries are benefitting from the ways in which emerging technologies offer opportunities to center users and move from a centralized and often siloed service model to embedded, collaborative engagement with the research and learning enterprise

RUSHES—an annotation and retrieval engine for multimedia semantic units

Multimedia analysis and reuse of raw un-edited audio visual content known as rushes is gaining acceptance by a large number of research labs and companies. A set of research projects are considering multimedia indexing, annotation, search and retrieval in the context of European funded research, but only the FP6 project RUSHES is focusing on automatic semantic annotation, indexing and retrieval of raw and un-edited audio-visual content. Even professional content creators and providers as well as home-users are dealing with this type of content and therefore novel technologies for semantic search and retrieval are required. In this paper, we present a summary of the most relevant achievements of the RUSHES project, focusing on specific approaches for automatic annotation as well as the main features of the final RUSHES search engine