A deep learning framework for quality assessment and restoration in video endoscopy

Endoscopy is a routine imaging technique used for both diagnosis and minimally invasive surgical treatment. Artifacts such as motion blur, bubbles, specular reflections, floating objects and pixel saturation impede the visual interpretation and the automated analysis of endoscopy videos. Given the widespread use of endoscopy in different clinical applications, we contend that the robust and reliable identification of such artifacts and the automated restoration of corrupted video frames is a fundamental medical imaging problem. Existing state-of-the-art methods only deal with the detection and restoration of selected artifacts. However, typically endoscopy videos contain numerous artifacts which motivates to establish a comprehensive solution. We propose a fully automatic framework that can: 1) detect and classify six different primary artifacts, 2) provide a quality score for each frame and 3) restore mildly corrupted frames. To detect different artifacts our framework exploits fast multi-scale, single stage convolutional neural network detector. We introduce a quality metric to assess frame quality and predict image restoration success. Generative adversarial networks with carefully chosen regularization are finally used to restore corrupted frames. Our detector yields the highest mean average precision (mAP at 5% threshold) of 49.0 and the lowest computational time of 88 ms allowing for accurate real-time processing. Our restoration models for blind deblurring, saturation correction and inpainting demonstrate significant improvements over previous methods. On a set of 10 test videos we show that our approach preserves an average of 68.7% which is 25% more frames than that retained from the raw videos.Comment: 14 page

Optimising additive manufacturing for fine art sculpture and digital restoration of archaeological artefacts

Additive manufacturing (AM) has shown itself to be beneficial in many application areas, including product design and manufacture, medical models and prosthetics, architectural modelling and artistic endeavours. For some of these applications, coupling AM with reverse engineering (RE) enables the utilisation of data from existing 3D shapes. This thesis describes the application of AM and RE within sculpture manufacture, in order to optimise the process chains for sculpture reproduction and relic conservation and restoration. This area poses particular problems since the original artefacts can often be fragile and inaccessible, and the finishing required on the AM replicas is both complex and varied. Several case studies within both literature and practical projects are presented, which cover essential knowledge of producing large scale sculptures from an original models as well as a wide range of artefact shapes and downstream finishing techniques. The combination of digital technologies and traditional art requires interdisciplinary knowledge across engineering and fine art. Also, definitions and requirements (e.g. ‘accuracy’), can be applied as both engineering and artistic terms when specifications and trade-offs are being considered. The thesis discusses the feasibility for using these technologies across domains, and explores the potential for developing new market opportunities for AM. It presents and analyses a number of case study projects undertaken by the author with a view to developing cost and time models for various processes used. These models have then been used to develop a series of "process maps", which enable users of AM in this area to decide upon the optimum process route to follow, under various circumstances. The maps were validated and user feedback obtained through the execution of two further sculpture manufacturing projects. The thesis finishes with conclusions about the feasibility of the approach, its constraints, the pros and cons of adopting AM in this area and recommendations for future research

Digital Techniques for Documenting and Preserving Cultural Heritage

In this unique collection the authors present a wide range of interdisciplinary methods to study, document, and conserve material cultural heritage. The methods used serve as exemplars of best practice with a wide variety of cultural heritage objects having been recorded, examined, and visualised. The objects range in date, scale, materials, and state of preservation and so pose different research questions and challenges for digitization, conservation, and ontological representation of knowledge. Heritage science and specialist digital technologies are presented in a way approachable to non-scientists, while a separate technical section provides details of methods and techniques, alongside examples of notable applications of spatial and spectral documentation of material cultural heritage, with selected literature and identification of future research. This book is an outcome of interdisciplinary research and debates conducted by the participants of the COST Action TD1201, Colour and Space in Cultural Heritage, 2012–16 and is an Open Access publication available under a CC BY-NC-ND licence.https://scholarworks.wmich.edu/mip_arc_cdh/1000/thumbnail.jp

Digital Techniques for Documenting and Preserving Cultural Heritage

This book presents interdisciplinary approaches to the examination and documentation of material cultural heritage, using non-invasive spatial and spectral optical technologies

3D Information Technologies in Cultural Heritage Preservation and Popularisation

This Special Issue of the journal Applied Sciences presents recent advances and developments in the use of digital 3D technologies to protect and preserve cultural heritage. While most of the articles focus on aspects of 3D scanning, modeling, and presenting in VR of cultural heritage objects from buildings to small artifacts and clothing, part of the issue is devoted to 3D sound utilization in the cultural heritage field

Prototype software for colorant formulation using Gamblin conservation colors

When selecting pigments from a large set for restorative inpainting, it can often be challenging to create a mixture that will provide an exact match to the original artwork under a range of viewing and illumination conditions. In this research, a prototype computer program was developed that will aid the user by providing a color match and paint recipe that exhibits minimal metamerism when compared to the original artwork. The Gamblin Conservation Colors, a set of 43 colorants specially formulated for inpainting, were characterized in terms of their optical properties, absorption and scattering, according to Kubelka-Munk turbid media theory. Formulations were made using traditional spectrophotometric measurements and image-based measurements. The multispectral imaging system consisted of a trichromatic CFA camera coupled with two absorption filters; spectral reflectance data for each pixel location was estimated with a transformation based on calibration target images. Three targets were used for testing formulation accuracy: a target consisting of mixtures of Gamblin Conservation Colors, and two oil paintings. Pigment selection was reasonably successful, and good predictions resulted from both measurement techniques, but for more complex tasks such as pigment identification, a more rigorous colorant characterization approach may be needed. Predictions from image-based measurements were generally less accurate, and improvements in the camera model would likely remedy this. It is expected that this software will be of assistance to conservators by simplifying the process of selecting from a large set of available pigments, as well as reducing the possibility of damage to painted surfaces in cases where direct measurements are impractical. The open source nature of the software provides the opportunity for changes and addition of features in the future

Study on quality in 3D digitisation of tangible cultural heritage: mapping parameters, formats, standards, benchmarks, methodologies and guidelines: final study report.

This study was commissioned by the Commission to help advance 3D digitisation across Europe and thereby to support the objectives of the Recommendation on a common European data space for cultural heritage (C(2021) 7953 final), adopted on 10 November 2021. The Recommendation encourages Member States to set up digital strategies for cultural heritage, which sets clear digitisation and digital preservation goals aiming at higher quality through the use of advanced technologies, notably 3D. The aim of the study is to map the parameters, formats, standards, benchmarks, methodologies and guidelines relating to 3D digitisation of tangible cultural heritage. The overall objective is to further the quality of 3D digitisation projects by enabling cultural heritage professionals, institutions, content-developers, stakeholders and academics to define and produce high-quality digitisation standards for tangible cultural heritage. This unique study identifies key parameters of the digitisation process, estimates the relative complexity and how it is linked to technology, its impact on quality and its various factors. It also identifies standards and formats used for 3D digitisation, including data types, data formats and metadata schemas for 3D structures. Finally, the study forecasts the potential impacts of future technological advances on 3D digitisation

Electronic Imaging & the Visual Arts. EVA 2012 Florence

The key aim of this Event is to provide a forum for the user, supplier and scientific research communities to meet and exchange experiences, ideas and plans in the wide area of Culture & Technology. Participants receive up to date news on new EC and international arts computing & telecommunications initiatives as well as on Projects in the visual arts field, in archaeology and history. Working Groups and new Projects are promoted. Scientific and technical demonstrations are presented

Digital Image Access & Retrieval

The 33th Annual Clinic on Library Applications of Data Processing, held at the University of Illinois at Urbana-Champaign in March of 1996, addressed the theme of "Digital Image Access & Retrieval." The papers from this conference cover a wide range of topics concerning digital imaging technology for visual resource collections. Papers covered three general areas: (1) systems, planning, and implementation; (2) automatic and semi-automatic indexing; and (3) preservation with the bulk of the conference focusing on indexing and retrieval.published or submitted for publicatio