A Novel Framework for Highlight Reflectance Transformation Imaging

We propose a novel pipeline and related software tools for processing the multi-light image collections (MLICs) acquired in different application contexts to obtain shape and appearance information of captured surfaces, as well as to derive compact relightable representations of them. Our pipeline extends the popular Highlight Reflectance Transformation Imaging (H-RTI) framework, which is widely used in the Cultural Heritage domain. We support, in particular, perspective camera modeling, per-pixel interpolated light direction estimation, as well as light normalization correcting vignetting and uneven non-directional illumination. Furthermore, we propose two novel easy-to-use software tools to simplify all processing steps. The tools, in addition to support easy processing and encoding of pixel data, implement a variety of visualizations, as well as multiple reflectance-model-fitting options. Experimental tests on synthetic and real-world MLICs demonstrate the usefulness of the novel algorithmic framework and the potential benefits of the proposed tools for end-user applications.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091DSURF project (PRIN 2015) funded by the Italian Ministry of University and ResearchSardinian Regional Authorities under projects VIGEC and Vis&VideoLa

Patient-specific bronchoscope simulation with pq-space-based 2D/3D registration

Objective: The use of patient-specific models for surgical simulation requires photorealistic rendering of 3D structure and surface properties. For bronchoscope simulation, this requires augmenting virtual bronchoscope views generated from 3D tomographic data with patient-specific bronchoscope videos. To facilitate matching of video images to the geometry extracted from 3D tomographic data, this paper presents a new pq-space-based 2D/3D registration method for camera pose estimation in bronchoscope tracking. Methods: The proposed technique involves the extraction of surface normals for each pixel of the video images by using a linear local shape-from-shading algorithm derived from the unique camera/lighting constraints of the endoscopes. The resultant pq-vectors are then matched to those of the 3D model by differentiation of the z-buffer. A similarity measure based on angular deviations of the pq-vectors is used to provide a robust 2D/3D registration framework. Localization of tissue deformation is considered by assessing the temporal variation of the pq-vectors between subsequent frames. Results: The accuracy of the proposed method was assessed by using an electromagnetic tracker and a specially constructed airway phantom. Preliminary in vivo validation of the proposed method was performed on a matched patient bronchoscope video sequence and 3D CT data. Comparison to existing intensity-based techniques was also made. Conclusion: The proposed method does not involve explicit feature extraction and is relatively immune to illumination changes. The temporal variation of the pq distribution also permits the identification of localized deformation, which offers an effective way of excluding such areas from the registration process

Astigmatism and Pseudoaccommodation in Pseudophakic Eyes

noAdvanced IOLs with circumferential zones of different power provide pseudoaccommodation. We investigated the potential for power variation with meridian, namely astigmatism, to provide pseudo-accommodation. With appropriate power and axis orientations, acceptable pseudo-accommodation can be achieved

Overcoming the Challenges Associated with Image-based Mapping of Small Bodies in Preparation for the OSIRIS-REx Mission to (101955) Bennu

The OSIRIS-REx Asteroid Sample Return Mission is the third mission in NASA's New Frontiers Program and is the first U.S. mission to return samples from an asteroid to Earth. The most important decision ahead of the OSIRIS-REx team is the selection of a prime sample-site on the surface of asteroid (101955) Bennu. Mission success hinges on identifying a site that is safe and has regolith that can readily be ingested by the spacecraft's sampling mechanism. To inform this mission-critical decision, the surface of Bennu is mapped using the OSIRIS-REx Camera Suite and the images are used to develop several foundational data products. Acquiring the necessary inputs to these data products requires observational strategies that are defined specifically to overcome the challenges associated with mapping a small irregular body. We present these strategies in the context of assessing candidate sample-sites at Bennu according to a framework of decisions regarding the relative safety, sampleability, and scientific value across the asteroid's surface. To create data products that aid these assessments, we describe the best practices developed by the OSIRIS-REx team for image-based mapping of irregular small bodies. We emphasize the importance of using 3D shape models and the ability to work in body-fixed rectangular coordinates when dealing with planetary surfaces that cannot be uniquely addressed by body-fixed latitude and longitude.Comment: 31 pages, 10 figures, 2 table

Wavefront sensing and conjugate adaptive optics in wide-field microscopy

The quality of microscopy imaging is often degraded by sample-induced aberrations. Adaptive optics (AO) is a standard approach to counter such aberrations. In common practice of AO, an active optical correction element, usually a deformable mirror (DM), is usually inserted in the pupil plane of the objective lens, namely pupil AO. However, as first proposed in the astronomy community and now gradually recognized by the optical microscopy community, the placement of the DM in a plane conjugate to a primary sample-induced aberration plane can be more advantageous, especially in situations where the aberration is spatially varying and arises mainly from a dominant layer. We refer to this technique as conjugate AO. In this thesis, we describe two novel implementations of sensor-based conjugate AO in wide-field microscopy, as well as the wavefront sensing techniques we developed for these implementations. Our first implementation is in trans-illumination configuration. The wavefront sensor is based on a technique called partitioned aperture wavefront (PAW) sensing, previously developed in our lab for quantitative phase contrast imaging. Our second conjugate AO is implemented with fluorescence microscopy. The wavefront sensing strategy is based on oblique back-illumination. In both implementations, we addressed the key challenges of developing wavefront sensors that are capable of operating with uncollimated light, which exhibits large diverging angles and may arbitrarily distribute as well. We show that both conjugate AO systems and their wavefront sensors are not only robust, well-suited for video-rate imaging, but also provide large corrected field of view, which is only limited by the microscope itself