Communication of Digital Material Appearance Based on Human Perception

Im alltägliche Leben begegnen wir digitalen Materialien in einer Vielzahl von Situationen wie beispielsweise bei Computerspielen, Filmen, Reklamewänden in zB U-Bahn Stationen oder beim Online-Kauf von Kleidungen. Während einige dieser Materialien durch digitale Modelle repräsentiert werden, welche das Aussehen einer bestimmten Oberfläche in Abhängigkeit des Materials der Fläche sowie den Beleuchtungsbedingungen beschreiben, basieren andere digitale Darstellungen auf der simplen Verwendung von Fotos der realen Materialien, was zB bei Online-Shopping häufig verwendet wird. Die Verwendung von computer-generierten Materialien ist im Vergleich zu einzelnen Fotos besonders vorteilhaft, da diese realistische Erfahrungen im Rahmen von virtuellen Szenarien, kooperativem Produkt-Design, Marketing während der prototypischen Entwicklungsphase oder der Ausstellung von Möbeln oder Accesoires in spezifischen Umgebungen erlauben. Während mittels aktueller Digitalisierungsmethoden bereits eine beeindruckende Reproduktionsqualität erzielt wird, wird eine hochpräzise photorealistische digitale Reproduktion von Materialien für die große Vielfalt von Materialtypen nicht erreicht. Daher verwenden viele Materialkataloge immer noch Fotos oder sogar physikalische Materialproben um ihre Kollektionen zu repräsentieren. Ein wichtiger Grund für diese Lücke in der Genauigkeit des Aussehens von digitalen zu echten Materialien liegt darin, dass die Zusammenhänge zwischen physikalischen Materialeigenschaften und der vom Menschen wahrgenommenen visuellen Qualität noch weitgehend unbekannt sind. Die im Rahmen dieser Arbeit durchgeführten Untersuchungen adressieren diesen Aspekt. Zu diesem Zweck werden etablierte digitalie Materialmodellen bezüglich ihrer Eignung zur Kommunikation von physikalischen und sujektiven Materialeigenschaften untersucht, wobei Beobachtungen darauf hinweisen, dass ein Teil der fühlbaren/haptischen Informationen wie z.B. Materialstärke oder Härtegrad aufgrund der dem Modell anhaftenden geometrische Abstraktion verloren gehen. Folglich wird im Rahmen der Arbeit das Zusammenspiel der verschiedenen Sinneswahrnehmungen (mit Fokus auf die visuellen und akustischen Modalitäten) untersucht um festzustellen, welche Informationen während des Digitalisierungsprozesses verloren gehen. Es zeigt sich, dass insbesondere akustische Informationen in Kombination mit der visuellen Wahrnehmung die Einschätzung fühlbarer Materialeigenschaften erleichtert. Eines der Defizite bei der Analyse des Aussehens von Materialien ist der Mangel bezüglich sich an der Wahnehmung richtenden Metriken die eine Beantwortung von Fragen wie z.B. "Sind die Materialien A und B sich ähnlicher als die Materialien C und D?" erlauben, wie sie in vielen Anwendungen der Computergrafik auftreten. Daher widmen sich die im Rahmen dieser Arbeit durchgeführten Studien auch dem Vergleich von unterschiedlichen Materialrepräsentationen im Hinblick auf. Zu diesem Zweck wird eine Methodik zur Berechnung der wahrgenommenen paarweisen Ähnlichkeit von Material-Texturen eingeführt, welche auf der Verwendung von Textursyntheseverfahren beruht und sich an der Idee/dem Begriff der geradenoch-wahrnehmbaren Unterschiede orientiert. Der vorgeschlagene Ansatz erlaubt das Überwinden einiger Probleme zuvor veröffentlichter Methoden zur Bestimmung der Änhlichkeit von Texturen und führt zu sinnvollen/plausiblen Distanzen von Materialprobem. Zusammenfassend führen die im Rahmen dieser Dissertation dargestellten Inhalte/Verfahren zu einem tieferen Verständnis bezüglich der menschlichen Wahnehmung von digitalen bzw. realen Materialien über unterschiedliche Sinne, einem besseren Verständnis bzgl. der Bewertung der Ähnlichkeit von Texturen durch die Entwicklung einer neuen perzeptuellen Metrik und liefern grundlegende Einsichten für zukünftige Untersuchungen im Bereich der Perzeption von digitalen Materialien.In daily life, we encounter digital materials and interact with them in numerous situations, for instance when we play computer games, watch a movie, see billboard in the metro station or buy new clothes online. While some of these virtual materials are given by computational models that describe the appearance of a particular surface based on its material and the illumination conditions, some others are presented as simple digital photographs of real materials, as is usually the case for material samples from online retailing stores. The utilization of computer-generated materials entails significant advantages over plain images as they allow realistic experiences in virtual scenarios, cooperative product design, advertising in prototype phase or exhibition of furniture and wearables in specific environments. However, even though exceptional material reproduction quality has been achieved in the domain of computer graphics, current technology is still far away from highly accurate photo-realistic virtual material reproductions for the wide range of existing categories and, for this reason, many material catalogs still use pictures or even physical material samples to illustrate their collections. An important reason for this gap between digital and real material appearance is that the connections between physical material characteristics and the visual quality perceived by humans are far from well-understood. Our investigations intend to shed some light in this direction. Concretely, we explore the ability of state-of-the-art digital material models in communicating physical and subjective material qualities, observing that part of the tactile/haptic information (eg thickness, hardness) is missing due to the geometric abstractions intrinsic to the model. Consequently, in order to account for the information deteriorated during the digitization process, we investigate the interplay between different sensing modalities (vision and hearing) and discover that particular sound cues, in combination with visual information, facilitate the estimation of such tactile material qualities. One of the shortcomings when studying material appearance is the lack of perceptually-derived metrics able to answer questions like "are materials A and B more similar than C and D?", which arise in many computer graphics applications. In the absence of such metrics, our studies compare different appearance models in terms of how capable are they to depict/transmit a collection of meaningful perceptual qualities. To address this problem, we introduce a methodology to compute the perceived pairwise similarity between textures from material samples that makes use of patch-based texture synthesis algorithms and is inspired on the notion of Just-Noticeable Differences. Our technique is able to overcome some of the issues posed by previous texture similarity collection methods and produces meaningful distances between samples. In summary, with the contents presented in this thesis we are able to delve deeply in how humans perceive digital and real materials through different senses, acquire a better understanding of texture similarity by developing a perceptually-based metric and provide a groundwork for further investigations in the perception of digital materials

Visual grasp point localization, classification and state recognition in robotic manipulation of cloth: an overview

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/Cloth manipulation by robots is gaining popularity among researchers because of its relevance, mainly (but not only) in domestic and assistive robotics. The required science and technologies begin to be ripe for the challenges posed by the manipulation of soft materials, and many contributions have appeared in the last years. This survey provides a systematic review of existing techniques for the basic perceptual tasks of grasp point localization, state estimation and classification of cloth items, from the perspective of their manipulation by robots. This choice is grounded on the fact that any manipulative action requires to instruct the robot where to grasp, and most garment handling activities depend on the correct recognition of the type to which the particular cloth item belongs and its state. The high inter- and intraclass variability of garments, the continuous nature of the possible deformations of cloth and the evident difficulties in predicting their localization and extension on the garment piece are challenges that have encouraged the researchers to provide a plethora of methods to confront such problems, with some promising results. The present review constitutes for the first time an effort in furnishing a structured framework of these works, with the aim of helping future contributors to gain both insight and perspective on the subjectPeer ReviewedPostprint (author's final draft

Haptics Rendering and Applications

There has been significant progress in haptic technologies but the incorporation of haptics into virtual environments is still in its infancy. A wide range of the new society's human activities including communication, education, art, entertainment, commerce and science would forever change if we learned how to capture, manipulate and reproduce haptic sensory stimuli that are nearly indistinguishable from reality. For the field to move forward, many commercial and technological barriers need to be overcome. By rendering how objects feel through haptic technology, we communicate information that might reflect a desire to speak a physically- based language that has never been explored before. Due to constant improvement in haptics technology and increasing levels of research into and development of haptics-related algorithms, protocols and devices, there is a belief that haptics technology has a promising future

The Perception of Surface Properties: Translucence and Gloss

The human visual system is sensitive to differences in gloss and translucence, two optical properties which are found in conjunction in many natural materials. They are driven by similar underlying physical properties of light transport - the degree to which light is scattered from the surface of a material, or within the material. This thesis aimed to address some fundamental questions about how gloss and translucence are perceived. Two psychophysical methods (maximum likelihood difference scaling, and conjoint measurement) were used throughout, as they provided an appropriate way of investigating how perceptual experiences related to physical variables. In the introduction, I review the literature on the perception of gloss and translucence. Study 1 investigated the relationship between variables controlling light transport in translucent volumes and percepts of translucence. The results show that translucence perception is not based on estimates of light transport properties per se, but probably uses spatially-related statistical pseudocues in conjunction with other cues. Study 2 examined a similar issue, but the translucent material was presented as a layer enveloping a solid object. Behavioural responses were similar for these translucent materials, which were perceived as glossy layers of coating. Study 3 further explored established findings that perceived translucence shows inconstancy under changes in viewing condition. Perceived translucence was dependent in a complex way on both light-scattering in the material and illumination direction in both volumes and layers of translucent materials. Study 4 used similar layers of subsurface light-scattering and -absorbing material and applied them to multiple base materials. Opacity and a lack of mirror-like reflections enabled observers to make the most accurate independent judgements of darkness and cloudiness. Study 5 explored observers' sensitivity to spatial variation of scatter across a surface using similar layers of coating, and the way in which observers might weight cues differently to answer subtly different questions (judgements of 'shininess' vs. 'cleanliness'). Layer thickness and variation of scatter significantly affected perceived shine and cleanliness, with layer thickness influencing decisions more than variation. Scatter variation contributed to decisions significantly more for judgements of cleanliness than shine. Study 6 investigated how tactile surface roughness influenced perceived gloss. Previous findings have shown that tactile compliance and friction influence perceived gloss, and that friction interacts with visual gloss. Our results showed that surface roughness and visual gloss both affected perceived gloss, but there was no interaction, suggesting that different types of haptic information are combined with visual information differently. Finally, study 7 explored the potential cortical basis of perceived translucence. Through testing a neuropsychological patient, we showed that perceived translucence is dependent on cortical areas not responsible for colour or texture discrimination. The thesis concludes with a discussion of additional recent findings, the implications of the research reported in this thesis, and proposals for future research

Navigation system based in motion tracking sensor for percutaneous renal access

Tese de Doutoramento em Engenharia BiomédicaMinimally-invasive kidney interventions are daily performed to diagnose and treat several renal diseases. Percutaneous renal access (PRA) is an essential but challenging stage for most of these procedures, since its outcome is directly linked to the physician’s ability to precisely visualize and reach the anatomical target. Nowadays, PRA is always guided with medical imaging assistance, most frequently using X-ray based imaging (e.g. fluoroscopy). Thus, radiation on the surgical theater represents a major risk to the medical team, where its exclusion from PRA has a direct impact diminishing the dose exposure on both patients and physicians. To solve the referred problems this thesis aims to develop a new hardware/software framework to intuitively and safely guide the surgeon during PRA planning and puncturing. In terms of surgical planning, a set of methodologies were developed to increase the certainty of reaching a specific target inside the kidney. The most relevant abdominal structures for PRA were automatically clustered into different 3D volumes. For that, primitive volumes were merged as a local optimization problem using the minimum description length principle and image statistical properties. A multi-volume Ray Cast method was then used to highlight each segmented volume. Results show that it is possible to detect all abdominal structures surrounding the kidney, with the ability to correctly estimate a virtual trajectory. Concerning the percutaneous puncturing stage, either an electromagnetic or optical solution were developed and tested in multiple in vitro, in vivo and ex vivo trials. The optical tracking solution aids in establishing the desired puncture site and choosing the best virtual puncture trajectory. However, this system required a line of sight to different optical markers placed at the needle base, limiting the accuracy when tracking inside the human body. Results show that the needle tip can deflect from its initial straight line trajectory with an error higher than 3 mm. Moreover, a complex registration procedure and initial setup is needed. On the other hand, a real-time electromagnetic tracking was developed. Hereto, a catheter was inserted trans-urethrally towards the renal target. This catheter has a position and orientation electromagnetic sensor on its tip that function as a real-time target locator. Then, a needle integrating a similar sensor is used. From the data provided by both sensors, one computes a virtual puncture trajectory, which is displayed in a 3D visualization software. In vivo tests showed a median renal and ureteral puncture times of 19 and 51 seconds, respectively (range 14 to 45 and 45 to 67 seconds). Such results represent a puncture time improvement between 75% and 85% when comparing to state of the art methods. 3D sound and vibrotactile feedback were also developed to provide additional information about the needle orientation. By using these kind of feedback, it was verified that the surgeon tends to follow a virtual puncture trajectory with a reduced amount of deviations from the ideal trajectory, being able to anticipate any movement even without looking to a monitor. Best results show that 3D sound sources were correctly identified 79.2 ± 8.1% of times with an average angulation error of 10.4º degrees. Vibration sources were accurately identified 91.1 ± 3.6% of times with an average angulation error of 8.0º degrees. Additionally to the EMT framework, three circular ultrasound transducers were built with a needle working channel. One explored different manufacture fabrication setups in terms of the piezoelectric materials, transducer construction, single vs. multi array configurations, backing and matching material design. The A-scan signals retrieved from each transducer were filtered and processed to automatically detect reflected echoes and to alert the surgeon when undesirable anatomical structures are in between the puncture path. The transducers were mapped in a water tank and tested in a study involving 45 phantoms. Results showed that the beam cross-sectional area oscillates around the ceramics radius and it was possible to automatically detect echo signals in phantoms with length higher than 80 mm. Hereupon, it is expected that the introduction of the proposed system on the PRA procedure, will allow to guide the surgeon through the optimal path towards the precise kidney target, increasing surgeon’s confidence and reducing complications (e.g. organ perforation) during PRA. Moreover, the developed framework has the potential to make the PRA free of radiation for both patient and surgeon and to broad the use of PRA to less specialized surgeons.Intervenções renais minimamente invasivas são realizadas diariamente para o tratamento e diagnóstico de várias doenças renais. O acesso renal percutâneo (ARP) é uma etapa essencial e desafiante na maior parte destes procedimentos. O seu resultado encontra-se diretamente relacionado com a capacidade do cirurgião visualizar e atingir com precisão o alvo anatómico. Hoje em dia, o ARP é sempre guiado com recurso a sistemas imagiológicos, na maior parte das vezes baseados em raios-X (p.e. a fluoroscopia). A radiação destes sistemas nas salas cirúrgicas representa um grande risco para a equipa médica, aonde a sua remoção levará a um impacto direto na diminuição da dose exposta aos pacientes e cirurgiões. De modo a resolver os problemas existentes, esta tese tem como objetivo o desenvolvimento de uma framework de hardware/software que permita, de forma intuitiva e segura, guiar o cirurgião durante o planeamento e punção do ARP. Em termos de planeamento, foi desenvolvido um conjunto de metodologias de modo a aumentar a eficácia com que o alvo anatómico é alcançado. As estruturas abdominais mais relevantes para o procedimento de ARP, foram automaticamente agrupadas em volumes 3D, através de um problema de optimização global com base no princípio de “minimum description length” e propriedades estatísticas da imagem. Por fim, um procedimento de Ray Cast, com múltiplas funções de transferência, foi utilizado para enfatizar as estruturas segmentadas. Os resultados mostram que é possível detetar todas as estruturas abdominais envolventes ao rim, com a capacidade para estimar corretamente uma trajetória virtual. No que diz respeito à fase de punção percutânea, foram testadas duas soluções de deteção de movimento (ótica e eletromagnética) em múltiplos ensaios in vitro, in vivo e ex vivo. A solução baseada em sensores óticos ajudou no cálculo do melhor ponto de punção e na definição da melhor trajetória a seguir. Contudo, este sistema necessita de uma linha de visão com diferentes marcadores óticos acoplados à base da agulha, limitando a precisão com que a agulha é detetada no interior do corpo humano. Os resultados indicam que a agulha pode sofrer deflexões à medida que vai sendo inserida, com erros superiores a 3 mm. Por outro lado, foi desenvolvida e testada uma solução com base em sensores eletromagnéticos. Para tal, um cateter que integra um sensor de posição e orientação na sua ponta, foi colocado por via trans-uretral junto do alvo renal. De seguida, uma agulha, integrando um sensor semelhante, é utilizada para a punção percutânea. A partir da diferença espacial de ambos os sensores, é possível gerar uma trajetória de punção virtual. A mediana do tempo necessário para puncionar o rim e ureter, segundo esta trajetória, foi de 19 e 51 segundos, respetivamente (variações de 14 a 45 e 45 a 67 segundos). Estes resultados representam uma melhoria do tempo de punção entre 75% e 85%, quando comparados com o estado da arte dos métodos atuais. Além do feedback visual, som 3D e feedback vibratório foram explorados de modo a fornecer informações complementares da posição da agulha. Verificou-se que com este tipo de feedback, o cirurgião tende a seguir uma trajetória de punção com desvios mínimos, sendo igualmente capaz de antecipar qualquer movimento, mesmo sem olhar para o monitor. Fontes de som e vibração podem ser corretamente detetadas em 79,2 ± 8,1% e 91,1 ± 3,6%, com erros médios de angulação de 10.4º e 8.0 graus, respetivamente. Adicionalmente ao sistema de navegação, foram também produzidos três transdutores de ultrassom circulares com um canal de trabalho para a agulha. Para tal, foram exploradas diferentes configurações de fabricação em termos de materiais piezoelétricos, transdutores multi-array ou singulares e espessura/material de layers de suporte. Os sinais originados em cada transdutor foram filtrados e processados de modo a detetar de forma automática os ecos refletidos, e assim, alertar o cirurgião quando existem variações anatómicas ao longo do caminho de punção. Os transdutores foram mapeados num tanque de água e testados em 45 phantoms. Os resultados mostraram que o feixe de área em corte transversal oscila em torno do raio de cerâmica, e que os ecos refletidos são detetados em phantoms com comprimentos superiores a 80 mm. Desta forma, é expectável que a introdução deste novo sistema a nível do ARP permitirá conduzir o cirurgião ao longo do caminho de punção ideal, aumentado a confiança do cirurgião e reduzindo possíveis complicações (p.e. a perfuração dos órgãos). Além disso, de realçar que este sistema apresenta o potencial de tornar o ARP livre de radiação e alarga-lo a cirurgiões menos especializados.The present work was only possible thanks to the support by the Portuguese Science and Technology Foundation through the PhD grant with reference SFRH/BD/74276/2010 funded by FCT/MEC (PIDDAC) and by Fundo Europeu de Desenvolvimento Regional (FEDER), Programa COMPETE - Programa Operacional Factores de Competitividade (POFC) do QREN

What is the Avatar? Fiction and Embodiment in Avatar-Based Singleplayer Computer Games: Revised and Commented Edition

What are the characteristic features of avatar-based singleplayer videogames, from Super Mario Bros. to Grand Theft Auto? The author examines this question with a particular focus on issues of fictionality and realism, and their relation to cinema and Virtual Reality. Through close-up analysis and philosophical discussion, the author argues that avatar-based gaming is a distinctive and dominant form of virtual self-embodiment in digital culture. This book is a revised edition of Rune Klevjer's pioneering work from 2007, featuring a new introduction by the author and afterword by Stephan Günzel, Jörg Sternagel, and Dieter Mersch

Objective Assessment of Area and Erythema of Psoriasis Lesion Using Digital Imaging and Colourimetry

Psoriasis is a non-contagious skin disease which typically consists of red plaques covered by silvery-white scales. It affects about 3% of world population. During treatment, dermatologists monitor the extent of psoriasis continuously to ascertain treatment efficacy. Psoriasis Area and Severity Index (PAS!) is the current gold standard method used to assess the extent of psoriasis. In PAS!, there are four parameters to be scored i.e., the surface area affected, erythema (redness), thickness and scaliness of the plaques. Determining PAS! score is a tedious task and thus it is not used in daily clinical practice. In addition, the PAS! parameters are visually determined and may result in intra-observer and inter-observer variations, even by experienced dermatologists. Objective methods in assessing area and erythema of psoriasis lesion have been developed in this thesis. Psoriasis lesion can be recognized by its colour dissimilarity with normal skin. Colour dissimilarity is represented by colour difference in CIELAB colour space, a widely used colour space to measure colour dissimilarity. Each pixel in CIELAB colour space can be represented by its lightness (L'), hue (hob), and chroma (Cab). Colour difference between psoriasis lesion and normal skin is analyzed in hue-chroma plane of CIELAB colour space. Centroids of normal skin and lesion in hue-chroma space are obtained from selected samples. Euclidean distances between all pixels with these two centroids are then calculated. Each pixel is assigned to the class of the nearest centroid. The erythema of psoriasis lesion is affected by degree of severity and skin pigmentation. In order to assess the erythema objectively, patients are grouped according to their skin pigmentation level. The L* value of normal skin which represents skin pigmentation level is utilized to group the patient into the three skin types namely fair, brown and dark skin types. Light difference (t.L*), hue difference (t.hab), and chroma difference (t.C'ab) of CIELAB colour space between reference lesions and the surrounding normal skin are analyzed. It is found that the erythema score of a lesion can be determined by their hue difference (t.hab) value within a particular skin type group. Out of 30 body regions, the proposed method is able to give the same PAS! area score as reference for 28 body regions. The proposed method is able to determine PAS! erythema score of 82 lesions obtained from 22 patients objectively without being influenced by other characteristic of the lesion such as area, pattern, and boundary

Brain Circuits of Emotion Discrimination

The prefrontal cortex (PFC) is implicated in processing of the affective state of others through non-verbal communication. This social cognitive function is thought to rely on an intact cortical neuronal excitatory and inhibitory balance. Here combining in vivo electrophysiology with a behavioral task for affective state discrimination in mice, we show a differential activation of medial PFC (mPFC) neurons during social exploration that depends on the affective state of the conspecific. Optogenetic manipulations revealed a double dissociation between the role of interneurons in social cognition. Specifically, inhibition of mPFC somatostatin (SOM+), but not of parvalbumin (PV+) interneurons, abolishes affective state discrimination. Accordingly, synchronized activation of mPFC SOM+ interneurons selectively induces social discrimination. As visualized by in vivo singlecell microendoscopic Ca2+ imaging, an increased synchronous activity of mPFC SOM+ interneurons, guiding inhibition of pyramidal neurons, is associated with affective state discrimination. Our findings provide new insights into the neurobiological mechanisms of affective state discrimination.Recognition of other\u2019s emotions influences the way social animals interact and adapt to the environment. The neuropeptide oxytocin (OXT) has been implicated in different aspects of emotion processing. However, the role of endogenous OXT brain pathways in the social response to different emotional states in conspecifics remains elusive. Here, using a combination of anatomical, genetic, and chemogenetic approaches, we investigated the contribution of endogenous OXT signaling in the ability of mice to discriminate unfamiliar conspecifics based on their emotional states. We found that OXTergic projections from the paraventricular nucleus of the hypothalamus (PVN) to the central amygdala (CeA) are crucial for the discrimination of both positively and negatively valenced emotional states. In contrast, blocking PVN OXT release into the nucleus accumbens, prefrontal cortex, and hippocampal CA2 did not alter this emotion discrimination. Furthermore, silencing each of these PVN OXT pathways did not influence basic social interaction. These findings were further supported by the demonstration that virally mediated enhancement of OXT signaling within the CeA was sufficient to rescue emotion discrimination deficits in a genetic mouse model of cognitive liability. Our results indicate that CeA OXT signaling plays a key role in emotion discrimination both in physiological and pathological conditions

What is the Avatar?

What are the characteristic features of avatar-based singleplayer videogames, from Super Mario Bros. to Grand Theft Auto? Rune Klevjer examines this question with a particular focus on issues of fictionality and realism, and their relation to cinema and Virtual Reality. Through close-up analysis and philosophical discussion, Klevjer argues that avatar-based gaming is a distinctive and dominant form of virtual self-embodiment in digital culture. This book is a revised edition of Rune Klevjer's pioneering work from 2007, featuring a new introduction by the author and afterword by Stephan Günzel, Jörg Sternagel, and Dieter Mersch