Adaptive Methods for Color Vision Impaired Users

Color plays a key role in the understanding of the information in computer environments. It happens that about 5% of the world population is affected by color vision deficiency (CVD), also called color blindness. This visual impairment hampers the color perception, ending up by limiting the overall perception that CVD people have about the surrounding environment, no matter it is real or virtual. In fact, a CVD individual may not distinguish between two different colors, what often originates confusion or a biased understanding of the reality, including web environments, whose web pages are plenty of media elements like text, still images, video, sprites, and so on. Aware of the difficulties that color-blind people may face in interpreting colored contents, a significant number of recoloring algorithms have been proposed in the literature with the purpose of improving the visual perception of those people somehow. However, most of those algorithms lack a systematic study of subjective assessment, what undermines their validity, not to say usefulness. Thus, in the sequel of the research work behind this Ph.D. thesis, the central question that needs to be answered is whether recoloring algorithms are of any usefulness and help for colorblind people or not. With this in mind, we conceived a few preliminary recoloring algorithms that were published in conference proceedings elsewhere. Except the algorithm detailed in Chapter 3, these conference algorithms are not described in this thesis, though they have been important to engender those presented here. The first algorithm (Chapter 3) was designed and implemented for people with dichromacy to improve their color perception. The idea is to project the reddish hues onto other hues that are perceived more regularly by dichromat people. The second algorithm (Chapter 4) is also intended for people with dichromacy to improve their perception of color, but its applicability covers the adaptation of text and image, in HTML5- compliant web environments. This enhancement of color contrast of text and imaging in web pages is done while keeping the naturalness of color as much as possible. Also, to the best of our knowledge, this is the first web recoloring approach targeted to dichromat people that takes into consideration both text and image recoloring in an integrated manner. The third algorithm (Chapter 5) primarily focuses on the enhancement of some of the object contours in still images, instead of recoloring the pixels of the regions bounded by such contours. Enhancing contours is particularly suited to increase contrast in images, where we find adjacent regions that are color indistinguishable from dichromat’s point of view. To our best knowledge, this is one of the first algorithms that take advantage of image analysis and processing techniques for region contours. After accurate subjective assessment studies for color-blind people, we concluded that the CVD adaptation methods are useful in general. Nevertheless, each method is not efficient enough to adapt all sorts of images, that is, the adequacy of each method depends on the type of image (photo-images, graphical representations, etc.). Furthermore, we noted that the experience-based perceptual learning of colorblind people throughout their lives determines their visual perception. That is, color adaptation algorithms must satisfy requirements such as color naturalness and consistency, to ensure that dichromat people improve their visual perception without artifacts. On the other hand, CVD adaptation algorithms should be object-oriented, instead of pixel-oriented (as typically done), to select judiciously pixels that should be adapted. This perspective opens an opportunity window for future research in color accessibility in the field of in human-computer interaction (HCI).A cor desempenha um papel fundamental na compreensão da informação em ambientes computacionais. Porém, cerca de 5% da população mundial é afetada pela deficiência de visão de cor (ou Color Vision Deficiency (CVD), do Inglês), correntemente designada por daltonismo. Esta insuficiência visual dificulta a perceção das cores, o que limita a perceção geral que os indivíduos têm sobre o meio, seja real ou virtual. Efetivamente, um indivíduo com CVD vê como iguais cores que são diferentes, o que origina confusão ou uma compreensão distorcida da realidade, assim como dos ambientes web, onde existe uma abundância de conteúdos média coloridos, como texto, imagens fixas e vídeo, entre outros. Com o intuito de mitigar as dificuldades que as pessoas com CVD enfrentam na interpretação de conteúdos coloridos, tem sido proposto na literatura um número significativo de algoritmos de recoloração, que têm como o objetivo melhorar, de alguma forma, a perceção visual de pessoas com CVD. Porém, a maioria desses trabalhos carece de um estudo sistemático de avaliação subjetiva, o que põe em causa a sua validação, se não mesmo a sua utilidade. Assim, a principal questão à qual se pretende responder, como resultado do trabalho de investigação subjacente a esta tese de doutoramento, é se os algoritmos de recoloração têm ou não uma real utilidade, constituindo assim uma ajuda efetiva às pessoas com daltonismo. Tendo em mente esta questão, concebemos alguns algoritmos de recoloração preliminares que foram publicados em atas de conferências. Com exceção do algoritmo descrito no Capítulo 3, esses algoritmos não são descritos nesta tese, não obstante a sua importância na conceção daqueles descritos nesta dissertação. O primeiro algoritmo (Capítulo 3) foi projetado e implementado para pessoas com dicromacia, a fim de melhorar a sua perceção da cor. A ideia consiste em projetar as cores de matiz avermelhada em matizes que são melhor percebidos pelas pessoas com os tipos de daltonismo em causa. O segundo algoritmo (Capítulo 4) também se destina a melhorar a perceção da cor por parte de pessoas com dicromacia, porém a sua aplicabilidade abrange a adaptação de texto e imagem, em ambientes web compatíveis com HTML5. Isto é conseguido através do realce do contraste de cores em blocos de texto e em imagens, em páginas da web, mantendo a naturalidade da cor tanto quanto possível. Além disso, tanto quanto sabemos, esta é a primeira abordagem de recoloração em ambiente web para pessoas com dicromacia, que trata o texto e a imagem de forma integrada. O terceiro algoritmo (Capítulo 5) centra-se principalmente na melhoria de alguns dos contornos de objetos em imagens, em vez de aplicar a recoloração aos pixels das regiões delimitadas por esses contornos. Esta abordagem é particularmente adequada para aumentar o contraste em imagens, quando existem regiões adjacentes que são de cor indistinguível sob a perspetiva dos observadores com dicromacia. Também neste caso, e tanto quanto é do nosso conhecimento, este é um dos primeiros algoritmos em que se recorre a técnicas de análise e processamento de contornos de regiões. Após rigorosos estudos de avaliação subjetiva com pessoas com daltonismo, concluiu-se que os métodos de adaptação CVD são úteis em geral. No entanto, cada método não é suficientemente eficiente para todos os tipo de imagens, isto é, o desempenho de cada método depende do tipo de imagem (fotografias, representações gráficas, etc.). Além disso, notámos que a aprendizagem perceptual baseada na experiência das pessoas daltónicas ao longo de suas vidas é determinante para perceber aquilo que vêem. Isto significa que os algoritmos de adaptação de cor devem satisfazer requisitos tais como a naturalidade e a consistência da cor, de modo a não pôr em causa aquilo que os destinatários consideram razoável ver no mundo real. Por outro lado, a abordagem seguida na adaptação CVD deve ser orientada aos objetos, em vez de ser orientada aos pixéis (como tem sido feito até ao momento), de forma a possibilitar uma seleção mais criteriosa dos pixéis que deverão ser sujeitos ao processo de adaptação. Esta perspectiva abre uma janela de oportunidade para futura investigação em acessibilidade da cor no domínio da interacção humano-computador (HCI)

Individual Colorimetric Observers for Personalized Color Imaging

Colors are typically described by three values such as RGB, XYZ, and HSV. This is rooted to the fact that humans possess three types of photoreceptors under photopic conditions, and human color vision can be characterized by a set of three color matching functions (CMFs). CMFs integrate spectra to produce three colorimetric values that are related to visual responses. In reality, large variations in CMFs exist among color-normal populations. Thus, a pair of two spectrally different stimuli might be a match for one person but a mismatch for another person, also known as observer metamerism. Observer metamerism is a serious issue in color-critical applications such as soft proofing in graphic arts and color grading in digital cinema, where colors are compared on different displays. Due to observer metamerism, calibrated displays might not appear correctly, and one person might disagree with color adjustments made by another person. The recent advent of wide color gamut display technologies (e.g., LEDs, OLEDs, lasers, and Quantum Dots) has made observer metamerism even more serious due to their spectrally narrow primaries. The variations among normal color vision and observer metamerism have been overlooked for many years. The current typical color imaging workflow uses a single standard observer assuming all the color-normal people possess the same CMFs. This dissertation provides a possible solution for observer metamerism in color-critical applications by personalized color imaging introducing individual colorimetric observers. In this dissertation, at first, color matching data were collected to derive and validate CMFs for individual colorimetric observers. The data from 151 color-normal observers were obtained at four different locations. Second, two types of individual colorimetric observer functions were derived and validated. One is an individual colorimetric observer model, an extension of the CIE 2006 physiological observer incorporating eight physiological parameters to model individuals in addition to age and field size inputs. The other is a set of categorical observer functions providing a more convenient approach towards the personalized color imaging. Third, two workflows were proposed to characterize human color vision: one using a nomaloscope and the other using proposed spectral pseudoisochromatic images. Finally, the personalized color imaging was evaluated in a color image matching study on an LCD monitor and a laser projector and in a perceived color difference study on a SHARP Quattron display. The personalized color imaging was implemented using a newly introduced ICC profile, iccMAX

A novel approach for multimodal graph dimensionality reduction

This thesis deals with the problem of multimodal dimensionality reduction (DR), which arises when the input objects, to be mapped on a low-dimensional space, consist of multiple vectorial representations, instead of a single one. Herein, the problem is addressed in two alternative manners. One is based on the traditional notion of modality fusion, but using a novel approach to determine the fusion weights. In order to optimally fuse the modalities, the known graph embedding DR framework is extended to multiple modalities by considering a weighted sum of the involved affinity matrices. The weights of the sum are automatically calculated by minimizing an introduced notion of inconsistency of the resulting multimodal affinity matrix. The other manner for dealing with the problem is an approach to consider all modalities simultaneously, without fusing them, which has the advantage of minimal information loss due to fusion. In order to avoid fusion, the problem is viewed as a multi-objective optimization problem. The multiple objective functions are defined based on graph representations of the data, so that their individual minimization leads to dimensionality reduction for each modality separately. The aim is to combine the multiple modalities without the need to assign importance weights to them, or at least postpone such an assignment as a last step. The proposed approaches were experimentally tested in mapping multimedia data on low-dimensional spaces for purposes of visualization, classification and clustering. The no-fusion approach, namely Multi-objective DR, was able to discover mappings revealing the structure of all modalities simultaneously, which cannot be discovered by weight-based fusion methods. However, it results in a set of optimal trade-offs, from which one needs to be selected, which is not trivial. The optimal-fusion approach, namely Multimodal Graph Embedding DR, is able to easily extend unimodal DR methods to multiple modalities, but depends on the limitations of the unimodal DR method used. Both the no-fusion and the optimal-fusion approaches were compared to state-of-the-art multimodal dimensionality reduction methods and the comparison showed performance improvement in visualization, classification and clustering tasks. The proposed approaches were also evaluated for different types of problems and data, in two diverse application fields, a visual-accessibility-enhanced search engine and a visualization tool for mobile network security data. The results verified their applicability in different domains and suggested promising directions for future advancements.Open Acces