Appreciating the Performance of Neuroscience Mining in NeuroIS research: A Case Study on Consumer's Product Perceptions in the Two UI Modes—Dark UI vs. Light UI

The goal of the current study was to provide information on the potential of neuroscience mining (NSM) for comprehending NeuroIS paradigms. NSM is an interdisciplinary field that combines neuroscience and business mining, which is the application of big data analytics, computational social science, and other fields to business problems. Therefore, NSM makes it possible to apply predictive models to NeuroIS datasets, such as machine learning and deep learning, to find intricate patterns that are hidden by conventional regression-based analysis. We predicted 28 individual EEG power spectra separated brainwave data using a Random Forest (RF) model. Next, we used NSM to precisely predict how consumers would perceive a product online, depending on whether a light or dark user interface (UI) mode was being used. The model was then used to extract more precise results that could not be obtained using more conventional linear-based analytical models using sensitivity analysis. The benefits of using NSM in NeuroIS research are as follows: (1) it can relieve the burden of the three-horned dilemma described by Runkel and McGrath; (2) it can enable more temporal data to be directly analyzed on the target variables; and (3) sensitivity analysis can be performed on a condition/individual basis, strengthening the rigor of findings by reducing sample bias that can be lost in grand averaging of data when analyzed with methods like GLM

Kuvanlaatukokemuksen arvionnin instrumentit

This dissertation describes the instruments available for image quality evaluation, develops new methods for subjective image quality evaluation and provides image and video databases for the assessment and development of image quality assessment (IQA) algorithms. The contributions of the thesis are based on six original publications. The first publication introduced the VQone toolbox for subjective image quality evaluation. It created a platform for free-form experimentation with standardized image quality methods and was the foundation for later studies. The second publication focused on the dilemma of reference in subjective experiments by proposing a new method for image quality evaluation: the absolute category rating with dynamic reference (ACR-DR). The third publication presented a database (CID2013) in which 480 images were evaluated by 188 observers using the ACR-DR method proposed in the prior publication. Providing databases of image files along with their quality ratings is essential in the field of IQA algorithm development. The fourth publication introduced a video database (CVD2014) based on having 210 observers rate 234 video clips. The temporal aspect of the stimuli creates peculiar artifacts and degradations, as well as challenges to experimental design and video quality assessment (VQA) algorithms. When the CID2013 and CVD2014 databases were published, most state-of-the-art I/VQAs had been trained on and tested against databases created by degrading an original image or video with a single distortion at a time. The novel aspect of CID2013 and CVD2014 was that they consisted of multiple concurrent distortions. To facilitate communication and understanding among professionals in various fields of image quality as well as among non-professionals, an attribute lexicon of image quality, the image quality wheel, was presented in the fifth publication of this thesis. Reference wheels and terminology lexicons have a long tradition in sensory evaluation contexts, such as taste experience studies, where they are used to facilitate communication among interested stakeholders; however, such an approach has not been common in visual experience domains, especially in studies on image quality. The sixth publication examined how the free descriptions given by the observers influenced the ratings of the images. Understanding how various elements, such as perceived sharpness and naturalness, affect subjective image quality can help to understand the decision-making processes behind image quality evaluation. Knowing the impact of each preferential attribute can then be used for I/VQA algorithm development; certain I/VQA algorithms already incorporate low-level human visual system (HVS) models in their algorithms.Väitöskirja tarkastelee ja kehittää uusia kuvanlaadun arvioinnin menetelmiä, sekä tarjoaa kuva- ja videotietokantoja kuvanlaadun arviointialgoritmien (IQA) testaamiseen ja kehittämiseen. Se, mikä koetaan kauniina ja miellyttävänä, on psykologisesti kiinnostava kysymys. Työllä on myös merkitystä teollisuuteen kameroiden kuvanlaadun kehittämisessä. Väitöskirja sisältää kuusi julkaisua, joissa tarkastellaan aihetta eri näkökulmista. I. julkaisussa kehitettiin sovellus keräämään ihmisten antamia arvioita esitetyistä kuvista tutkijoiden vapaaseen käyttöön. Se antoi mahdollisuuden testata standardoituja kuvanlaadun arviointiin kehitettyjä menetelmiä ja kehittää niiden pohjalta myös uusia menetelmiä luoden perustan myöhemmille tutkimuksille. II. julkaisussa kehitettiin uusi kuvanlaadun arviointimenetelmä. Menetelmä hyödyntää sarjallista kuvien esitystapaa, jolla muodostettiin henkilöille mielikuva kuvien laatuvaihtelusta ennen varsinaista arviointia. Tämän todettiin vähentävän tulosten hajontaa ja erottelevan pienempiä kuvanlaatueroja. III. julkaisussa kuvaillaan tietokanta, jossa on 188 henkilön 480 kuvasta antamat laatuarviot ja niihin liittyvät kuvatiedostot. Tietokannat ovat arvokas työkalu pyrittäessä kehittämään algoritmeja kuvanlaadun automaattiseen arvosteluun. Niitä tarvitaan mm. opetusmateriaalina tekoälyyn pohjautuvien algoritmien kehityksessä sekä vertailtaessa eri algoritmien suorituskykyä toisiinsa. Mitä paremmin algoritmin tuottama ennuste korreloi ihmisten antamiin laatuarvioihin, sen parempi suorituskyky sillä voidaan sanoa olevan. IV. julkaisussa esitellään tietokanta, jossa on 210 henkilön 234 videoleikkeestä tekemät laatuarviot ja niihin liittyvät videotiedostot. Ajallisen ulottuvuuden vuoksi videoärsykkeiden virheet ovat erilaisia kuin kuvissa, mikä tuo omat haasteensa videoiden laatua arvioiville algoritmeille (VQA). Aikaisempien tietokantojen ärsykkeet on muodostettu esimerkiksi sumentamalla yksittäistä kuvaa asteittain, jolloin ne sisältävät vain yksiulotteisia vääristymiä. Nyt esitetyt tietokannat poikkeavat aikaisemmista ja sisältävät useita samanaikaisia vääristymistä, joiden interaktio kuvanlaadulle voi olla merkittävää. V. julkaisussa esitellään kuvanlaatuympyrä (image quality wheel). Se on kuvanlaadun käsitteiden sanasto, joka on kerätty analysoimalla 146 henkilön tuottamat 39 415 kuvanlaadun sanallista kuvausta. Sanastoilla on pitkät perinteet aistinvaraisen arvioinnin tutkimusperinteessä, mutta niitä ei ole aikaisemmin kehitetty kuvanlaadulle. VI. tutkimuksessa tutkittiin, kuinka arvioitsijoiden antamat käsitteet vaikuttavat kuvien laadun arviointiin. Esimerkiksi kuvien arvioitu terävyys tai luonnollisuus auttaa ymmärtämään laadunarvioinnin taustalla olevia päätöksentekoprosesseja. Tietoa voidaan käyttää esimerkiksi kuvan- ja videonlaadun arviointialgoritmien (I/VQA) kehitystyössä

Image Sampling with Quasicrystals

We investigate the use of quasicrystals in image sampling. Quasicrystals produce space-filling, non-periodic point sets that are uniformly discrete and relatively dense, thereby ensuring the sample sites are evenly spread out throughout the sampled image. Their self-similar structure can be attractive for creating sampling patterns endowed with a decorative symmetry. We present a brief general overview of the algebraic theory of cut-and-project quasicrystals based on the geometry of the golden ratio. To assess the practical utility of quasicrystal sampling, we evaluate the visual effects of a variety of non-adaptive image sampling strategies on photorealistic image reconstruction and non-photorealistic image rendering used in multiresolution image representations. For computer visualization of point sets used in image sampling, we introduce a mosaic rendering technique.Comment: For a full resolution version of this paper, along with supplementary materials, please visit at http://www.Eyemaginary.com/Portfolio/Publications.htm

Passive visual behavior modifiers and consumer psychophysiology online.

Through an examination of the electroencephalography (EEG) data collected from 27 university students, this study examined the efficacy of three known passive visual behavior modifiers -- color, layout, and motion -- in an e-commerce environment. These three variables have significant scholarly support in the context of traditional media, but their effect online is still largely unsubstantiated. Using EEG readings taken from regions of interest Fp1 and Fp2, the researcher attempted to measure and compare sustained evoked response upon exposure to six fictitious e-commerce web pages, each exhibiting a different passive visual behavior modifier. It was hypothesized that (H1) a product in a subtle state of motion, (H2) a greater proportion of image to text, and (H3) a color system with a dominant wavelength of approximately 650nm would evoke higher average levels of amplitude (μV) and frequency (Hz) in the ventromedial prefrontal cortex compared to stimuli exhibiting inverse properties: a static product image, a greater proportion of text to image, and a color system with a dominant wavelength of approximately 490nm. The biofeedback measurement was supplemented by a qualitative interview. Participant responses were analyzed for key words, phrases, and trends related to consumer attitude and product preference. While no significant differences were found between the visual stimuli, this study provides insight, limitations, and direction for future psychophysiological research relating to e-commerce.--Abstract

Dataremix: Aesthetic Experiences of Big Data and Data Abstraction

This PhD by published work expands on the contribution to knowledge in two recent large-scale transdisciplinary artistic research projects: ATLAS in silico and INSTRUMENT | One Antarctic Night and their exhibited and published outputs. The thesis reflects upon this practice-based artistic research that interrogates data abstraction: the digitization, datafication and abstraction of culture and nature, as vast and abstract digital data. The research is situated in digital arts practices that engage a combination of big (scientific) data as artistic material, embodied interaction in virtual environments, and poetic recombination. A transdisciplinary and collaborative artistic practice, x-resonance, provides a framework for the hybrid processes, outcomes, and contributions to knowledge from the research. These are purposefully and productively situated at the objective | subjective interface, have potential to convey multiple meanings simultaneously to a variety of audiences and resist disciplinary definition. In the course of the research, a novel methodology emerges, dataremix, which is employed and iteratively evolved through artistic practice to address the research questions: 1) How can a visceral and poetic experience of data abstraction be created? and 2) How would one go about generating an artistically-informed (scientific) discovery? Several interconnected contributions to knowledge arise through the first research question: creation of representational elements for artistic visualization of big (scientific) data that includes four new forms (genomic calligraphy, algorithmic objects as natural specimens, scalable auditory data signatures, and signal objects); an aesthetic of slowness that contributes an extension to the operative forces in Jevbratt’s inverted sublime of looking down and in to also include looking fast and slow; an extension of Corby’s objective and subjective image consisting of “informational and aesthetic components” to novel virtual environments created from big 3 (scientific) data that extend Davies’ poetic virtual spatiality to poetic objective | subjective generative virtual spaces; and an extension of Seaman’s embodied interactive recombinant poetics through embodied interaction in virtual environments as a recapitulation of scientific (objective) and algorithmic processes through aesthetic (subjective) physical gestures. These contributions holistically combine in the artworks ATLAS in silico and INSTRUMENT | One Antarctic Night to create visceral poetic experiences of big data abstraction. Contributions to knowledge from the first research question develop artworks that are visceral and poetic experiences of data abstraction, and which manifest the objective | subjective through art. Contributions to knowledge from the second research question occur through the process of the artworks functioning as experimental systems in which experiments using analytical tools from the scientific domain are enacted within the process of creation of the artwork. The results are “returned” into the artwork. These contributions are: elucidating differences in DNA helix bending and curvature along regions of gene sequences specified as either introns or exons, revealing nuanced differences in BLAST results in relation to genomics sequence metadata, and cross-correlation of astronomical data to identify putative variable signals from astronomical objects for further scientific evaluation

Active sampling, scaling and dataset merging for large-scale image quality assessment

The field of subjective assessment is concerned with eliciting human judgements about a set of stimuli. Collecting such data is costly and time-consuming, especially when the subjective study is to be conducted in a controlled environment and using a specialized equipment. Thus, data from these studies are usually scarce. One of the areas, for which obtaining subjective measurements is difficult is image quality assessment. The results from these studies are used to develop and train automated or objective image quality metrics, which, with the advent of deep learning, require large amounts of versatile and heterogeneous data. I present three main contributions in this dissertation. First, I propose a new active sampling method for efficient collection of pairwise comparisons in subjective assessment experiments. In these experiments observers are asked to express a preference between two conditions. However, many pairwise comparison protocols require a large number of comparisons to infer accurate scores, which may be unfeasible when each comparison is time-consuming (e.g. videos) or expensive (e.g. medical imaging). This motivates the use of an active sampling algorithm that chooses only the most informative pairs for comparison. I demonstrate, with real and synthetic data, that my algorithm offers the highest accuracy of inferred scores given a fixed number of measurements compared to the existing methods. Second, I propose a probabilistic framework to fuse the outcomes of different psychophysical experimental protocols, namely rating and pairwise comparisons experiments. Such a method can be used for merging existing datasets of subjective nature and for experiments in which both measurements are collected. Third, with a new dataset merging technique and by collecting additional cross-dataset quality comparisons I create a Unified Photometric Image Quality (UPIQ) dataset with over 4,000 images by realigning and merging existing high-dynamic-range (HDR) and standard-dynamic-range (SDR) datasets. The realigned quality scores share the same unified quality scale across all datasets. I then use the new dataset to retrain existing HDR metrics and show that the dataset is sufficiently large for training deep architectures. I show the utility of the dataset and metrics in an application to image compression that accounts for viewing conditions, including screen brightness and the viewing distance

트리 구조를 이용한 3차원 공간 내 데이터 시각화 연구

학위논문 (박사)-- 서울대학교 대학원 : 미술대학 디자인학부 디자인전공, 2019. 2. 김수정.Speculative visualization combines both data visualization methods and aesthetics to draw attention to specific social, political and environmental issues. The speculative data visualization project proposed in this work explores electronic waste trade and the environmental performance of various nations. Illegal trading of electronic waste without proper disposal and recycling measures has a severe impact on both human health and the environment. This trade can be represented as a network data structure. The overall environmental health and ecosystem vitality of those trading countries, represented by their Environmental Performance Index (EPI), can also give greater insight into this issue. This EPI data has a hierarchical structure. This work explores methods to visualize these two data sets simultaneously in a manner that allows for analytical exploration of the data while communicating its underlying meaning. This project-based design research specifically focuses on visualizing hierarchical datasets with a node-link type tree structure and suggests a novel data visualization method, called the data garden, to visualize these hierarchical datasets within a spatial network. This draws inspiration from networks found between trees in nature. This is applied to the illegal e-waste trade and environmental datasets to provoke discussion, provide a holistic understanding and improve the peoples awareness on these issues. This uses both analytical data visualization techniques, along with a more aesthetic approach. The data garden approach is used to create a 3D interactive data visualization that users can use to navigate and explore the data in a meaningful way while also providing an emotional connection to the subject. This is due to the ability of the data garden approach to accurately show the underlying data while also closely mimicking natural structures. The visualization project intends to encourage creative professionals to create both visually appealing and thought-provoking data visualizations on significant issues that can reach a mass audience and improve awareness of citizens. Additionally, this design research intends to cause further discussion on the role of aesthetics and creative practices in data visualizations.사변적 시각화(speculative visualization)는 데이터 시각화 방법과 미학을 결합하여 특정한 사회, 정치 및 환경 문제에 관심을 유도하는 것입니다. 제안한 사변적 데이터 시각화 프로젝트를 통해 다양한 국가의 전자 폐기물 거래와 환경 성과를 살펴봅니다. 적절한 처리와 재활용 조치가 이뤄지지 않은 전자폐기물의 불법 거래는 환경과 인간에 심각한 영향을 미칩니다. 이 거래는 네트워크 데이터 구조로 표현할 수 있습니다. 환경성과지수(EPI)를 통해 이 거래에 참여하는 국가들의 전반적인 환경 보건과 생태계 활력을 살펴보는 것은 이 문제에 더 깊은 통찰력을 제공할 수 있습니다. 이 환경성과지수는 계층 구조로 되어 있습니다. 이 연구는 데이터를 분석적으로 탐구할 수 있도록 하는 방법을 통해 두 가지 데이터를 동시에 시각화하고, 이를 통해 표면에 드러나지 않는 데이터의 의미를 전달하는 방법을 탐구합니다. 본 연구는 프로젝트를 기반으로 하는 디자인 연구로, 노드 링크 유형 트리 구조를 통해 계층적 데이터를 시각화하는 것에 중점을 두고 있습니다. 자연에서 발견할 수 있는 나무 간 네트워크에서 영감을 얻어 공간 네트워크에서 계층적 데이터 세트를 시각화합니다. 데이터 정원이라고 하는 이 새로운 데이터 시각화 방법을 불법 전자 폐기물 거래와 환경 데이터에 적용하여 토론을 유발하고 전체적인 이해를 제공하며 이러한 문제에 대한 사람들의 인식을 개선하고자 합니다. 이는 보다 미적인 접근과 분석적 데이터 시각화 기술을 모두 사용합니다. 데이터 정원을 통한 접근으로 삼차원 대화형 데이터 시각화를 만들 수 있습니다. 이 시각화를 통해 사용자는 데이터를 의미 있는 방식으로 살펴보는 동시에 주제와 감성적인 연결을 받을 수 있습니다. 이는 데이터 정원 방법이 데이터를 정확하게 보여주는 동시에 자연 구조를 면밀하게 모방하기 때문입니다. 본 시각화 프로젝트는 창의적인 전문가들이 중요한 문제에 대해 시각적으로 매력적이고 생각을 자극하는 데이터 시각화를 만들어 대중에게 도달하고 시민들의 인식을 향상할 수 있도록 권장합니다. 또한, 본 디자인 연구는 데이터 시각화에서 미학과 창조적인 실천의 역할에 대한 더 많은 논의를 유도하고자 합니다.Abstract I Table of Contents III List of Figures VI 1. Introduction 1 1.1 Research Background 2 1.2 Research Goal and Method 6 1.3 Terminology 9 2. Hierarchical Relationships: Trees 14 2.1 The History of Tree Diagrams 16 2.1.1 Significance of Trees 16 2.1.2 Aristotles Hierarchical Order of Life 19 2.1.3 Early Religious Depictions of Hierarchical Structures 22 2.1.4 Depicting Evolution 26 2.2 Tree Structures 29 2.3 Tree Layouts 31 3. Complex Relationships: Networks 34 3.1 Attributes of Networks 36 3.1.1 Interdependence and Interconnectedness 38 3.1.2 Decentralization 42 3.1.3 Nonlinearity 45 3.1.4 Multiplicity 46 3.2 Spatial Networks 46 3.3 Combining Tree Structures and Networks 48 4. Design Study Goals and Criteria 51 4.1 Objectives of the Design Study 71 4.2 Data Visualization Approaches 54 4.3 Criteria of Data Visualization 57 4.3.1 Aesthetics 58 4.3.2 Information Visualization Principles 62 4.3.2.1 Visual Cues in Data Visualization 62 4.3.2.2 Gestalt Principles 65 4.3.2.3 Increasing Efficiency of Network Visualizations 67 4.4 Case Study 70 5. Design Study: Data Garden Method 78 5.1 Concept of the Data Garden Structure 79 5.2 Data Garden Tree Structure 84 5.2.1 360°Vertical Branches 85 5.2.2 Break Point of the Branches 87 5.2.3 Aligning Hierarchy Levels 89 5.2.3.1 Design 01 – Extend Method 90 5.2.3.2 Design 02 – Collapse Method 91 5.2.4 Node Placement Technique 92 5.3 Conveying 3D Information 95 6. Design Study: Visualization Project 98 6.1 Theme 99 6.1.1 E-waste Trade 100 6.1.2 Environmental Performance Index 102 6.2 Visual Design Concept 104 6.3 Assigning Attributes 105 6.4 Visual Design Process 107 6.4.1 Leaf (Node) Design Process 107 6.4.1.1 Leaf Inspiration 107 6.4.1.2 Leaf Design 108 6.4.1.3 Leaf Area Calculation and Alignment 113 6.4.2 Stem (Branch) Design Process 116 6.4.3 Root (Link) Design Process 117 6.5 Interaction Design 118 6.5.1 Navigation 118 6.5.2 User Interface 119 6.5.3 Free and Detail Modes 120 6.5.4 Data Details 121 6.6 Visualization Renders 122 6.7 Exhibition 129 7. Conclusion 131 7.1 Conclusion 132 7.2 Limitations and Further Research 133 Bibliography 135 국문초록 (Abstract in Korean) 144Docto