From Capture to Display: A Survey on Volumetric Video

Volumetric video, which offers immersive viewing experiences, is gaining increasing prominence. With its six degrees of freedom, it provides viewers with greater immersion and interactivity compared to traditional videos. Despite their potential, volumetric video services poses significant challenges. This survey conducts a comprehensive review of the existing literature on volumetric video. We firstly provide a general framework of volumetric video services, followed by a discussion on prerequisites for volumetric video, encompassing representations, open datasets, and quality assessment metrics. Then we delve into the current methodologies for each stage of the volumetric video service pipeline, detailing capturing, compression, transmission, rendering, and display techniques. Lastly, we explore various applications enabled by this pioneering technology and we present an array of research challenges and opportunities in the domain of volumetric video services. This survey aspires to provide a holistic understanding of this burgeoning field and shed light on potential future research trajectories, aiming to bring the vision of volumetric video to fruition.Comment: Submitte

ISMCR 1994: Topical Workshop on Virtual Reality. Proceedings of the Fourth International Symposium on Measurement and Control in Robotics

This symposium on measurement and control in robotics included sessions on: (1) rendering, including tactile perception and applied virtual reality; (2) applications in simulated medical procedures and telerobotics; (3) tracking sensors in a virtual environment; (4) displays for virtual reality applications; (5) sensory feedback including a virtual environment application with partial gravity simulation; and (6) applications in education, entertainment, technical writing, and animation

Ontology-Driven, Guided Visualisation Supporting Explicit and Composable Mappings

Data masses on the World Wide Web can hardly be managed by humans or machines. One option is the formal description and linking of data sources using Semantic Web and Linked Data technologies. Ontologies written in standardised languages foster the sharing and linking of data as they provide a means to formally define concepts and relations between these concepts. A second option is visualisation. The visual representation allows humans to perceive information more directly, using the highly developed visual sense. Relatively few efforts have been made on combining both options, although the formality and rich semantics of ontological data make it an ideal candidate for visualisation. Advanced visualisation design systems support the visualisation of tabular, typically statistical data. However, visualisations of ontological data still have to be created manually, since automated solutions are often limited to generic lists or node-link diagrams. Also, the semantics of ontological data are not exploited for guiding users through visualisation tasks. Finally, once a good visualisation setting has been created, it cannot easily be reused and shared. Trying to tackle these problems, we had to answer how to define composable and shareable mappings from ontological data to visual means and how to guide the visual mapping of ontological data. We present an approach that allows for the guided visualisation of ontological data, the creation of effective graphics and the reuse of visualisation settings. Instead of generic graphics, we aim at tailor-made graphics, produced using the whole palette of visual means in a flexible, bottom-up approach. It not only allows for visualising ontologies, but uses ontologies to guide users when visualising data and to drive the visualisation process at various places: First, as a rich source of information on data characteristics, second, as a means to formally describe the vocabulary for building abstract graphics, and third, as a knowledge base of facts on visualisation. This is why we call our approach ontology-driven. We suggest generating an Abstract Visual Model (AVM) to represent and »synthesise« a graphic following a role-based approach, inspired by the one used by J. v. Engelhardt for the analysis of graphics. It consists of graphic objects and relations formalised in the Visualisation Ontology (VISO). A mappings model, based on the declarative RDFS/OWL Visualisation Language (RVL), determines a set of transformations from the domain data to the AVM. RVL allows for composable visual mappings that can be shared and reused across platforms. To guide the user, for example, we discourage the construction of mappings that are suboptimal according to an effectiveness ranking formalised in the fact base and suggest more effective mappings instead. The guidance process is flexible, since it is based on exchangeable rules. VISO, RVL and the AVM are additional contributions of this thesis. Further, we initially analysed the state of the art in visualisation and RDF-presentation comparing 10 approaches by 29 criteria. Our approach is unique because it combines ontology-driven guidance with composable visual mappings. Finally, we compare three prototypes covering the essential parts of our approach to show its feasibility. We show how the mapping process can be supported by tools displaying warning messages for non-optimal visual mappings, e.g., by considering relation characteristics such as »symmetry«. In a constructive evaluation, we challenge both the RVL language and the latest prototype trying to regenerate sketches of graphics we created manually during analysis. We demonstrate how graphics can be varied and complex mappings can be composed from simple ones. Two thirds of the sketches can be almost or completely specified and half of them can be almost or completely implemented.:Legend and Overview of Prefixes xiii 1 Introduction 1 2 Background 11 2.1 Visualisation 11 2.1.1 What is Visualisation? 11 2.1.2 What are the Benefits of Visualisation? 12 2.1.3 Visualisation Related Terms Used in this Thesis 12 2.1.4 Visualisation Models and Architectural Patterns 12 2.1.5 Visualisation Design Systems 14 2.1.6 What is the Difference between Visual Mapping and Styling? 14 2.1.7 Lessons Learned from Style Sheet Languages 15 2.2 Data 16 2.2.1 Data – Information – Knowledge 17 2.2.2 Structured Data 17 2.2.3 Ontologies in Computer Science 19 2.2.4 The Semantic Web and its Languages 19 2.2.5 Linked Data and Open Data 20 2.2.6 The Metamodelling Technological Space 21 2.2.7 SPIN 21 2.3 Guidance 22 2.3.1 Guidance in Visualisation 22 3 Problem Analysis 23 3.1 Problems of Ontology Visualisation Approaches 24 3.2 Research Questions 25 3.3 Set up of the Case Studies 25 3.3.1 Case Studies in the Life Sciences Domain 26 3.3.2 Case Studies in the Publishing Domain 26 3.3.3 Case Studies in the Software Technology Domain 27 3.4 Analysis of the Case Studies’ Ontologies 27 3.5 Manual Sketching of Graphics 29 3.6 Analysis of the Graphics for Typical Visualisation Cases 29 3.7 Requirements 33 3.7.1 Requirements for Visualisation and Interaction 34 3.7.2 Requirements for Data Awareness 34 3.7.3 Requirements for Reuse and Composition 34 3.7.4 Requirements for Variability 35 3.7.5 Requirements for Tooling Support and Guidance 35 3.7.6 Optional Features and Limitations 36 4 Analysis of the State of the Art 37 4.1 Related Visualisation Approaches 38 4.1.1 Short Overview of the Approaches 38 4.1.2 Detailed Comparison by Criteria 46 4.1.3 Conclusion – What Is Still Missing? 60 4.2 Visualisation Languages 62 4.2.1 Short Overview of the Compared Languages 62 4.2.2 Detailed Comparison by Language Criteria 66 4.2.3 Conclusion – What Is Still Missing? 71 4.3 RDF Presentation Languages 72 4.3.1 Short Overview of the Compared Languages 72 4.3.2 Detailed Comparison by Language Criteria 76 4.3.3 Additional Criteria for RDF Display Languages 87 4.3.4 Conclusion – What Is Still Missing? 89 4.4 Model-Driven Interfaces 90 4.4.1 Metamodel-Driven Interfaces 90 4.4.2 Ontology-Driven Interfaces 92 4.4.3 Combined Usage of the Metamodelling and Ontology Technological Space 94 5 A Visualisation Ontology – VISO 97 5.1 Methodology Used for Ontology Creation 100 5.2 Requirements for a Visualisation Ontology 100 5.3 Existing Approaches to Modelling in the Field of Visualisation 101 5.3.1 Terminologies and Taxonomies 101 5.3.2 Existing Visualisation Ontologies 102 5.3.3 Other Visualisation Models and Approaches to Formalisation 103 5.3.4 Summary 103 5.4 Technical Aspects of VISO 103 5.5 VISO/graphic Module – Graphic Vocabulary 104 5.5.1 Graphic Representations and Graphic Objects 105 5.5.2 Graphic Relations and Syntactic Structures 107 5.6 VISO/data Module – Characterising Data 110 5.6.1 Data Structure and Characteristics of Relations 110 5.6.2 The Scale of Measurement and Units 112 5.6.3 Properties for Characterising Data Variables in Statistical Data 113 5.7 VISO/facts Module – Facts for Vis. Constraints and Rules 115 5.7.1 Expressiveness of Graphic Relations 116 5.7.2 Effectiveness Ranking of Graphic Relations 118 5.7.3 Rules for Composing Graphics 119 5.7.4 Other Rules to Consider for Visual Mapping 124 5.7.5 Providing Named Value Collections 124 5.7.6 Existing Approaches to the Formalisation of Visualisation Knowledge . . 126 5.7.7 The VISO/facts/empiric Example Knowledge Base 126 5.8 Other VISO Modules 126 5.9 Conclusions and Future Work 127 5.10 Further Use Cases for VISO 127 5.11 VISO on the Web – Sharing the Vocabulary to Build a Community 128 6 A VISO-Based Abstract Visual Model – AVM 129 6.1 Graphical Notation Used in this Chapter 129 6.2 Elementary Graphic Objects and Graphic Attributes 131 6.3 N-Ary Relations 131 6.4 Binary Relations 131 6.5 Composition of Graphic Objects Using Roles 132 6.6 Composition of Graphic Relations Using Roles 132 6.7 Composition of Visual Mappings Using the AVM 135 6.8 Tracing 135 6.9 Is it Worth Having an Abstract Visual Model? 135 6.10 Discussion of Fresnel as a Related Language 137 6.11 Related Work 139 6.12 Limitations 139 6.13 Conclusions 140 7 A Language for RDFS/OWL Visualisation – RVL 141 7.1 Language Requirements 142 7.2 Main RVL Constructs 145 7.2.1 Mapping 145 7.2.2 Property Mapping 146 7.2.3 Identity Mapping 146 7.2.4 Value Mapping 147 7.2.5 Inheriting RVL Settings 147 7.2.6 Resource Mapping 148 7.2.7 Simplifications 149 7.3 Calculating Value Mappings 150 7.4 Defining Scale of Measurement 153 7.4.1 Determining the Scale of Measurement 154 7.5 Addressing Values in Value Mappings 156 7.5.1 Determining the Set of Addressed Source Values 156 7.5.2 Determining the Set of Addressed Target Values 157 7.6 Overlapping Value Mappings 158 7.7 Default Value Mapping 158 7.8 Default Labelling 159 7.9 Defining Interaction 159 7.10 Mapping Composition and Submappings 160 7.11 A Schema Language for RVL 160 7.11.1 Concrete Examples of the RVL Schema 163 7.12 Conclusions and Future Work 166 8 The OGVIC Approach 169 8.1 Ontology-Driven, Guided Editing of Visual Mappings 172 8.1.1 Classification of Constraints 172 8.1.2 Levels of Guidance 173 8.1.3 Implementing Constraint-Based Guidance 173 8.2 Support of Explicit and Composable Visual Mappings 177 8.2.1 Mapping Composition Cases 178 8.2.2 Selecting a Context 180 8.2.3 Using the Same Graphic Relation Multiple Times 181 8.3 Prototype P1 (TopBraid-Composer-based) 182 8.4 Prototype P2 (OntoWiki-based) 184 8.5 Prototype P3 (Java Implementation of RVL) 187 8.6 Lessons Learned from Prototypes & Future Work 190 8.6.1 Checking RVL Constraints and Visualisation Rules 190 8.6.2 A User Interface for Editing RVL Mappings 190 8.6.3 Graph Transformations with SPIN and SPARQL 1.1 Update 192 8.6.4 Selection and Filtering of Data 193 8.6.5 Interactivity and Incremental Processing 193 8.6.6 Rendering the Final Platform-Specific Code 196 9 Application 197 9.1 Coverage of Case Study Sketches and Necessary Features 198 9.2 Coverage of Visualisation Cases 201 9.3 Coverage of Requirements 205 9.4 Full Example 206 10 Conclusions 211 10.1 Contributions 211 10.2 Constructive Evaluation 212 10.3 Research Questions 213 10.4 Transfer to Other Models and Constraint Languages 213 10.5 Limitations 214 10.6 Future Work 214 Appendices 217 A Case Study Sketches 219 B VISO – Comparison of Visualisation Literature 229 C RVL 231 D RVL Example Mappings and Application 233 D.1 Listings of RVL Example Mappings as Required by Prototype P3 233 D.2 Features Required for Implementing all Sketches 235 D.3 JSON Format for Processing the AVM with D3 – Hierarchical Variant 238 Bibliography 238 List of Figures 251 List of Tables 254 List of Listings 257Datenmassen im World Wide Web können kaum von Menschen oder Maschinen erfasst werden. Eine Option ist die formale Beschreibung und Verknüpfung von Datenquellen mit Semantic-Web- und Linked-Data-Technologien. Ontologien, in standardisierten Sprachen geschrieben, befördern das Teilen und Verknüpfen von Daten, da sie ein Mittel zur formalen Definition von Konzepten und Beziehungen zwischen diesen Konzepten darstellen. Eine zweite Option ist die Visualisierung. Die visuelle Repräsentation ermöglicht es dem Menschen, Informationen direkter wahrzunehmen, indem er seinen hochentwickelten Sehsinn verwendet. Relativ wenige Anstrengungen wurden unternommen, um beide Optionen zu kombinieren, obwohl die Formalität und die reichhaltige Semantik ontologische Daten zu einem idealen Kandidaten für die Visualisierung machen. Visualisierungsdesignsysteme unterstützen Nutzer bei der Visualisierung von tabellarischen, typischerweise statistischen Daten. Visualisierungen ontologischer Daten jedoch müssen noch manuell erstellt werden, da automatisierte Lösungen häufig auf generische Listendarstellungen oder Knoten-Kanten-Diagramme beschränkt sind. Auch die Semantik der ontologischen Daten wird nicht ausgenutzt, um Benutzer durch Visualisierungsaufgaben zu führen. Einmal erstellte Visualisierungseinstellungen können nicht einfach wiederverwendet und geteilt werden. Um diese Probleme zu lösen, mussten wir eine Antwort darauf finden, wie die Definition komponierbarer und wiederverwendbarer Abbildungen von ontologischen Daten auf visuelle Mittel geschehen könnte und wie Nutzer bei dieser Abbildung geführt werden könnten. Wir stellen einen Ansatz vor, der die geführte Visualisierung von ontologischen Daten, die Erstellung effektiver Grafiken und die Wiederverwendung von Visualisierungseinstellungen ermöglicht. Statt auf generische Grafiken zielt der Ansatz auf maßgeschneiderte Grafiken ab, die mit der gesamten Palette visueller Mittel in einem flexiblen Bottom-Up-Ansatz erstellt werden. Er erlaubt nicht nur die Visualisierung von Ontologien, sondern verwendet auch Ontologien, um Benutzer bei der Visualisierung von Daten zu führen und den Visualisierungsprozess an verschiedenen Stellen zu steuern: Erstens als eine reichhaltige Informationsquelle zu Datencharakteristiken, zweitens als Mittel zur formalen Beschreibung des Vokabulars für den Aufbau von abstrakten Grafiken und drittens als Wissensbasis von Visualisierungsfakten. Deshalb nennen wir unseren Ansatz ontologie-getrieben. Wir schlagen vor, ein Abstract Visual Model (AVM) zu generieren, um eine Grafik rollenbasiert zu synthetisieren, angelehnt an einen Ansatz der von J. v. Engelhardt verwendet wird, um Grafiken zu analysieren. Das AVM besteht aus grafischen Objekten und Relationen, die in der Visualisation Ontology (VISO) formalisiert sind. Ein Mapping-Modell, das auf der deklarativen RDFS/OWL Visualisation Language (RVL) basiert, bestimmt eine Menge von Transformationen von den Quelldaten zum AVM. RVL ermöglicht zusammensetzbare »Mappings«, visuelle Abbildungen, die über Plattformen hinweg geteilt und wiederverwendet werden können. Um den Benutzer zu führen, bewerten wir Mappings anhand eines in der Faktenbasis formalisierten Effektivitätsrankings und schlagen ggf. effektivere Mappings vor. Der Beratungsprozess ist flexibel, da er auf austauschbaren Regeln basiert. VISO, RVL und das AVM sind weitere Beiträge dieser Arbeit. Darüber hinaus analysieren wir zunächst den Stand der Technik in der Visualisierung und RDF-Präsentation, indem wir 10 Ansätze nach 29 Kriterien vergleichen. Unser Ansatz ist einzigartig, da er eine ontologie-getriebene Nutzerführung mit komponierbaren visuellen Mappings vereint. Schließlich vergleichen wir drei Prototypen, welche die wesentlichen Teile unseres Ansatzes umsetzen, um seine Machbarkeit zu zeigen. Wir zeigen, wie der Mapping-Prozess durch Tools unterstützt werden kann, die Warnmeldungen für nicht optimale visuelle Abbildungen anzeigen, z. B. durch Berücksichtigung von Charakteristiken der Relationen wie »Symmetrie«. In einer konstruktiven Evaluation fordern wir sowohl die RVL-Sprache als auch den neuesten Prototyp heraus, indem wir versuchen Skizzen von Grafiken umzusetzen, die wir während der Analyse manuell erstellt haben. Wir zeigen, wie Grafiken variiert werden können und komplexe Mappings aus einfachen zusammengesetzt werden können. Zwei Drittel der Skizzen können fast vollständig oder vollständig spezifiziert werden und die Hälfte kann fast vollständig oder vollständig umgesetzt werden.:Legend and Overview of Prefixes xiii 1 Introduction 1 2 Background 11 2.1 Visualisation 11 2.1.1 What is Visualisation? 11 2.1.2 What are the Benefits of Visualisation? 12 2.1.3 Visualisation Related Terms Used in this Thesis 12 2.1.4 Visualisation Models and Architectural Patterns 12 2.1.5 Visualisation Design Systems 14 2.1.6 What is the Difference between Visual Mapping and Styling? 14 2.1.7 Lessons Learned from Style Sheet Languages 15 2.2 Data 16 2.2.1 Data – Information – Knowledge 17 2.2.2 Structured Data 17 2.2.3 Ontologies in Computer Science 19 2.2.4 The Semantic Web and its Languages 19 2.2.5 Linked Data and Open Data 20 2.2.6 The Metamodelling Technological Space 21 2.2.7 SPIN 21 2.3 Guidance 22 2.3.1 Guidance in Visualisation 22 3 Problem Analysis 23 3.1 Problems of Ontology Visualisation Approaches 24 3.2 Research Questions 25 3.3 Set up of the Case Studies 25 3.3.1 Case Studies in the Life Sciences Domain 26 3.3.2 Case Studies in the Publishing Domain 26 3.3.3 Case Studies in the Software Technology Domain 27 3.4 Analysis of the Case Studies’ Ontologies 27 3.5 Manual Sketching of Graphics 29 3.6 Analysis of the Graphics for Typical Visualisation Cases 29 3.7 Requirements 33 3.7.1 Requirements for Visualisation and Interaction 34 3.7.2 Requirements for Data Awareness 34 3.7.3 Requirements for Reuse and Composition 34 3.7.4 Requirements for Variability 35 3.7.5 Requirements for Tooling Support and Guidance 35 3.7.6 Optional Features and Limitations 36 4 Analysis of the State of the Art 37 4.1 Related Visualisation Approaches 38 4.1.1 Short Overview of the Approaches 38 4.1.2 Detailed Comparison by Criteria 46 4.1.3 Conclusion – What Is Still Missing? 60 4.2 Visualisation Languages 62 4.2.1 Short Overview of the Compared Languages 62 4.2.2 Detailed Comparison by Language Criteria 66 4.2.3 Conclusion – What Is Still Missing? 71 4.3 RDF Presentation Languages 72 4.3.1 Short Overview of the Compared Languages 72 4.3.2 Detailed Comparison by Language Criteria 76 4.3.3 Additional Criteria for RDF Display Languages 87 4.3.4 Conclusion – What Is Still Missing? 89 4.4 Model-Driven Interfaces 90 4.4.1 Metamodel-Driven Interfaces 90 4.4.2 Ontology-Driven Interfaces 92 4.4.3 Combined Usage of the Metamodelling and Ontology Technological Space 94 5 A Visualisation Ontology – VISO 97 5.1 Methodology Used for Ontology Creation 100 5.2 Requirements for a Visualisation Ontology 100 5.3 Existing Approaches to Modelling in the Field of Visualisation 101 5.3.1 Terminologies and Taxonomies 101 5.3.2 Existing Visualisation Ontologies 102 5.3.3 Other Visualisation Models and Approaches to Formalisation 103 5.3.4 Summary 103 5.4 Technical Aspects of VISO 103 5.5 VISO/graphic Module – Graphic Vocabulary 104 5.5.1 Graphic Representations and Graphic Objects 105 5.5.2 Graphic Relations and Syntactic Structures 107 5.6 VISO/data Module – Characterising Data 110 5.6.1 Data Structure and Characteristics of Relations 110 5.6.2 The Scale of Measurement and Units 112 5.6.3 Properties for Characterising Data Variables in Statistical Data 113 5.7 VISO/facts Module – Facts for Vis. Constraints and Rules 115 5.7.1 Expressiveness of Graphic Relations 116 5.7.2 Effectiveness Ranking of Graphic Relations 118 5.7.3 Rules for Composing Graphics 119 5.7.4 Other Rules to Consider for Visual Mapping 124 5.7.5 Providing Named Value Collections 124 5.7.6 Existing Approaches to the Formalisation of Visualisation Knowledge . . 126 5.7.7 The VISO/facts/empiric Example Knowledge Base 126 5.8 Other VISO Modules 126 5.9 Conclusions and Future Work 127 5.10 Further Use Cases for VISO 127 5.11 VISO on the Web – Sharing the Vocabulary to Build a Community 128 6 A VISO-Based Abstract Visual Model – AVM 129 6.1 Graphical Notation Used in this Chapter 129 6.2 Elementary Graphic Objects and Graphic Attributes 131 6.3 N-Ary Relations 131 6.4 Binary Relations 131 6.5 Composition of Graphic Objects Using Roles 132 6.6 Composition of Graphic Relations Using Roles 132 6.7 Composition of Visual Mappings Using the AVM 135 6.8 Tracing 135 6.9 Is it Worth Having an Abstract Visual Model? 135 6.10 Discussion of Fresnel as a Related Language 137 6.11 Related Work 139 6.12 Limitations 139 6.13 Conclusions 140 7 A Language for RDFS/OWL Visualisation – RVL 141 7.1 Language Requirements 142 7.2 Main RVL Constructs 145 7.2.1 Mapping 145 7.2.2 Property Mapping 146 7.2.3 Identity Mapping 146 7.2.4 Value Mapping 147 7.2.5 Inheriting RVL Settings 147 7.2.6 Resource Mapping 148 7.2.7 Simplifications 149 7.3 Calculating Value Mappings 150 7.4 Defining Scale of Measurement 153 7.4.1 Determining the Scale of Measurement 154 7.5 Addressing Values in Value Mappings 156 7.5.1 Determining the Set of Addressed Source Values 156 7.5.2 Determining the Set of Addressed Target Values 157 7.6 Overlapping Value Mappings 158 7.7 Default Value Mapping 158 7.8 Default Labelling 159 7.9 Defining Interaction 159 7.10 Mapping Composition and Submappings 160 7.11 A Schema Language for RVL 160 7.11.1 Concrete Examples of the RVL Schema 163 7.12 Conclusions and Future Work 166 8 The OGVIC Approach 169 8.1 Ontology-Driven, Guided Editing of Visual Mappings 172 8.1.1 Classification of Constraints 172 8.1.2 Levels of Guidance 173 8.1.3 Implementing Constraint-Based Guidance 173 8.2 Support of Explicit and Composable Visual Mappings 177 8.2.1 Mapping Composition Cases 178 8.2.2 Selecting a Context 180 8.2.3 Using the Same Graphic Relation Multiple Times 181 8.3 Prototype P1 (TopBraid-Composer-based) 182 8.4 Prototype P2 (OntoWiki-based) 184 8.5 Prototype P3 (Java Implementation of RVL) 187 8.6 Lessons Learned from Prototypes & Future Work 190 8.6.1 Checking RVL Constraints and Visualisation Rules 190 8.6.2 A User Interface for Editing RVL Mappings 190 8.6.3 Graph Transformations with SPIN and SPARQL 1.1 Update 192 8.6.4 Selection and Filtering of Data 193 8.6.5 Interactivity and Incremental Processing 193 8.6.6 Rendering the Final Platform-Specific Code 196 9 Application 197 9.1 Coverage of Case Study Sketches and Necessary Features 198 9.2 Coverage of Visualisation Cases 201 9.3 Coverage of Requirements 205 9.4 Full Example 206 10 Conclusions 211 10.1 Contributions 211 10.2 Constructive Evaluation 212 10.3 Research Questions 213 10.4 Transfer to Other Models and Constraint Languages 213 10.5 Limitations 214 10.6 Future Work 214 Appendices 217 A Case Study Sketches 219 B VISO – Comparison of Visualisation Literature 229 C RVL 231 D RVL Example Mappings and Application 233 D.1 Listings of RVL Example Mappings as Required by Protot

Mixed Reality for Manufacturing Execution Systems

A quarta revolução industrial, também conhecida como Indústria 4.0, marca o início de uma nova era onde novos avanços tecnológicos são incluídos e integrados em processos industriais com o intuito de os tornarem mais eficientes. A introdução de sistemas Cyber-Physical, Industrial Internet of Things, Cloud Computing, entre outros, permite a implementação de uma fábrica inteligente, onde todos os sistemas comunicam e interagem de uma forma autónoma e assíncrona, resultando num processo de tomada de decisão descentralizada. Manufacturing Execution Systems (MES)¸ são sistemas que permitem, entre muitas funcionalidades, a gestão e execução de processos de manufatura e do chão de fábrica, com o objetivo de manter um alto desempenho num ambiente complexo e com mudanças constantes. Com esta nova revolução industrial, surgiu a necessidade de os repensar e reestruturar, de modo a que contemplem e sejam compatíveis com as novas tecnologias, de modo a que possam oferecer uma abordagem de tomada de decisão descentralizada. Atualmente, a Critical Manufacturing desenvolve e oferece um sistema MES que está adaptado às necessidades da Indústria 4.0 através da implementação de tecnologias avançadas. Porém, estes avanços tecnológicos originam uma alta quantidade de dados que nem sempre são transformados em conhecimento. O uso de sistemas inteligentes e interoperáveis na Indústria 4.0 não implica que os operadores e outros trabalhadores sejam substituídos. Pelo contrário, estes terão ainda mais responsabilidades e uma maior complexidade nas suas tarefas diárias, exigindo que sejam altamente flexíveis e adaptáveis. Contudo, ainda que se conformem com este tipo de ambiente, a complexidade acrescida resulta numa diminuição de produtividade. A Realidade Mista é um conceito onde o mundo real é combinado com conteúdo virtual que, ao contrário do que acontece na Realidade Aumentada, está ligado e conectado com a realidade, implicando que variações num extremo provoquem alterações no outro. Esta mistura é visualmente exibida através de dispositivos de apresentação, proporcionando ao utilizar uma perceção aumentada do seu meio envolvente e oferecendo funcionalidades através da interação com o conteúdo virtual. O uso deste conceito na Indústria 4.0 abre uma nova porta na interação entre humanos e máquinas e suporte ao operador. Contudo, verifica-se uma falta de investigação e estudo nesta área, onde utilizações e casos de uso básicos ainda não estão explorados. Esta dissertação propõe uma solução onde a realidade mista é usada e integrada no sistema MES da Critical Manufacturing, tirando partido das suas tecnologias e dos dados gerados por outros sistemas e tecnologias de forma a oferecer uma nova abordagem de interação com os objetos físicos presentes no chão de fábrica. Oobjetivo principal é o de diminuir a complexidade das tarefas diárias dos operadores e dos engenheiros, contribuindo para a preservação do alto desempenho dos processos de manufatura e aumento da produtividade. A intenção desta solução é de servir como uma prova de conceito, revelando o potencial da introdução da realidade mista em sistemas MES e no contexto da Indústria 4.0.The fourth industrial revolution, also referred to as Industry 4.0, outlines the beginning of a new era where new technological advancements are put together to make industrial processes more efficient. The introduction of Cyber-Physical Systems (CPS), Industrial Internet of Things (IIoT), cloud computing, and other concepts, drives what is referred to as a “smart factory” in which the different systems are able to autonomously communicate and interact, resulting in a decentralized decision-making process. With this revolution, Manufacturing Execution Systems (MES) need to be rationalized and restructured, contemplating the introduction of these advancements and being able to offer a decentralized approach to manufacturing processes. Currently, Critical Manufacturing develops and delivers a MES that is adapted to the requirements of the Industry 4.0. However, the amount of data generated by these technologies is not always transformed into knowledge for the customer. The use of intelligent and interoperable systems in the Industry 4.0 does not mean that human workers are being replaced. On the contrary, shop-floor operators and engineers are beginning to perceive an increased complexity in their daily tasks, requiring them to be highly flexible and adaptable in a hard-working environment. Nevertheless, this increased complexity results in a decrease in the overall productivity. Mixed Reality (MR) allows to blend the real world with digital content that is connected to it, resulting in a mixed environment that isfurther presented to the user in the same display device. The usage of this technology in the Industry 4.0 opens a new door to human-machine interaction and operator support. However, this topic is still very under-researched and basic use cases are still unexplored. This dissertation proposes a solution where mixed reality is used and integrated with Critical Manufacturing’s Industry 4.0-ready MES, taking advantage of its technologies and data flowing in the system in order to ease the shop-floor engineer daily tasks. The main intention of this subject is to serve as a proof of concept, revealing the potential of the inclusion of mixed reality in a manufacturing execution system in the context of the Industry 4.0

Enabling Collaborative Visual Analysis across Heterogeneous Devices

We are surrounded by novel device technologies emerging at an unprecedented pace. These devices are heterogeneous in nature: in large and small sizes with many input and sensing mechanisms. When many such devices are used by multiple users with a shared goal, they form a heterogeneous device ecosystem. A device ecosystem has great potential in data science to act as a natural medium for multiple analysts to make sense of data using visualization. It is essential as today's big data problems require more than a single mind or a single machine to solve them. Towards this vision, I introduce the concept of collaborative, cross-device visual analytics (C2-VA) and outline a reference model to develop user interfaces for C2-VA. This dissertation covers interaction models, coordination techniques, and software platforms to enable full stack support for C2-VA. Firstly, we connected devices to form an ecosystem using software primitives introduced in the early frameworks from this dissertation. To work in a device ecosystem, we designed multi-user interaction for visual analysis in front of large displays by finding a balance between proxemics and mid-air gestures. Extending these techniques, we considered the roles of different devices–large and small–to present a conceptual framework for utilizing multiple devices for visual analytics. When applying this framework, findings from a user study showcase flexibility in the analytic workflow and potential for generation of complex insights in device ecosystems. Beyond this, we supported coordination between multiple users in a device ecosystem by depicting the presence, attention, and data coverage of each analyst within a group. Building on these parts of the C2-VA stack, the culmination of this dissertation is a platform called Vistrates. This platform introduces a component model for modular creation of user interfaces that work across multiple devices and users. A component is an analytical primitive–a data processing method, a visualization, or an interaction technique–that is reusable, composable, and extensible. Together, components can support a complex analytical activity. On top of the component model, the support for collaboration and device ecosystems comes for granted in Vistrates. Overall, this enables the exploration of new research ideas within C2-VA

Practical, appropriate, empirically-validated guidelines for designing educational games

There has recently been a great deal of interest in the potential of computer games to function as innovative educational tools. However, there is very little evidence of games fulfilling that potential. Indeed, the process of merging the disparate goals of education and games design appears problematic, and there are currently no practical guidelines for how to do so in a coherent manner. In this paper, we describe the successful, empirically validated teaching methods developed by behavioural psychologists and point out how they are uniquely suited to take advantage of the benefits that games offer to education. We conclude by proposing some practical steps for designing educational games, based on the techniques of Applied Behaviour Analysis. It is intended that this paper can both focus educational games designers on the features of games that are genuinely useful for education, and also introduce a successful form of teaching that this audience may not yet be familiar with

Vision science and the visual arts: an enquiry into the science of perception and the art of immersion

ABSTRACT This research addresses questions surrounding two currently prominent visual art topics. The first revolves around the concept of immersion and perceptions of immersive art, the second concerns current interests in art and science and the practice of art and science collaboration. Current debates and theories of immersion revolve around technologically informed virtual models and interactive virtual art environments, however, the history of immersion as a creative medium has been traced to pre-technological, even pre-historical beginnings. In this study, a pre- technological concept of immersion per se is explored, and a new perceptual model of immersion is developed based on the scientific laws and principles of vision. This model is shown to encompass and a long history of immersive traditions and a type of contemporary art practice that until now was not recognised as an immersive art genre. Working at the interstices of visual immersive art and visual science disciplines, collaborative experimental environments are developed that embody both the concepts and definitions proposed in the thesis, and the scientific laws and principles on which they are founded. Pre-technological/visual and contemporary/virtual concepts are further distinguished by their respective scientific and technological alignments and by the types of perceptual phenomena they engender. It is shown that as a creative medium, immersion comprises scientifically informed visual and technologically informed virtual immersive sub-genres, which together provide a broader and more comprehensive account of historical and contemporary immersive art practice

SInCom 2015

2nd Baden-Württemberg Center of Applied Research Symposium on Information and Communication Systems, SInCom 2015, 13. November 2015 in Konstan

Representation Challenges

Augmented Reality (AR) and Artificial Intelligence (AI) are technological domains that closely interact with space at architectural and urban scale in the broader ambits of cultural heritage and innovative design. The growing interest is perceivable in many fields of knowledge, supported by the rapid development and advancement of theory and application, software and devices, fueling a pervasive phenomenon within our daily lives. These technologies demonstrate to be best exploited when their application and other information and communication technology (ICT) advancements achieve a continuum. In particular, AR defines an alternative path to observe, analyze and communicate space and artifacts. Besides, AI opens future scenarios in data processing, redefining the relationship between man and computer. In the last few years, the AR/AI expansion and relationship have raised deep transdisciplinary speculation. The research experiences have shown many cross-relations in Architecture and Design domains. Representation studies could arise an international debate as a convergence place of multidisciplinary theoretical and applicative contributions related to architecture, city, environment, tangible and intangible Cultural Heritage. This book collects 66 papers and identify eight lines of research that may guide future developments