Storytelling in the Metaverse: From Desktop to Immersive Virtual Reality Storyboarding

Creatives from the animation and film industries have always been experimenting with innovative tools and methodologies to improve the creation of prototypes of their visual sequences before bringing them to life. In recent years, as realistic real-time rendering techniques have emerged, the increasing popularity of virtual reality (VR) can lead to new approaches and solutions, leveraging the immersive and interactive features provided by 3D immersive experiences. A 3D desktop application and a novel storyboarding pipeline, which can automatically generate a storyboard including camera details and a textual description of the actions performed in three-dimensional environments, have already been investigated in previous work. The aim was to exploit new technologies to improve existing 3D storytelling approaches, thus providing a software solution for expert and novice storyboarders. This research investigates 3D storyboarding in immersive virtual reality (IVR) to move toward a new storyboarding paradigm. IVR systems provide peculiarities such as body-controlled exploration of the 3D scene and a head-dependant camera view that can extend features of traditional storyboarding tools. The proposed system enables users to set up the virtual stage, adding elements to the scene and exploring the environment as they build it. After that, users can select the available characters or the camera, control them in first person, position them in the scene, and perform actions selecting from a list of options, each paired with a corresponding animation. Relying on the concept of state-machine, the system can automatically generate the list of available actions depending on the context. Finally, the descriptions for each storyboard panel are automatically generated based on the history of activities performed. The proposed application maintains all the functionalities of the desktop version and can be effectively used to create storyboards in immersive virtual environments

Visual Design Process: Communicating with Storyboarding and Augmented Reality

In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety.In the building profession, it is essential for architects to communicate clearly and effectively to numerous people. This thesis introduces a design process utilizing two visual tools— Storyboarding and Augmented Reality—as a methodology for stronger visual communication. Storyboarding is a visual design tool that communicates concepts, ideas, and specifics quickly through clear illustrations. It allows designers to visually communicate to a team and change ideas during the development phase of a project. Augmented Reality is a visual tool that communicates superimposed digital information through a composite view of the real and digital world. This gives designers the ability to present more information about completed projects. Through further investigation of this design process, I have found that it is the balance between the two and the transition of information between design phases that allows for a brand new process of visual communication throughout a design project’s entirety

CuriosityXR: Contextualizing Learning through Immersive Mixed Reality Experiences Beyond the Classroom

The focus of education is shifting towards a learner-centered approach that highlights the importance of engagement, interaction, and personalization in learning. This thesis explores new technologies to facilitate immersive, self-directed, curiosity-driven learning experiences aimed at addressing these key factors. I explore the use of Mixed Reality (MR) to build a context-aware system that can support learners’ curiosity and improve knowledge recall. I design and build “Curiosity XR,” an application for MR headsets using a research-through-design methodology. Curiosity XR is also a platform that enables educators to create contextual multi-modal interactive mini-lessons, and learners can engage with these lessons and other AI-assisted learning content. To evaluate my design, I conduct a user participant study followed by interviews. The participants’ responses show higher levels of engagement, curiosity to learn more, and better visual retention of the learning content. I hope this work will inspire others in the MR community and advance the use of MR and AI hybrid designs for the future of curiosity-driven education

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Sketching & drawing as future inquiry in HCI

Creating visual imagery helps us to situate ourselves within unknown worlds, processes, make connections, and find solutions. By exploring drawn ideas for novel technologies, we can examine the implications of their place in the world. Drawing, or sketching, for future inquiry in Human Computer Interaction (HCI) can be a stand-alone investigative approach, part of a wider ‘world-building’ in design fiction, or simply ideation around a concept. By examining instances of existing practice in HCI, in this paper we establish recommendations and rationales for those wishing to utilise sketching and drawing within their research. We examine approaches ranging from ideation, diagramming, scenario building, comics creation and artistic representation to create a model for sketching and drawing as future inquiry for HCI. This work also reflects on the ways in which these arts can inform and elucidate research and practice in HCI, and makes recommendations for the field, within its teaching, processes and outcomes

Virtual creation - Architectural design with virtual reality

Virtual reality (VR) shows promise as an important tool for architecture and design. Recent advancements in the technology are evolving the ways in which the tool is being implemented in the design field. There is a need to research the use of VR in design industry. The thesis aims to illustrate the possible uses of this new technology by answering the research question of how can virtual reality help designers to improve the design process. The use of virtual reality is examined in detail with qualitative research methods, interviews and observation, and as a tool in a creative process. The hypothesis of the study is that virtual reality is a valuable architectural design tool which improves the efficiency and quality of the design work while also effectively aiding clients, constructors, and fellow designers in understanding the creative vision of design work through all phases of a project. This study identifies that virtual reality can be utilized in all design phases, reducing risks and saving time and money in design projects. VR is helping to make design decisions, discover design mistakes, comprehend the relations of the spaces and allow designers to prototype their designs in full scale. Virtual reality provides a new tool for designers to express, reflect, represent, market and sell their designs. The outcome of the thesis outlines the process of using VR as a design tool and utilizes the knowledge in the creative process. The creative process of the thesis was created by designing three modular prefabricated houses by utilizing virtual reality as a design tool in the design development phase. Overall, the study strengthens the hypothesis that the use of virtual reality improves the design process and collaboration between clients, designers and the project team. Several challenges, however, can impact the use of VR such as motion sickness, organizational issues and lack of control. Although adopting new technologies such as VR can cause uncertainty and disruptions in the industry, it can also diversify the work opportunities for designers. Future research should focus on overall adoption rates of VR in architecture. In addition, more knowledge is needed on the design phases using VR in wider contexts as well as collaboration of the project team.Virtuaalitodellisuus (VR) nähdään lupaavana työkaluna suunnittelussa. Teknologian kehittyminen on parantanut työkalun hyödyntämismahdollisuuksia arkkitehtuuristen tilojen suunnittelussa. Opinnäytetyöni vastaa tarpeeseen selvittää uuden teknologian mahdollisuuksia suunnittelussa. Opinnäytetyön tarkoituksena on selvittää kuinka virtuaalitodellisuus voi auttaa suunnittelijoita parantamaan suunnitteluprosessia. Tutkin virtuaalitodellisuuden käyttöä kvalitatiivisilla tutkimusmenetelmillä, haastatteluilla ja havainnoinnilla, sekä luovan prosessin välineenä. Tutkimuksen hypoteesi on, että virtuaalitodellisuus on arvokas arkkitehtoninen suunnittelutyökalu, joka parantaa suunnittelijoiden, rakentajien ja asiakkaiden ymmärrystä sekä projektin laatua kaikkien suunnitteluvaiheiden aikana. Tutkimuksessa todetaan, että virtuaalitodellisuutta hyödynnetään kaikissa suunnitteluvaiheissa, se säästää rahaa ja aikaa sekä vähentää riskejä. Virtuaalitodellisuus tehostaa suunnittelua ja auttaa tekemään päätöksiä, löytämään virheitä, ymmärtämään tilojen suhteita ja testaamaan prototyyppiä sen luonnollisessa mittakaavassa. Virtuaalitodellisuudesta on tullut uusi työkalu suunnitelmien ilmaisuun, esitykseen, markkinointiin ja myyntiin. Opinnäytetyössä kuvataan virtuaalitodellisuutta suunnittelun työkaluna ja hyödynnetään informaatiota tutkimuksen luovassa osuudessa. Luova osuus kuvastaa kolmen modulaarisen esivalmistetun talon suunnitteluprosessia, jossa hyödynnetään virtuaalitodellisuutta suunnittelun tukena. Tutkimus vahvistaa hypoteesia siitä, että virtuaalitodellisuuden käyttö parantaa suunnitteluprosessia ja yhteistyötä asiakkaiden, suunnittelijoiden ja projektiryhmän välillä. Virtuaalitodellisuuden käyttöön voivat vaikuttaa monet haasteet, kuten esimerkiksi pahoinvointi, organisaation adoptoinnin haasteet ja VR kokemuksen kontrolloinnin vaikeudet. Vaikka uuden teknologian käyttöönotto voi aiheuttaa epävarmuutta ja häiriöitä, se voi myös monipuolistaa suunnittelijoiden työmahdollisuuksia. Tutkimuksen tulosten pohjalta suositellaan tutkimustiedon kartoittamista VR:n käytön määrästä tilasuunnittelussa. Laaja-alaisempaa lisätietoa tarvitaan myös suunnitteluvaiheiden ja projektiryhmän yhteistyön helpottumisesta virtuaalitodellisuuden avulla