Gesture-Controlled Interaction with Aesthetic Information Sonification

Information representation in augmented and virtual reality systems, and social physical (building) spaces can enhance the efficacy of interacting with and assimilating abstract, non-visual data. Sanification is the process of automatically generated real time information representation. There is a gap in our implementation and knowledge of auditory display systems used to enhance interaction in virtual and augmented reality. This paper addresses that gap by examining methodologies for mapping socio-spatial data to spatialised sanification manipulated with gestural controllers. This is a system of interactive knowledge representation that completes the human integration loop, enabling the user to interact with and manipulate data using 3D spatial gesture and 3D auditory display. Benefits include 1) added immersion in an augmented or virtual reality interface; 2) auditory display avoids visual overload in visually-saturated processes such as designing, evacuation in emergencies, flying aircraft; computer gaming; and 3) bi-modal or auditory representation, due to its time-based character, facilitates cognition of complex information

Using the Proteus virtual environment to train future IT professionals

Abstract. Based on literature review it was established that the use of augmented reality as an innovative technology of student training occurs in following directions: 3D image rendering; recognition and marking of real objects; interaction of a virtual object with a person in real time. The main advantages of using AR and VR in the educational process are highlighted: clarity, ability to simulate processes and phenomena, integration of educational disciplines, building an open education system, increasing motivation for learning, etc. It has been found that in the field of physical process modelling the Proteus Physics Laboratory is a popular example of augmented reality. Using the Proteus environment allows to visualize the functioning of the functional nodes of the computing system at the micro level. This is especially important for programming systems with limited resources, such as microcontrollers in the process of training future IT professionals. Experiment took place at Borys Grinchenko Kyiv University and Sumy State Pedagogical University named after A. S. Makarenko with students majoring in Computer Science (field of knowledge is Secondary Education (Informatics)). It was found that computer modelling has a positive effect on mastering the basics of microelectronics. The ways of further scientific researches for grounding, development and experimental verification of forms, methods and augmented reality, and can be used in the professional training of future IT specialists are outlined in the article