Automated Conversion of Text Instructions to Human Motion Animation

Text to animation is the conversion of textual instructions to animations following and depicting those instructions. Text to animation is important to fields such as crime scene investigation, military special operations and storytelling where text is utilized. It is also useful for physical therapy where a doctor can factor in disability parameters in the 3D models and determine the safest therapy exercises for patients. It can be used in robotics, where robot assistants can be instructed to perform tasks through text instructions. In this research, a system was created to generate 3D animation of workouts from their textual instructions. An algorithm was developed to generate animation sequences from test cases. The algorithm utilized an animation graph which was created from a training set of 100 workouts with the nodes as postures and the links as animation-clip names along with their action data. The algorithm is designed to find and generate the closest animations available. A testing set of 40 similar workouts was used to test the algorithm and obtain an output of sequence of animations which were later depicted by the Unity Game Engine. In user evaluations, for 25 of the 40 test workouts, or 62.5% of the test workouts, the animation was determined by the users, to have the same or almost the same human motions as compared to a video of a human performing the workout. Analysis of the 15 workouts that were not similar to the human video (37.5%) showed that their issues can be fixed and the animation search can be improved by training the animation graph on more varieties of the workouts and on different text instructions of the same workouts from different sources. For 64.5% of the 156 sentences of all the text workouts, the animations were determined to correctly depict the text instructions. This research provides an initial animation graph and animation library which can be expanded to include more poses and animations from other domains such as yoga or dance. In the future system has the potential to provide users with no 3D modelling/animation expertise, a way to create 3D animations with just texts

Virtual Movement from Natural Language Text

It is a challenging task for machines to follow a textual instruction. Properly understanding and using the meaning of the textual instruction in some application areas, such as robotics, animation, etc. is very difficult for machines. The interpretation of textual instructions for the automatic generation of the corresponding motions (e.g. exercises) and the validation of these movements are difficult tasks. To achieve our initial goal of having machines properly understand textual instructions and generate some motions accordingly, we recorded five different exercises in random order with the help of seven amateur performers using a Microsoft Kinect device. During the recording, we found that the same exercise was interpreted differently by each human performer even though they were given identical textual instructions. We performed a quality assessment study based on the derived data using a crowdsourcing approach. Later, we tested the inter-rater agreement for different types of visualization, and found the RGB-based visualization showed the best agreement among the annotatorsa animation with a virtual character standing in second position. In the next phase we worked with physical exercise instructions. Physical exercise is an everyday activity domain in which textual exercise descriptions are usually focused on body movements. Body movements are considered to be a common element across a broad range of activities that are of interest for robotic automation. Our main goal is to develop a text-to-animation system which we can use in different application areas and which we can also use to develop multiple-purpose robots whose operations are based on textual instructions. This system could be also used in different text to scene and text to animation systems. To generate a text-based animation system for physical exercises the process requires the robot to have natural language understanding (NLU) including understanding non-declarative sentences. It also requires the extraction of semantic information from complex syntactic structures with a large number of potential interpretations. Despite a comparatively high density of semantic references to body movements, exercise instructions still contain large amounts of underspecified information. Detecting, and bridging and/or filling such underspecified elements is extremely challenging when relying on methods from NLU alone. However, humans can often add such implicit information with ease due to its embodied nature. We present a process that contains the combination of a semantic parser and a Bayesian network. In the semantic parser, the system extracts all the information present in the instruction to generate the animation. The Bayesian network adds some brain to the system to extract the information that is implicit in the instruction. This information is very important for correctly generating the animation and is very easy for a human to extract but very difficult for machines. Using crowdsourcing, with the help of human brains, we updated the Bayesian network. The combination of the semantic parser and the Bayesian network explicates the information that is contained in textual movement instructions so that an animation execution of the motion sequences performed by a virtual humanoid character can be rendered. To generate the animation from the information we basically used two different types of Markup languages. Behaviour Markup Language is used for 2D animation. Humanoid Animation uses Virtual Reality Markup Language for 3D animation

Acting on behalf of the concept

This paper discusses how drama process and techniques are providing alternative approaches to product concept generation. An investigation that used drama techniques for concept generation sessions observed that there appears to be an implicit response among designers to investigate functionality before, or instead of form. However, it was proposed that through practice the approach of ‘concept-acting’ would provide support for the designer’s kinaesthetic needs for touch, feel and positional experience. It was also observed that whilst an increasing number of people in the US are actively embracing this type of approach, through a variety of techniques, UK designers appear somewhat more sceptical of the value of drama to their design processes

Improving the Quality of Technology-Enhanced Learning for Computer Programming Courses

Teaching computing courses is a major challenge for the majority of lecturers in Libyan higher learning institutions. These courses contain numerous abstract concepts that cannot be easily explained using traditional educational methods. This paper describes the rationale, design, development and implementation stages of an e-learning package (including multimedia resources such as simulations, animations, and videos) using the ASSURE model. This training package can be used by students before they attend practical computer lab sessions, preparing them by developing technical skills and applying concepts and theories presented in lecture through supplementary study and exercises

Active Learning in Sophomore Mathematics: A Cautionary Tale

Math 245: Multivariate Calculus, Linear Algebra, and Differential Equations with Computer I is the first half of a year-long sophomore sequence that emphasizes the subjects\u27 interconnections and grounding in real-world applications. The sequence is aimed primarily at students from physical and mathematical sciences and engineering. In Fall, 1998, as a result of my affiliation with the Science, Technology, Engineering, and Mathematics Teacher Education Collaborative (STEMTEC), I continued and extended previously-introduced reforms in Math 245, including: motivating mathematical ideas with real-world phenomena; student use of computer technology; and, learning by discovery and experimentation. I also introduced additional pedagogical strategies for more actively involving the students in their own learning—a collaborative exam component and in-class problem-solving exercises. The in-class exercises were well received and usually productive; two were especially effective at revealing normally unarticulated thinking. The collaborative exam component was of questionable benefit and was subsequently abandoned. Overall student performance, as measured by traditional means, was disappointing. Among the plausible reasons for this result is that too much material was covered in too short a time. Experience here suggests that active-learning strategies can be useful, but are unlikely to succeed unless one sets realistic limits to content coverage

Emerging technologies in physics education

Three emerging technologies in physics education are evaluated from the interdisciplinary perspective of cognitive science and physics education research. The technologies - Physlet Physics, the Andes Intelligent Tutoring System (ITS), and Microcomputer-Based Laboratory (MBL) Tools - are assessed particularly in terms of their potential at promoting conceptual change, developing expert-like problem-solving skills, and achieving the goals of the traditional physics laboratory. Pedagogical methods to maximize the potential of each educational technology are suggested.Comment: Accepted for publication in the Journal of Science Education and Technology; 20 page

A mobile fitness companion

The paper introduces a Mobile Companion prototype, which helps users to plan and keep track of their exercise activities via an interface based mainly on speech input and output. The Mobile Companion runs on a PDA and is based on a stand-alone, speaker-independent solution, making it fairly unique among mobile spoken dialogue systems, where the common solution is to run the ASR on a separate server or to restrict the speech input to some specific set of users. The prototype uses a GPS receiver to collect position, distance and speed data while the user is exercising, and allows the data to be compared to previous exercises. It communicates over the mobile network with a stationary system, placed in the user’s home. This allows plans for exercise activities to be downloaded from the stationary to the mobile system, and exercise result data to be uploaded once an exercise has been completed