An Augmented Interaction Strategy For Designing Human-Machine Interfaces For Hydraulic Excavators

Lack of adequate information feedback and work visibility, and fatigue due to repetition have been identified as the major usability gaps in the human-machine interface (HMI) design of modern hydraulic excavators that subject operators to undue mental and physical workload, resulting in poor performance. To address these gaps, this work proposed an innovative interaction strategy, termed “augmented interaction”, for enhancing the usability of the hydraulic excavator. Augmented interaction involves the embodiment of heads-up display and coordinated control schemes into an efficient, effective and safe HMI. Augmented interaction was demonstrated using a framework consisting of three phases: Design, Implementation/Visualization, and Evaluation (D.IV.E). Guided by this framework, two alternative HMI design concepts (Design A: featuring heads-up display and coordinated control; and Design B: featuring heads-up display and joystick controls) in addition to the existing HMI design (Design C: featuring monitor display and joystick controls) were prototyped. A mixed reality seating buck simulator, named the Hydraulic Excavator Augmented Reality Simulator (H.E.A.R.S), was used to implement the designs and simulate a work environment along with a rock excavation task scenario. A usability evaluation was conducted with twenty participants to characterize the impact of the new HMI types using quantitative (task completion time, TCT; and operating error, OER) and qualitative (subjective workload and user preference) metrics. The results indicated that participants had a shorter TCT with Design A. For OER, there was a lower error probability due to collisions (PER1) with Design A, and lower error probability due to misses (PER2)with Design B. The subjective measures showed a lower overall workload and a high preference for Design B. It was concluded that augmented interaction provides a viable solution for enhancing the usability of the HMI of a hydraulic excavator

Multimodal Human-Machine Interface For Haptic-Controlled Excavators

The goal of this research is to develop a human-excavator interface for the hapticcontrolled excavator that makes use of the multiple human sensing modalities (visual, auditory haptic), and efficiently integrates these modalities to ensure intuitive, efficient interface that is easy to learn and use, and is responsive to operator commands. Two empirical studies were conducted to investigate conflict in the haptic-controlled excavator interface and identify the level of force feedback for best operator performance

Contributions to virtual reality

153 p.The thesis contributes in three Virtual Reality areas: ¿ Visual perception: a calibration algorithm is proposed to estimate stereo projection parameters in head-mounted displays, so that correct shapes and distances can be perceived, and calibration and control procedures are proposed to obtain desired accommodation stimuli at different virtual distances.¿ Immersive scenarios: the thesis analyzes several use cases demanding varying degrees of immersion and special, innovative visualization solutions are proposed to fulfil their requirements. Contributions focus on machinery simulators, weather radar volumetric visualization and manual arc welding simulation.¿ Ubiquitous visualization: contributions are presented to scenarios where users access interactive 3D applications remotely. The thesis follows the evolution of Web3D standards and technologies to propose original visualization solutions for volume rendering of weather radar data, e-learning on energy efficiency, virtual e-commerce and visual product configurators

Using Serious Games and Virtual Simulations in Construction Technology Education

In tertiary construction education programs, it is becoming increasingly difficult to take students to site to obtain first hand experience of construction technology. This is especially the case with large classes, sites that have limited safety capacity to take students, and the general logistical problems of getting students to and from site. The brevity of site visits provides novelty but restricts the depth of learning. Computer gaming simulation offers a potential alternative. Though not the real thing, the simulation environment provides targeted activities and problems; it also offers the potential to engage students for long periods of time and in an immersive environment. This paper looks at the use of virtual simulation games available in construction and then focuses on the implementation of a specific type of simulation suited to construction technology and construction processes. It involves the hands-on virtual usage of site equipment including excavators, cranes, forklifts and concrete placement equipment in the context of an active construction site. Students control and manipulate the machinery according to assigned tasks. The pros, cons and educational merits of such an approach are analysed including student evaluation of the simulation as a learning vehicle

Acquisition, retention and transfer of heavy equipment operator skills through simulator training

Initiatives and collaborations among heavy construction equipment manufacturing companies and training technology firms to develop and employ simulators for varied training purposes are becoming commonplace. However, human factors research on simulator training for operators of construction equipment is still sparse. For simulator training to be effective, it is necessary to understand how skills are learned using the simulator, how those skills are transferred to other tasks, devices, and real scenarios, and how well skills are retained after simulator training. ^ This research is on skill development, specifically as it applies to operator training for two specific types of heavy construction equipment: excavator and wheel loader. It aims at decomposing the complexity of equipment operation and distinguishing the skills to be acquired for each machine. It consists of five studies, three conducted with students at Purdue and two with expert operators at John Deere. ^ Study 1 investigated whether operation of a simulated hydraulic excavator is influenced by an intervening task performed between initial practice on the excavator and a subsequent retention test using a controls familiarization task (which involves just knowing the control functions). Two intervening tasks were inserted: practicing on a simulated loader, and reading an unrelated text intended to distract the participants. Performance on the simulator was compared against that of a group of participants who practiced on the simulated excavator throughout. The results showed no performance cost attributable to inserting practice on the simulated loader while learning the controls on the simulated excavator. The learning trends, however, prompted the question of whether the same results would bear true for learning a more complex perceptual-motor task. ^ Study 2 was intended to verify whether the alternating equipment sequence yields the same outcome for a more complex truck loading task that involves multiple operations. Besides the two experimental groups (control and loader groups) in Study 1, an additional group which was given practice on the two machines (but with a different practice schedule from the original loader group) was added to address the question of whether the duration of practice on an alternative machine affects performance on the previously learned machine. The number of sessions was also increased, from three to five, to examine the possible influence when participants continue to switch between the machines. Those participants who engaged in intervening practice on the simulated loader showed a smaller performance improvement on learning the truck loading task on the simulated excavator than did the control group who practiced on the simulated excavator for all five sessions. This outcome confirms that the controls familiarization tasks on both machines studied in the preliminary study may have been too simple for the full effects of switching between the machines to be evident. This finding of continued skill improvement upon return to the previously practiced machine inspires consideration of concurrent simulator-based training rather than the practice of learning to operate only one machine at a time. ^ Study 3 analyzed skill transfer using hierarchical task analysis (HTA) to investigate the degree of overlap in specific task components by studying the similarity and dissimilarity of the truck loading task performed in Study 2 on excavator and wheel loader simulators. After the modification and verification by operators of the initial HTAs, the finalized HTAs revealed that the lack of positive transfer found in performing the truck loading task alternately with the excavator and loader was likely due to the differences between loader and excavator in terms of the controls, physical constraints, and the explicit goals and subgoals of the task. In addition, comparing the number of levels of subgoals of HTAs did not evidence any level-of-difficulty differences between tasks. ^ Studies 4 and 5 investigated whether there is a cost when switching between different types of training modules within the same machine. Study 4 was conducted with experienced operators, who provided information on how the four selected tasks on the loader should be performed and classified the perceived difficulty level of each. Verbal protocol analysis was used to decompose the tasks of the four training modules on the loader simulator: 1) Simple Bucket Loading (B1), Filling a Trench (B2), Truck Loading (B3), and Fork Lifting (F). A nine phase, systematic method for deriving the HTAs from the think-aloud protocols was also developed in this study, which successfully generated the four HTAs. The findings show that 1) the HTA of the Fork Lifting module is significantly different than those of the three bucket loading tasks, and 2) although all three bucket loading tasks shared a similar mechanism, the operators ranked B1 as the easiest, followed by B2 and then B3 due to the corresponding accessibility of the dump targets, and fork lifting was ranked as the most difficult task. The results were used to justify the hypotheses for Study 5. ^ Study 5 sought to verify whether an alternating practice sequence within the same machine, i.e. training with an alternative tool (a wide fork) and returning to the original learned tool (a bucket) on a loader simulator, yields better skill transfer and retention (after a one-week interval). Four groups of undergraduate students were tested. Two groups were given two tasks involving bucket loading to practice in the first two sessions, whereas the other two groups were given a bucket loading task in the first session and the fork lifting task in the second session. The transfer and retention tasks both involved a bucket loading task that had not been performed in Sessions 1 and 2. The results showed that the groups who were assigned to practice on two tasks involving the manipulation of buckets performed better in the skill transfer test when the new task was introduced that also involved manipulation of the bucket. The results support thespecificity of training principle (for which the practice conditions match the test conditions and thus facilitate retention or transfer) but not the progressive difficulty training principle (for which difficulty impedes performance in the learning stage but facilitates retention). It is suggested that, when training perceptual-motor tasks, tasks practiced during the learning phase should match the transfer task. Manipulation of task difficulty may play a role only if the tasks share task-relevant cognitive processes and mental models. ^ The overall findings of this research provide: 1) better understanding of skill development for the operation of construction equipment, and 2) evidence as to how the trainees can better utilize their time when training on a single machine and concurrently on multiple machines. The findings add to the general body of knowledge on perceptual-motor skill acquisition and to that on training in a specific domain via a specific technology. The findings are expected to generalize to heavy equipment training in related domains, such as forestry and mining, and domains requiring instrument handling skills and robotic arms, such as surgery and orbital space vessel external operations

Application of virtual reality for risk assessment and training in the minerals industry

The minerals industry often requires people to work in hazardous environments, these environments are constantly increasing in size and complexity as organisations look for new more cost-effective ways of extracting resources. Not only does this size and complexity bring with it additional safety concerns, the introduction of new legislation has placed the responsibility of employee safety with the organisation. Safety has become an important consideration, where once it might have been viewed as costly and counterproductive, organisations are now seeking to gain competitive advantage in this area. Two key areas of a successful safety management programme are risk assessment and training. These are important in designing systems and environments that are as safe as possible and in educating and training personnel to operate safely within those environments. Virtual Reality (VR) technology is one tool that has been applied successfully to the training requirements across a wide range of industries. In the past two years there is evidence to show that VR technology is becoming more widely used, partly due to the reduced cost and a reduction in the perceived technological complexity. As the cost of computing falls and the fidelity of the virtual worlds increases, VR is considered a viable option for a number of applications. Two prototype VR systems were designed and built. The first, a risk visualisation system, enhances a virtual environment with a risk-based overlay. The relationships between dangerous areas and equipment can be visualised in 3D. It also provides a framework for evaluating the risk programmatically at an arbitrary location. The second is a surface mine simulator that uses a hazard identification system as a tool to aid the training of haul truck operators in surface mine. This system includes a world construction tool that allows users to import and prepare the terrain, construct the virtual world, and specify any hazards. The training system can evaluate the performance of a trainee in the virtual world using a simple scoring algorithm

The Plant Simulator as viable means to prevent and manage risk through competencies management: Experiment results

Making decisions and managing competences in complex systems is a challenging task to accomplish. Specifically, the process industry is known for its complexity and sensitivity to critical procedures. Recent disasters like the ‘‘Deepwater Horizon” (2010, 11 fatalities), BP Texas City (2005, 15 fatalities), and AZF Toulouse (2001, 29 fatalities), clearly showed the risk to which we are all exposed. The increasing complexity of processes, due to the simultaneous escalation of automation, optimisation and intensification processes (followed to face globalisation challenges), are moving the attention to the management of abnormal situations, which are even more complex in nature and frequent. This increasing complexity, coupled with the fact that abnormal situations may lead to irreversible losses, is imposing the adoption of adequate approaches and tools that allow for better learning and properly managing abnormal situations. The paper presents a simulation-enabled, experiment-based approach that can be used to prevent and manage risk through competencies management. More specifically, the paper presents the results of the first experiment campaign performed in a Plant Simulator (PS), the first known in the process industry domain, and shows the efficacy of using Immersive Virtual Environments (IVE) both to make decisions and to train teams (not just single operators). The experiment results presented in the paper show the effectiveness of IVE to increase the competencies and train operators and managers. In addition, they explain how conveniently the data collected by means of the PS can be used for making daily decisions to better prevent and manage risks

The Plant Simulator as viable means to prevent and manage risk through competencies management: Experiment results

AbstractMaking decisions and managing competences in complex systems is a challenging task to accomplish. Specifically, the process industry is known for its complexity and sensitivity to critical procedures. Recent disasters like the “Deepwater Horizon” (2010, 11 fatalities), BP Texas City (2005, 15 fatalities), and AZF Toulouse (2001, 29 fatalities), clearly showed the risk to which we are all exposed. The increasing complexity of processes, due to the simultaneous escalation of automation, optimisation and intensification processes (followed to face globalisation challenges), are moving the attention to the management of abnormal situations, which are even more complex in nature and frequent. This increasing complexity, coupled with the fact that abnormal situations may lead to irreversible losses, is imposing the adoption of adequate approaches and tools that allow for better learning and properly managing abnormal situations. The paper presents a simulation-enabled, experiment-based approach that can be used to prevent and manage risk through competencies management. More specifically, the paper presents the results of the first experiment campaign performed in a Plant Simulator (PS), the first known in the process industry domain, and shows the efficacy of using Immersive Virtual Environments (IVE) both to make decisions and to train teams (not just single operators).The experiment results presented in the paper show the effectiveness of IVE to increase the competencies and train operators and managers. In addition, they explain how conveniently the data collected by means of the PS can be used for making daily decisions to better prevent and manage risks