(Always) Playing the Camera: Cyborg Vision and Embodied Surveillance in Digital Games

As the increasingly ubiquitous field of surveillance has transformed how we interact with each other and the world around us, surveillance interactions with virtual others in virtual worlds have gone largely unnoticed. This article examines representations of digital games’ diegetic surveillance cameras and their relation to the player character and player. Building on a dataset of forty-one titles and in-depth analyses of two 2020 digital games that present embodied surveillance camera perspectives, Final Fantasy VII Remake (Square Enix 2020) and Watch Dogs: Legion (Ubisoft Toronto 2020), I demonstrate that the camera is crucial in how we organize, understand, and maneuver the fictional environment and its inhabitants. These digital games reveal how both surveillance power fantasies and their critique can coexist within a space of play. Moreover, digital games often present a perspective that blurs the boundaries between the physical and the technically mediated through a flattening of the player’s “camera” screen and in-game surveillance cameras. Embodied surveillance cameras in digital games make the camera metaphor explicit as an aesthetic, narrative, and mechanical preoccupation. We think and play with and through cameras, drawing attention to and problematizing the partial perspectives with which worlds are viewed. I propose the term cyborg vision to account for this simultaneously human and nonhuman vision that’s both pluralistic and situated and argue that, through cyborg vision, digital games offer an embodied experience of surveillance that’s going to be increasingly relevant in the future.publishedVersio

Can Science Explain Consciousness?

For diverse reasons, the problem of phenomenal consciousness is persistently challenging. Mental terms are characteristically ambiguous, researchers have philosophical biases, secondary qualities are excluded from objective description, and philosophers love to argue. Adhering to a regime of efficient causes and third-person descriptions, science as it has been defined has no place for subjectivity or teleology. A solution to the “hard problem” of consciousness will require a radical approach: to take the point of view of the cognitive system itself. To facilitate this approach, a concept of agency is introduced along with a different understanding of intentionality. Following this approach reveals that the autopoietic cognitive system constructs phenomenality through acts of fiat, which underlie perceptual completion effects and “filling in”—and, by implication, phenomenology in general. It creates phenomenality much as we create meaning in language, through the use of symbols that it assigns meaning in the context of an embodied evolutionary history that is the source of valuation upon which meaning depends. Phenomenality is a virtual representation to itself by an executive agent (the conscious self) tasked with monitoring the state of the organism and its environment, planning future action, and coordinating various sub- agencies. Consciousness is not epiphenomenal, but serves a function for higher organisms that is distinct from that of unconscious processing. While a strictly scientific solution to the hard problem is not possible for a science that excludes the subjectivity it seeks to explain, there is hope to at least psychologically bridge the explanatory gulf between mind and matter, and perhaps hope for a broader definition of science

A survey of new technology for cockpit application to 1990's transport aircraft simulators

Two problems were investigated: inter-equipment data transfer, both on board the aircraft and between air and ground; and crew equipment communication via the cockpit displays and controls. Inter-equipment data transfer is discussed in terms of data bus and data link requirements. Crew equipment communication is discussed regarding the availability of CRT display systems for use in research simulators to represent flat panel displays of the future, and of software controllable touch panels

Functional requirements for the man-vehicle systems research facility

The NASA Ames Research Center proposed a man-vehicle systems research facility to support flight simulation studies which are needed for identifying and correcting the sources of human error associated with current and future air carrier operations. The organization of research facility is reviewed and functional requirements and related priorities for the facility are recommended based on a review of potentially critical operational scenarios. Requirements are included for the experimenter's simulation control and data acquisition functions, as well as for the visual field, motion, sound, computation, crew station, and intercommunications subsystems. The related issues of functional fidelity and level of simulation are addressed, and specific criteria for quantitative assessment of various aspects of fidelity are offered. Recommendations for facility integration, checkout, and staffing are included

Nonlinear Modeling and Control of Driving Interfaces and Continuum Robots for System Performance Gains

With the rise of (semi)autonomous vehicles and continuum robotics technology and applications, there has been an increasing interest in controller and haptic interface designs. The presence of nonlinearities in the vehicle dynamics is the main challenge in the selection of control algorithms for real-time regulation and tracking of (semi)autonomous vehicles. Moreover, control of continuum structures with infinite dimensions proves to be difficult due to their complex dynamics plus the soft and flexible nature of the manipulator body. The trajectory tracking and control of automobile and robotic systems requires control algorithms that can effectively deal with the nonlinearities of the system without the need for approximation, modeling uncertainties, and input disturbances. Control strategies based on a linearized model are often inadequate in meeting precise performance requirements. To cope with these challenges, one must consider nonlinear techniques. Nonlinear control systems provide tools and methodologies for enabling the design and realization of (semi)autonomous vehicle and continuum robots with extended specifications based on the operational mission profiles. This dissertation provides an insight into various nonlinear controllers developed for (semi)autonomous vehicles and continuum robots as a guideline for future applications in the automobile and soft robotics field. A comprehensive assessment of the approaches and control strategies, as well as insight into the future areas of research in this field, are presented.First, two vehicle haptic interfaces, including a robotic grip and a joystick, both of which are accompanied by nonlinear sliding mode control, have been developed and studied on a steer-by-wire platform integrated with a virtual reality driving environment. An operator-in-the-loop evaluation that included 30 human test subjects was used to investigate these haptic steering interfaces over a prescribed series of driving maneuvers through real time data logging and post-test questionnaires. A conventional steering wheel with a robust sliding mode controller was used for all the driving events for comparison. Test subjects operated these interfaces for a given track comprised of a double lane-change maneuver and a country road driving event. Subjective and objective results demonstrate that the driver’s experience can be enhanced up to 75.3% with a robotic steering input when compared to the traditional steering wheel during extreme maneuvers such as high-speed driving and sharp turn (e.g., hairpin turn) passing. Second, a cellphone-inspired portable human-machine-interface (HMI) that incorporated the directional control of the vehicle as well as the brake and throttle functionality into a single holistic device will be presented. A nonlinear adaptive control technique and an optimal control approach based on driver intent were also proposed to accompany the mechatronic system for combined longitudinal and lateral vehicle guidance. Assisting the disabled drivers by excluding extensive arm and leg movements ergonomically, the device has been tested in a driving simulator platform. Human test subjects evaluated the mechatronic system with various control configurations through obstacle avoidance and city road driving test, and a conventional set of steering wheel and pedals were also utilized for comparison. Subjective and objective results from the tests demonstrate that the mobile driving interface with the proposed control scheme can enhance the driver’s performance by up to 55.8% when compared to the traditional driving system during aggressive maneuvers. The system’s superior performance during certain vehicle maneuvers and approval received from the participants demonstrated its potential as an alternative driving adaptation for disabled drivers. Third, a novel strategy is designed for trajectory control of a multi-section continuum robot in three-dimensional space to achieve accurate orientation, curvature, and section length tracking. The formulation connects the continuum manipulator dynamic behavior to a virtual discrete-jointed robot whose degrees of freedom are directly mapped to those of a continuum robot section under the hypothesis of constant curvature. Based on this connection, a computed torque control architecture is developed for the virtual robot, for which inverse kinematics and dynamic equations are constructed and exploited, with appropriate transformations developed for implementation on the continuum robot. The control algorithm is validated in a realistic simulation and implemented on a six degree-of-freedom two-section OctArm continuum manipulator. Both simulation and experimental results show that the proposed method could manage simultaneous extension/contraction, bending, and torsion actions on multi-section continuum robots with decent tracking performance (e.g. steady state arc length and curvature tracking error of 3.3mm and 130mm-1, respectively). Last, semi-autonomous vehicles equipped with assistive control systems may experience degraded lateral behaviors when aggressive driver steering commands compete with high levels of autonomy. This challenge can be mitigated with effective operator intent recognition, which can configure automated systems in context-specific situations where the driver intends to perform a steering maneuver. In this article, an ensemble learning-based driver intent recognition strategy has been developed. A nonlinear model predictive control algorithm has been designed and implemented to generate haptic feedback for lateral vehicle guidance, assisting the drivers in accomplishing their intended action. To validate the framework, operator-in-the-loop testing with 30 human subjects was conducted on a steer-by-wire platform with a virtual reality driving environment. The roadway scenarios included lane change, obstacle avoidance, intersection turns, and highway exit. The automated system with learning-based driver intent recognition was compared to both the automated system with a finite state machine-based driver intent estimator and the automated system without any driver intent prediction for all driving events. Test results demonstrate that semi-autonomous vehicle performance can be enhanced by up to 74.1% with a learning-based intent predictor. The proposed holistic framework that integrates human intelligence, machine learning algorithms, and vehicle control can help solve the driver-system conflict problem leading to safer vehicle operations

A Topology of Shared Control Systems—Finding Common Ground in Diversity

Shared control is an increasingly popular approach to facilitate control and communication between humans and intelligent machines. However, there is little consensus in guidelines for design and evaluation of shared control, or even in a definition of what constitutes shared control. This lack of consensus complicates cross fertilization of shared control research between different application domains. This paper provides a definition for shared control in context with previous definitions, and a set of general axioms for design and evaluation of shared control solutions. The utility of the definition and axioms are demonstrated by applying them to four application domains: automotive, robot-assisted surgery, brain–machine interfaces, and learning. Literature is discussed for each of these four domains in light of the proposed definition and axioms. Finally, to facilitate design choices for other applications, we propose a hierarchical framework for shared control that links the shared control literature with traded control, co-operative control, and other human–automation interaction methods. Future work should reveal the generalizability and utility of the proposed shared control framework in designing useful, safe, and comfortable interaction between humans and intelligent machines

An Approach to Large Scale Radar-Based Modeling and Simulation

This research presents a method of aggregating, or reducing the resolution, of a commonly available Department of Defense (DoD) simulation. It addresses the differences between varying levels of resolution and scope used in the DoD’s hierarchy of models pyramid. A data representation that aggregates engagement-level simulation data to use at a lower resolution level, the mission-level, is presented and analyzed. Two formats of implementing this data representation are developed and compared: the rigid cylinder format and the expanding tables format. The rigid cylinder format provides an intuitive way to visualize the data and is used to develop the theory. The expanding tables format expands upon the capabilities of the rigid cylinder format and reduces the simulation time. Tests are run to show the effects of each format for various combinations of engagement-level simulation inputs. A final set of tests highlight the loss in accuracy incurred from reducing the number of samples used by the mission-level simulation. These tests culminate the work by deriving a notional scenario, applying the data cylinder representation, and exploring the realistic problem of comparing accuracy and computational constraints

The Social Cognitive Actor

Multi-Agent Simulation (MAS) of organisations is a methodology that is adopted in this dissertation in order to study and understand human behaviour in organisations. The aim of the research is to design and implementat a cognitive and social multi-agent simulation model based on a selection of social and cognitive theories to fulfill the need for a complex cognitive and social model. The emphasis of this dissertation is the relationship between behaviour of individuals (micro-level) in an organisation and the behaviour of the organisation as a whole (macro-level)

Enhancing user experience and safety in the context of automated driving through uncertainty communication

Operators of highly automated driving systems may exhibit behaviour characteristic of overtrust issues due to an insufficient awareness of automation fallibility. Consequently, situation awareness in critical situations is reduced and safe driving performance following emergency takeovers is impeded. Previous research has indicated that conveying system uncertainties may alleviate these issues. However, existing approaches require drivers to attend the uncertainty information with focal attention, likely resulting in missed changes when engaged in non-driving-related tasks. This research project expands on existing work regarding uncertainty communication in the context of automated driving. Specifically, it aims to investigate the implications of conveying uncertainties under consideration of non-driving-related tasks and, based on the outcomes, develop and evaluate an uncertainty display that enhances both user experience and driving safety. In a first step, the impact of visually conveying uncertainties was investigated under consideration of workload, trust, monitoring behaviour, non-driving-related tasks, takeover performance, and situation awareness. For this, an anthropomorphic visual uncertainty display located in the instrument cluster was developed. While the hypothesised benefits for trust calibration and situation awareness were confirmed, the results indicate that visually conveying uncertainties leads to an increased perceived effort due to a higher frequency of monitoring glances. Building on these findings, peripheral awareness displays were explored as a means for conveying uncertainties without the need for focused attention to reduce monitoring glances. As a prerequisite for developing such a display, a systematic literature review was conducted to identify evaluation methods and criteria, which were then coerced into a comprehensive framework. Grounded in this framework, a peripheral awareness display for uncertainty communication was developed and subsequently compared with the initially proposed visual anthropomorphic uncertainty display in a driving simulator study. Eye tracking and subjective workload data indicate that the peripheral awareness display reduces the monitoring effort relative to the visual display, while driving performance and trust data highlight that the benefits of uncertainty communication are maintained. Further, this research project addresses the implications of increasing the functional detail of uncertainty information. Results of a driving simulator study indicate that particularly workload should be considered when increasing the functional detail of uncertainty information. Expanding upon this approach, an augmented reality display concept was developed and a set of visual variables was explored in a forced choice sorting task to assess their ordinal characteristics. Particularly changes in colour hue and animation-based variables received high preference ratings and were ordered consistently from low to high uncertainty. This research project has contributed a series of novel insights and ideas to the field of human factors in automated driving. It confirmed that conveying uncertainties improves trust calibration and situation awareness, but highlighted that using a visual display lessens the positive effects. Addressing this shortcoming, a peripheral awareness display was designed applying a dedicated evaluation framework. Compared with the previously employed visual display, it decreased monitoring glances and, consequentially, perceived effort. Further, an augmented reality-based uncertainty display concept was developed to minimise the workload increments associated with increases in the functional detail of uncertainty information.</div

Fractional Order State Feedback Control for Improved Lateral Stability of Semi-Autonomous Commercial Heavy Vehicles

With the growing development of autonomous and semi-autonomous large commercial heavy vehicles, the lateral stability control of articulated vehicles have caught the attention of researchers recently. Active vehicle front steering (AFS) can enhance the handling performance and stability of articulated vehicles for an emergency highway maneuver scenario. However, with large vehicles such tractor-trailers, the system becomes more complex to control and there is an increased occurrence of instabilities. This research investigates a new control scheme based on fractional calculus as a technique that ensures lateral stability of articulated large heavy vehicles during evasive highway maneuvering scenarios. The control method is first implemented to a passenger vehicle model with 2-axles based on the well-known “bicycle model”. The model is then extended and applied onto larger three-axle commercial heavy vehicles in platooning operations. To validate the proposed new control algorithm, the system is linearized and a fractional order PI state feedback control is developed based on the linearized model. Then using Matlab/Simulink, the developed fractional-order linear controller is implemented onto the non-linear tractor-trailer dynamic model. The tractor-trailer system is modeled based on the conventional integer-order techniques and then a non-integer linear controller is developed to control the system. Overall, results confirm that the proposed controller improves the lateral stability of a tractor-trailer response time by 20% as compared to a professional truck driver during an evasive highway maneuvering scenario. In addition, the effects of variable truck cargo loading and longitudinal speed are evaluated to confirm the robustness of the new control method under a variety of potential operating conditions