Barcelona and Madrid

Suggested readings for the Penn Alumni travel trip to Barcelona and Madrid. See the Library Guide for this bibliography here

Cuba

Suggested resources for the Penn Alumni Travel excursion to Cuba. See the Library Guide for this bibliography here

Improving particle swarm optimization path planning through inclusion of flight mechanics

Military engagements are continuing the movement toward automated and unmanned vehicles for a variety of simple and complex tasks. This allows humans to stay away from dangerous situations and use their skills for more difficult tasks. One important piece of this strategy is the use of automated path planners for unmanned aerial vehicles (UAVs). Current UAV operation requires multiple individuals to control a single plane, tying up important human resources. Often paths are planned by creating waypoints for a vehicle to fly through, with the intention of doing reconnaissance while avoiding as much danger to the plane as possible. Path planners often plan routes without taking into consideration the UAV\u27s ability to perform the maneuvers required to fly the specified waypoints, instead relying upon them to fly as close as possible. This thesis presents a path planner solution incorporating vehicle mechanics to insure feasible flight paths. This path planner uses Particle Swarm Optimization (PSO) and digital pheromones to generate multiple three-dimensional flight paths for the operator to choose from. B-spline curves are generated using universal interpolation with each path waypoint representing a control point. The b-spline curve represents the flight path of the UAV. Each point along the curve is evaluated for fuel efficiency, threat avoidance, reconnaissance, terrain avoidance, and vehicle mechanics. Optimization of the flight path occurs based on operator defined performance characteristics, such as maximum threat avoidance or minimum vehicle dynamics cost. These performance characteristics can be defined for each unique aircraft, allowing the same formulation to be used for any aircraft. The vehicle mechanics conditions considered are pull-out, glide, climb, and steady, level, co-ordinate turns. Calculating the flight mechanics requires knowing the velocity and angle of the plane, calculated using the derivative of the point on the curve. The flight mechanics of the path allows the path planner to determine whether the path exceeds the maximum load factor (G-force), minimum velocity (stall velocity), or the minimum turning radius. Comparing the results between PSO Path Planner with flight mechanics and PSO Path Planner without flight mechanics over five scenarios indicates an increase in the feasibility of the returned paths. Visualizing the flight paths was improved by changing the original waypoint based visualization to a b-spline curve representation. Using b-spline curves allows for an accurate representation of the actual UAV flight path especially when considering turns. Operators no longer must create a mental representation of the flight path to match the waypoints

Dynamic Volume Rendering of Functional Medical Data on Dissimilar Hardware Platforms

In the last 30 years, medical imaging has become one of the most used diagnostic tools in the medical profession. Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) technologies have become widely adopted because of their ability to capture the human body in a non-invasive manner. A volumetric dataset is a series of orthogonal 2D slices captured at a regular interval, typically along the axis of the body from the head to the feet. Volume rendering is a computer graphics technique that allows volumetric data to be visualized and manipulated as a single 3D object. Iso-surface rendering, image splatting, shear warp, texture slicing, and raycasting are volume rendering methods, each with associated advantages and disadvantages. Raycasting is widely regarded as the highest quality renderer of these methods. Originally, CT and MRI hardware was limited to providing a single 3D scan of the human body. The technology has improved to allow a set of scans capable of capturing anatomical movements like a beating heart. The capturing of anatomical data over time is referred to as functional imaging. Functional MRI (fMRI) is used to capture changes in the human body over time. While fMRI’s can be used to capture any anatomical data over time, one of the more common uses of fMRI is to capture brain activity. The fMRI scanning process is typically broken up into a time consuming high resolution anatomical scan and a series of quick low resolution scans capturing activity. The low resolution activity data is mapped onto the high resolution anatomical data to show changes over time. Academic research has advanced volume rendering and specifically fMRI volume rendering. Unfortunately, academic research is typically a one-off solution to a singular medical case or set of data, causing any advances to be problem specific as opposed to a general capability. Additionally, academic volume renderers are often designed to work on a specific device and operating system under controlled conditions. This prevents volume rendering from being used across the ever expanding number of different computing devices, such as desktops, laptops, immersive virtual reality systems, and mobile computers like phones or tablets. This research will investigate the feasibility of creating a generic software capability to perform real-time 4D volume rendering, via raycasting, on desktop, mobile, and immersive virtual reality platforms. Implementing a GPU-based 4D volume raycasting method for mobile devices will harness the power of the increasing number of mobile computational devices being used by medical professionals. Developing support for immersive virtual reality can enhance medical professionals’ interpretation of 3D physiology with the additional depth information provided by stereoscopic 3D. The results of this research will help expand the use of 4D volume rendering beyond the traditional desktop computer in the medical field. Developing the same 4D volume rendering capabilities across dissimilar platforms has many challenges. Each platform relies on their own coding languages, libraries, and hardware support. There are tradeoffs between using languages and libraries native to each platform and using a generic cross-platform system, such as a game engine. Native libraries will generally be more efficient during application run-time, but they require different coding implementations for each platform. The decision was made to use platform native languages and libraries in this research, whenever practical, in an attempt to achieve the best possible frame rates. 4D volume raycasting provides unique challenges independent of the platform. Specifically, fMRI data loading, volume animation, and multiple volume rendering. Additionally, real-time raycasting has never been successfully performed on a mobile device. Previous research relied on less computationally expensive methods, such as orthogonal texture slicing, to achieve real-time frame rates. These challenges will be addressed as the contributions of this research. The first contribution was exploring the feasibility of generic functional data input across desktop, mobile, and immersive virtual reality. To visualize 4D fMRI data it was necessary to build in the capability to read Neuroimaging Informatics Technology Initiative (NIfTI) files. The NIfTI format was designed to overcome limitations of 3D file formats like DICOM and store functional imagery with a single high-resolution anatomical scan and a set of low-resolution anatomical scans. Allowing input of the NIfTI binary data required creating custom C++ routines, as no object oriented APIs freely available for use existed. The NIfTI input code was built using C++ and the C++ Standard Library to be both light weight and cross-platform. Multi-volume rendering is another challenge of fMRI data visualization and a contribution of this work. fMRI data is typically broken into a single high-resolution anatomical volume and a series of low-resolution volumes that capture anatomical changes. Visualizing two volumes at the same time is known as multi-volume visualization. Therefore, the ability to correctly align and scale the volumes relative to each other was necessary. It was also necessary to develop a compositing method to combine data from both volumes into a single cohesive representation. Three prototype applications were built for the different platforms to test the feasibility of 4D volume raycasting. One each for desktop, mobile, and virtual reality. Although the backend implementations were required to be different between the three platforms, the raycasting functionality and features were identical. Therefore, the same fMRI dataset resulted in the same 3D visualization independent of the platform itself. Each platform uses the same NIfTI data loader and provides support for dataset coloring and windowing (tissue density manipulation). The fMRI data can be viewed changing over time by either animation through the time steps, like a movie, or using an interface slider to “scrub” through the different time steps of the data. The prototype applications data load times and frame rates were tested to determine if they achieved the real-time interaction goal. Real-time interaction was defined by achieving 10 frames per second (fps) or better, based on the work of Miller [1]. The desktop version was evaluated on a 2013 MacBook Pro running OS X 10.12 with a 2.6 GHz Intel Core i7 processor, 16 GB of RAM, and a NVIDIA GeForce GT 750M graphics card. The immersive application was tested in the C6 CAVE™, a 96 graphics node computer cluster comprised of NVIDIA Quadro 6000 graphics cards running Red Hat Enterprise Linux. The mobile application was evaluated on a 2016 9.7” iPad Pro running iOS 9.3.4. The iPad had a 64-bit Apple A9X dual core processor with 2 GB of built in memory. Two different fMRI brain activity datasets with different voxel resolutions were used as test datasets. Datasets were tested using both the 3D structural data, the 4D functional data, and a combination of the two. Frame rates for the desktop implementation were consistently above 10 fps, indicating that real-time 4D volume raycasting is possible on desktop hardware. The mobile and virtual reality platforms were able to perform real-time 3D volume raycasting consistently. This is a marked improvement for 3D mobile volume raycasting that was previously only able to achieve under one frame per second [2]. Both VR and mobile platforms were able to raycast the 4D only data at real-time frame rates, but did not consistently meet 10 fps when rendering both the 3D structural and 4D functional data simultaneously. However, 7 frames per second was the lowest frame rate recorded, indicating that hardware advances will allow consistent real-time raycasting of 4D fMRI data in the near future

Fusing Self-Reported and Sensor Data from Mixed-Reality Training

Military and industrial use of smaller, more accurate sensors are allowing increasing amounts of data to be acquired at diminishing costs during training. Traditional human subject testing often collects qualitative data from participants through self-reported questionnaires. This qualitative information is valuable but often incomplete to assess training outcomes. Quantitative information such as motion tracking data, communication frequency, and heart rate can offer the missing pieces in training outcome assessment. The successful fusion and analysis of qualitative and quantitative information sources is necessary for collaborative, mixed-reality, and augmented-reality training to reach its full potential. The challenge is determining a reliable framework combining these multiple types of data. Methods were developed to analyze data acquired during a formal user study assessing the use of augmented reality as a delivery mechanism for digital work instructions. A between-subjects experiment was conducted to analyze the use of a desktop computer, mobile tablet, or mobile tablet with augmented reality as a delivery method of these instructions. Study participants were asked to complete a multi-step technical assembly. Participants’ head position and orientation were tracked using an infrared tracking system. User interaction in the form of interface button presses was recorded and time stamped on each step of the assembly. A trained observer took notes on task performance during the study through a set of camera views that recorded the work area. Finally, participants each completed pre and post-surveys involving self-reported evaluation. The combination of quantitative and qualitative data revealed trends in the data such as the most difficult tasks across each device, which would have been impossible to determine from self-reporting alone. This paper describes the methods developed to fuse the qualitative data with quantified measurements recorded during the study

Comparing Visual Assembly Aids for Augmented Reality Work Instructions

Increased product complexity and the focus on zero defects, especially when manufacturing complex engineered products, means new tools are required for helping workers conduct challenging assembly tasks. Augmented reality (AR) has shown considerable promise in delivering work instructions over traditional methods. Many proof-of-concept systems have demonstrated the feasibility of AR but little work has been devoted to understanding how users perceive different AR work instruction interface elements. This paper presents a between-subjects study looking at how interface elements for object depth placement in a scene impact a user’s ability to quickly and accurately assemble a mock aircraft wing in a standard work cell. For object depth placement, modes with varying degrees of 3D modeled occlusion were tested, including a control group with no occlusion, virtual occlusion, and occlusion by contours. Results for total assembly time and total errors indicated no statistically significant difference between interfaces, leading the authors to conclude a floor has been reached for optimizing the current assembly when using AR for work instruction delivery. However, looking at a handful of highly error prone steps showed the impact different types of occlusion have on helping users correctly complete an assembly task. The results of the study provide insight into how to construct an interface for delivering AR work instructions using occlusion. Based on these results, the authors recommend customizing the occlusion method based on the features of the required assembly task. The authors also identified a floor effect for the steps of the assembly process, which involved picking the necessary parts from tables and bins. The authors recommend using vibrant outlines and large textual cues (e.g., numbers on parts bins) as interface elements to guide users during these types of “picking” steps