Dynamic Illumination for Augmented Reality with Real-Time Interaction

Current augmented and mixed reality systems suffer a lack of correct illumination modeling where the virtual objects render the same lighting condition as the real environment. While we are experiencing astonishing results from the entertainment industry in multiple media forms, the procedure is mostly accomplished offline. The illumination information extracted from the physical scene is used to interactively render the virtual objects which results in a more realistic output in real-time. In this paper, we present a method that detects the physical illumination with dynamic scene, then uses the extracted illumination to render the virtual objects added to the scene. The method has three steps that are assumed to be working concurrently in real-time. The first is the estimation of the direct illumination (incident light) from the physical scene using computer vision techniques through a 360° live-feed camera connected to AR device. The second is the simulation of indirect illumination (reflected light) from the real-world surfaces to virtual objects rendering using region capture of 2D texture from the AR camera view. The third is defining the virtual objects with proper lighting and shadowing characteristics using shader language through multiple passes. Finally, we tested our work with multiple lighting conditions to evaluate the accuracy of results based on the shadow falling from the virtual objects which should be consistent with the shadow falling from the real objects with a reduced performance cost

Polyarginine Molecular Weight Determines Transfecion Efficiency of Calcium Condensed Complexes

Cell penetrating peptides (CPPs) have been extensively studied in polyelectrolyte complexes as a means to enhance the transfection efficiency of plasmid DNA (pDNA). Increasing the molecular weight of CPPs often enhances gene expression, but poses a risk of increased cytotoxicity and immunogenicity compared to low molecular weight CCPs. Conversely, low molecular weight CPPs typically have low transfection efficiency due to large complex size. Complexes made using low molecular weight CPPs were found to be condensed to a small size by adding calcium. In this study, complexes of low molecular weight polyarginine and pDNA were condensed with calcium. These complexes showed high transfection efficiency and low cytotoxicity in A549 carcinomic human alveolar basal epithelial cells. The relationship between transfection efficiency and polyarginine size (5, 7, 9 or 11 amino acids), polyarginine/pDNA charge ratios, and calcium concentrations were studied. Polyarginine 7 was significantly more effective than other polyarginines under most formulation conditions suggesting a link between cell penetration ability and transfection efficiency

Noncovalently Associated Cell Penetrating Peptides for Nonviral Gene Delivery

Gene therapy has become a promising strategy for treatment of numerous diseases such as cancer, hemophilia, and neurodegenerative diseases. Glybera® (alipogene tiparvovec) became the first gene therapy approved in the European Union, for the treatment of lipoprotein lipase deficiency (LPLD). Promising late-stage clinical trials of this drug may herald the first gene therapy to be approved in the United States. The advancement of vectors (viral and nonviral) for efficient and safe gene delivery has garnered significant attention recently. Although viral vectors (e.g., retroviruses and adenoviruses) are the most effective vectors, applied in 70% of gene therapy clinical trials, they present several notable challenges including safety concerns (e.g., immunogenicity and pathogenicity), production difficulties, and rapid clearance from circulation. In contrast, nonviral vectors could be promising gene carriers due to their low cost, ease of synthesis, and decreased immunogenicity relative to viral vectors. Key progress has been made in the development of several nonviral gene delivery vectors. The use of cell penetrating peptides (CPPs) to deliver genetic materials for gene therapy has been a topic of interest for more than 20 years. One strategy is through covalent conjugation of CPPs with genetic materials, which requires complex synthesis procedures. In contrast, electrostatic complexation of CPPs with genetic materials is relatively simple and has been demonstrated to improve gene delivery both in vitro and in vivo. Reported herein, a simple method to generate small CPP complexes (100-200 nm) capable of high transfection efficiency was explored. Positively charged CPPs (e.g., polyarginine 9 [R9] and polylysine 9 [K9]) were complexed with plasmid DNA (pDNA), which resulted in unstable large particles (~1 micron). These were then condensed into small nanoparticles using Ca2+. CPPs also displayed negligible cytotoxicity. These CPP-pDNA-Ca2+ complexes showed high transfection efficiency and low cytotoxicity in vitro (human and mouse cell lines) and in vivo (syngeneic mice). Thus, Ca2+-condensed CPP complexes emerged as simple, attractive candidates for future studies on nonviral gene delivery. Futhermore, the relationships between transfection efficiency and polyarginine molecular weight, polyarginine-pDNA charge ratios, and calcium concentrations were studied. Polyarginine 7 was significantly more effective than other polyarginines under most formulation conditions, suggesting a link between molecular weight and transfection efficiency. Furthermore, Polylysine 9 was complexed with angiotensin II type 2 receptor (AT2R) plasmid DNA (pAT2R). The polylysine 9 complexe (K9-pAT2R-Ca2+) showed high transfection efficiency and negligible in vitro cytotoxicity towards human and mouse cell lines. This complex demonstrated cancer-targeted gene delivery in vivo when administered via intravenous injection or intratracheal spray. A single administration of this complex markedly attenuated lung cancer growth. Mechanistic understanding of CPP-mediated membrane insertion and intracellular translocation of nonviral gene complexes would allow rational design of next-generation CPPs for gene delivery. To this aim, we employed zwitterionic and anionic phospholipid monolayers as models to mimic the membrane composition of the outer leaflet of cell plasma and intracellular vesicular membranes at relevant intracellular pH. Subsequently, we investigated the membrane insertion potential of CPPs and gene complexes (CPP-pDNA-Ca2+ complexes) into model membranes. The insertion potential of CPPs and complexes were recorded using a Langmuir monolayer approach that records peptides and complexes adsorption to model membranes. Results showed that small changes to amino acids and peptide sequences resulted in dramatically different insertion potentials and membrane reorganization. Lastly, the effect of CPP charge type, charge spacing, and hydrophobicity on transfection efficiency was investigated by replacing three residues of the polyarginine 9 with a hydrophilic residue (histidine) or hydrophobic residues (alanine, leucine, and tryptophan) at positions 3, 4, and 7. R9 and RW9 complexes appeared especially effective compared to other CPP complexes, whereas RH9, RA9, and RL9 complexes seemed to have moderate- to low-gene expression. Initially, this suggested CPPs with better membrane penetration yielded higher gene expression. After further exploration, we discovered the charge spacing of CPPs affected the ability of CPPs to complex with nucleic acids and this property correlated to gene expression levels. In conclusion, our complexes emeraged as simple, attractive candidates for further in vivo studies on nonviral gene delivery

Dynamic Measurements of Membrane Insertion Potential of Synthetic Cell Penetrating Peptide/pDNA/Ca2+ Complexes

This is the published version. Copyright © 2015 Biophysical Society. Published by Elsevier Inc. All rights reserved.Noncovalent complexation of plasmid DNA (pDNA) using cell penetrating peptides (CPPs) has been less explored due to the relatively large complex size formed and the low-level gene expression. Here, condensing synthetic CPP polyplexes using CaCl2 produced small and stable complexes, which show higher level of in vitro gene expression. Anionic (i.e., POPS and POPG) or zwitterion (i.e., POPC) phospholipid monolayers at the air-water interface are used as model cell membranes to monitor the membrane insertion potential of synthetic CPPs. The insertion potential of complexes having different cationic (dTAT, H9, K9, R9, and RH9) and amphiphilic (RA9, RL9, and RW9) peptides were recorded using a Langmuir monolayer approach that records complexes adsorption to model membranes. Further, to mimic the pH of early endosome and late endosome and lysosome, phospholipid complex interactions were recorded at normal (pH 7.4) and low (pH 4.4) pH. All the complexes studied induced disruptions in phospholipid packing, which were most pronounced for the complexes having amphiphilic CPPs (i.e., RW9 and RL9). Particularly, the surface pressure of the complexes was significantly lower at normal pH when compared to acidic pH in the presence of POPC and POPS monolayers, except for RL9 and RW9 complexes. In contrast, the surface pressure of the complexes was significantly higher at normal pH when compared to acidic pH in the presence of POPG monolayer. Since the late endosomes contain an abundance of PC lipids and low pH, these results may be highly relevant to understand the efficiency of endosomal escape of these complexes

Extraction and Integration of Physical Illumination in Dynamic Augmented Reality Environments

Indiana University-Purdue University Indianapolis (IUPUI)Although current augmented, virtual, and mixed reality (AR/VR/MR) systems are facing advanced and immersive experience in the entertainment industry with countless media forms. Theses systems suffer a lack of correct direct and indirect illumination modeling where the virtual objects render with the same lighting condition as the real environment. Some systems are using baked GI, pre-recorded textures, and light probes that are mostly accomplished offline to compensate for precomputed real-time global illumination (GI). Thus, illumination information can be extracted from the physical scene for interactively rendering the virtual objects into the real world which produces a more realistic final scene in real-time. This work approaches the problem of visual coherence in AR by proposing a system that detects the real-world lighting conditions in dynamic scenes, then uses the extracted illumination information to render the objects added to the scene. The system covers several major components to achieve a more realistic augmented reality outcome. First, the detection of the incident light (direct illumination) from the physical scene with the use of computer vision techniques based on the topological structural analysis of 2D images using a live-feed 360-degree camera instrumented on an AR device that captures the entire radiance map. Also, the physics-based light polarization eliminates or reduces false-positive lights such as white surfaces, reflections, or glare which negatively affect the light detection process. Second, the simulation of the reflected light (indirect illumination) that bounce between the real-world surfaces to be rendered into the virtual objects and reflect their existence in the virtual world. Third, defining the shading characteristic/properties of the virtual object to depict the correct lighting assets with a suitable shadow casting. Fourth, the geometric properties of real-scene including plane detection, 3D surface reconstruction, and simple meshing are incorporated with the virtual scene for more realistic depth interactions between the real and virtual objects. These components are developed methods which assumed to be working simultaneously in real-time for photo-realistic AR. The system is tested with several lighting conditions to evaluate the accuracy of the results based on the error incurred between the real/virtual objects casting shadow and interactions. For system efficiency, the rendering time is compared with previous works and research. Further evaluation of human perception is conducted through a user study. The overall performance of the system is investigated to reduce the cost to a minimum

Real-time Illumination and Visual Coherence for Photorealistic Augmented/Mixed Reality

A realistically inserted virtual object in the real-time physical environment is a desirable feature in augmented reality (AR) applications and mixed reality (MR) in general. This problem is considered a vital research area in computer graphics, a field that is experiencing ongoing discovery. The algorithms and methods used to obtain dynamic and real-time illumination measurement, estimating, and rendering of augmented reality scenes are utilized in many applications to achieve a realistic perception by humans. We cannot deny the powerful impact of the continuous development of computer vision and machine learning techniques accompanied by the original computer graphics and image processing methods to provide a significant range of novel AR/MR techniques. These techniques include methods for light source acquisition through image-based lighting or sampling, registering and estimating the lighting conditions, and composition of global illumination. In this review, we discussed the pipeline stages with the details elaborated about the methods and techniques that contributed to the development of providing a photo-realistic rendering, visual coherence, and interactive real-time illumination results in AR/MR

Effect of Lipid Headgroup Charge and pH on the Stability and Membrane Insertion Potential of Calcium Condensed Gene Complexes

This document is the Accepted Manuscript version of a Published Work that appeared in final form in Langmuir, copyright © American Chemical Society after peer review and technical editing by the publisher. To access the final edited and published work see http://doi.org/10.1021/la504970n.Noncovalently condensed complexes of genetic material, cell penetrating peptides (CPPs), and calcium chloride present a nonviral route to improve transfection efficiency of nucleic acids (e.g., pDNA and siRNA). However, the exact mechanisms of membrane insertion and delivery of macromolecule complexes to intracellular locations as well as their stability in the intracellular environment are not understood. We show that calcium condensed gene complexes containing different hydrophilic (i.e., dTAT, K9, R9, and RH9) and amphiphilic (i.e., RA9, RL9, and RW9) CPPs formed stable cationic complexes of hydrodynamic radii 100 nm at neutral pH. However, increasing the acidity caused the complexes to become neutral or anionic and increase in size. Using zwitterionic and anionic phospholipid monolayers as models that mimic the membrane composition of the outer leaflet of cell membranes and intracellular vesicles and pHs that mimic the intracellular environment, we study the membrane insertion potential of these seven gene complexes (CPP/pDNA/Ca2+ complexes) into model membranes. At neutral pH, all gene complexes demonstrated the highest insertion potential into anionic phospholipid membranes, with complexes containing amphiphilic peptides showing the maximum insertion. However, at acidic pH, the gene complexes demonstrated maximum monolayer insertion into zwitterionic lipids, irrespective of the chemical composition of the CPP in the complexes. Our results suggest

Designing embodied interactions for informal learning: two open research challenges

Interactive installations that are controlled with gestures and body movements have been widely used in museums due to their tremendous educational potential. The design of such systems, however, remains problematic. In this paper, we reflect on two open research challenges that we observed when crafting a Kinect-based prototype installation for data exploration at a science museum: (1) making the user aware that the system is interactive; and, (2) increasing the discoverability of hand gestures and body movements

Image Denoising Using A Generative Adversarial Network

Animation studios render 3D scenes using a technique called path tracing which enables them to create high quality photorealistic frames. Path tracing involves shooting 1000's of rays into a pixel randomly (Monte Carlo) which will then hit the objects in the scene and, based on the reflective property of the object, these rays reflect or refract or get absorbed. The colors returned by these rays are averaged to determine the color of the pixel. This process is repeated for all the pixels. Due to the computational complexity it might take 8-16 hours to render a single frame. We implemented a neural network-based solution to reduce the time it takes to render a frame to less than a second using a generative adversarial network (GAN), once the network is trained. The main idea behind this proposed method is to render the image using a much smaller number of samples per pixel than is normal for path tracing (e.g., 1, 4, or 8 samples instead of, say, 32,000 samples) and then pass the noisy, incompletely rendered image to our network, which is capable of generating a high-quality photorealistic image

Show Me How You Interact, I Will Tell You What You Think: Exploring the Effect of the Interaction Style on Users’ Sensemaking about Correlation and Causation in Data

Findings from embodied cognition suggest that our whole body (not just our eyes) plays an important role in how we make sense of data when we interact with data visualizations. In this paper, we present the results of a study that explores how different designs of the ”interaction” (with a data visualization) alter the way in which people report and discuss correlation and causation in data. We conducted a lab study with two experimental conditions: Full body (participants interacted with a 65” display showing geo-referenced data using gestures and body movements); and, Gamepad (people used a joypad to control the system). Participants tended to agree less with statements that portray correlation and causation in data after using the Gamepad system. Additionally, discourse analysis based on Conceptual Metaphor Theory revealed that users made fewer remarks based on FORCE schemata in Gamepad than in Full-Body