GL-Socket: A CG plugin-based framework for teaching and assessment

In this paper we describe a plugin-based C++ framework for teaching OpenGL and GLSL in introductory Computer Graphics courses. The main strength of the framework architecture is that student assignments are mostly independent and thus can be completed, tested and evaluated in any order. When students complete a task, the plugin interface forces a clear separation of initialization, interaction and drawing code, which in turn facilitates code reusability. Plugin code can access scene, camera, and OpenGL window methods through a simple API. The plugin interface is flexible enough to allow students to complete tasks requiring shader development, object drawing, and multiple rendering passes. Students are provided with sample plugins with basic scene drawing and camera control features. One of the plugins that the students receive contains a shader development framework with self-assessment features. We describe the lessons learned after using the tool for four years in a Computer Graphics course involving more than one hundred Computer Science students per year.Peer ReviewedPostprint (published version

Syntactic and semantic analysis for extended feedback on computer graphics assignments

©2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Modern computer graphics courses require students to complete assignments involving computer programming. The evaluation of student programs, either by the student (self-assessment) or by the instructors (grading) can take a considerable amount of time and does not scale well with large groups. Interactive judges giving a pass/fail verdict do constitute a scalable solution, but they only provide feedback on output correctness. In this article, we present a tool to provide extensive feedback on student submissions. The feedback is based both on checking the output against test sets, as well as on syntactic and semantic analysis of the code. These analyses are performed through a set of code features and instructor-defined rubrics. The tool is built with Python and supports shader programs written in GLSL. Our experiments demonstrate that the tool provides extensive feedback that can be useful to support self-assessment, facilitate grading, and identify frequent programming mistakes.This work has been partially funded by the Spanish Ministry of Economy and Competitiveness and FEDER Grant TIN2017-88515-C2-1-R.Peer ReviewedPostprint (author's final draft

Gallery of interactive applications with 3D components

This thesis has the intention of exploring the potential of cutting-edge web technologies for creating immersive 3D experiences in the browser. This exploration will review diff erent available technologies, and end up taking the form of a gallery of interactive applications, evaluated for user experience and optimized for performance. The results demonstrate the potential of these technologies for web-based 3D graphics and interactive applications

High-Level GPU Programming: Domain-Specific Optimization and Inference

When writing computer software one is often forced to balance the need for high run-time performance with high programmer productivity. By using a high-level language it is often possible to cut development times, but this typically comes at the cost of reduced run-time performance. Using a lower-level language, programs can be made very efficient but at the cost of increased development time. Real-time computer graphics is an area where there are very high demands on both performance and visual quality. Typically, large portions of such applications are written in lower-level languages and also rely on dedicated hardware, in the form of programmable graphics processing units (GPUs), for handling computationally demanding rendering algorithms. These GPUs are parallel stream processors, specialized towards computer graphics, that have computational performance more than a magnitude higher than corresponding CPUs. This has revolutionized computer graphics and also led to GPUs being used to solve more general numerical problems, such as fluid and physics simulation, protein folding, image processing, and databases. Unfortunately, the highly specialized nature of GPUs has also made them difficult to program. In this dissertation we show that GPUs can be programmed at a higher level, while maintaining performance, compared to current lower-level languages. By constructing a domain-specific language (DSL), which provides appropriate domain-specific abstractions and user-annotations, it is possible to write programs in a more abstract and modular manner. Using knowledge of the domain it is possible for the DSL compiler to generate very efficient code. We show that, by experiment, the performance of our DSLs is equal to that of GPU programs written by hand using current low-level languages. Also, control over the trade-offs between visual quality and performance is retained. In the papers included in this dissertation, we present domain-specific languages targeted at numerical processing and computer graphics, respectively. These DSL have been implemented as embedded languages in Python, a dynamic programming language that provide a rich set of high-level features. In this dissertation we show how these features can be used to facilitate the construction of embedded languages

Ratcave: A 3D graphics python package for cognitive psychology experiments

We present here a free, open source Python 3D graphics library called Ratcave that extends existing Python psychology stimulus software by allowing scientists to load, display, and transform 3D stimuli created in 3D modeling software. This library makes 3D programming intuitive to new users by providing 3D graphics engine concepts (Mesh, Scene, Light, and Camera classes) that can be manipulated using an interface similar to existing 2D stimulus libraries. In addition, the use of modern OpenGL constructs by Ratcave helps scientists create fast, hardware-accelerated dynamic stimuli using the same intuitive high-level, lightweight interface. Because Ratcave supplements, rather than replaces, existing Python stimulus libraries, scientists can continue to use their preferred libraries by simply adding Ratcave graphics to their existing experiments. We hope this tool will be useful both as a stimulus library and as an example of how tightly-focused libraries can add quality to the existing scientific open-source software ecosystem. Cognitive psychology and neuroscience experiments use software that presents stimuli to a subject, detects subject responses, and logs events for future analysis, all with high temporal accuracy. An ever-expanding list of other features included in this software are compatibility with third-party hardware devices (e.g. button boxes, amplifiers, eye tracking systems), support for custom experimental designs, and online analysis for adaptive stimulus sequences; these tools are available both as self-enclosed software solutions (e.g. Neurobs Presentation, BCI2000, SuperLab, E-Prime) and open-source libraries (e.g. Psychtoolbox by Brainard, 1997; PsychoPy by Peirce, 2007; VisionEgg by Straw, 2008; Expyriment by Krause & Lindemann, 2013; for a review of psychophysics libraries, see Kötter, 2009). However, these popular libraries are missing 3D graphics support, needed for a wide range visual psychophysics experiments, such as 3D mental rotation or object recognition, virtual reality in spatial navigation research, to name a few. While 3D graphics libraries do exist in Python (e.g. Panda3D, PyOgre, Vizard) and other languages (e.g. Unity3D, Unreal Engine), the stimuli, logging, and hardware support of all of these libraries are designed to work with the windows and event loops they supply, making it difficult to integrate 3D graphics functionality into different psychophysics libraries without (sometimes-extensive) modification (e.g. to mix PsychoPy’s DotStim and Expyriment’s video support). In practice, this means that each software suite is relatively self-contained; researchers who require 3D stimuli, for example, have to, thereby, resort to use or develop different experiment control software when employing 3D visual stimuli (essentially, building interface to 3D game engines), losing out on the rich features that exist in the psychophysics software ecosystem developed for the 2D graphics. Extension libraries help reduce these feature-tradeoff decisions; for example, OpenSesame, a Python-powered GUI (Mathôt & Theeuwes, 2012), uses PsychoPy, Expyriment, and PyGame as “backends” to its experiment-building graphical interface, thereby supporting all researchers who rely on those libraries. A similar extension approach could be used for 3D stimuli--not to compete with the existing 3D frameworks on a feature-by-feature basis, but to simply add simple-to-use 3D stimulus presentation and manipulation support to the feature list of existing 2D stimulus libraries in Python. In this paper, we present an open-source, cross-platform Python library called Ratcave that adds 3D stimulus support to all OpenGL-based 2D Python stimulus libraries, including VisionEgg, Psychopy, Pyglet, and PyGame. We review the core features of Ratcave (https://github.com/ratcave/ratcave) and highlight key connections of its interface to underlying graphics programming strategies (a thorough manual, complete with API guide and tutorials for first-time users can be found at https://ratcave.readthedocs.org). This library, which derives its name from our high-speed RatcaveVR experimental setup (Del Grosso, Graboski, Chen, Hernández, & Sirota, 2017), is designed to increase accessibility of 3D graphics programming to the existing ecosystem of psychology software for Python

Towards Expressive and Versatile Visualization-as-a-Service (VaaS)

The rapid growth of data in scientific visualization has posed significant challenges to the scalability and availability of interactive visualization tools. These challenges can be largely attributed to the limitations of traditional monolithic applications in handling large datasets and accommodating multiple users or devices. To address these issues, the Visualization-as-a-Service (VaaS) architecture has emerged as a promising solution. VaaS leverages cloud-based visualization capabilities to provide on-demand and cost-effective interactive visualization. Existing VaaS has been simplistic by design with focuses on task-parallelism with single-user-per-device tasks for predetermined visualizations. This dissertation aims to extend the capabilities of VaaS by exploring data-parallel visualization services with multi-device support and hypothesis-driven explorations. By incorporating stateful information and enabling dynamic computation, VaaS\u27 performance and flexibility for various real-world applications is improved. This dissertation explores the history of monolithic and VaaS architectures, the design and implementations of 3 new VaaS applications, and a final exploration of the future of VaaS. This research contributes to the advancement of interactive scientific visualization, addressing the challenges posed by large datasets and remote collaboration scenarios

Dynamical hologram generation for high speed optical trapping of smart droplet microtools

This paper demonstrates spatially selective sampling of the plasma membrane by the implementation of time-multiplexed holographic optical tweezers for Smart Droplet Microtools (SDMs). High speed (>1000fps) dynamical hologram generation was computed on the graphics processing unit of a standard display card and controlled by a user friendly LabView interface. Time multiplexed binary holograms were displayed in real time and mirrored to a ferroelectric Spatial Light Modulator. SDMs were manufactured with both liquid cores (as previously described) and solid cores, which confer significant advantages in terms of stability, polydispersity and ease of use. These were coated with a number of detergents, the most successful based upon lipids doped with transfection reagents. In order to validate these, trapped SDMs were maneuvered up to the plasma membrane of giant vesicles containing Nile Red and human biliary epithelial (BE) colon cancer cells with green fluorescent labeled protein (GFP)-labeled CAAX (a motif belonging to the Ras protein). Bright field and fluorescence images showed that successful trapping and manipulation of multiple SDMs in x, y, z was achieved with success rates of 30-50% and that subsequent membrane-SDM interactions led to the uptake of Nile Red or GFP-CAAX into the SDM

Shape Display Shader Language (SDSL): a new programming model for shape changing displays

Shape-changing displays' dynamic physical affordances have inspired a range of novel hardware designs to support new types of interaction. Despite rapid technological progress, the community lacks a common programming model for developing applications for these visually and physically-dynamic display surfaces. This results in complex, hardware-specific, custom-code that requires significant development effort and prevents researchers from easily building on and sharing their applications across hardware platforms. As a first attempt to address these issues we introduce SDSL, a Shape-Display Shader Language for easily programming shape-changing displays in a hardware-independent manner. We introduce the (graphics-derived) pipeline model of SDSL, an open-source implementation that includes a compiler, runtime, IDE, debugger, and simulator, and show demonstrator applications running on two shape-changing hardware setups