Specialization Opportunities in Graphical Workloads

Computer games are complex performance-critical graphical applications which require specialized GPU hardware. For this reason, GPU drivers often include many heuristics to help optimize throughput. Recently however, new APIs are emerging which sacrifice many heuristics for lower-level hardware control and more predictable driver behavior. This shifts the burden for many optimizations from GPU driver developers to game programmers, but also provides numerous opportunities to exploit application-specific knowledge."br/""br/"This paper examines different opportunities for specializing GPU code and reducing redundant data transfers. Static analysis of commercial games shows that 5-18% of GPU code is specializable by pruning dead data elements or moving portions to different graphics pipeline stages. In some games, up to 97% of the programs’ data inputs of a particular type, namely uniform variables, are unused, as well as up to 62% of those in the GPU internal vertex-fragment interface. This shows potential for improving memory usage and communication overheads. Insome test scenarios, removing dead uniform data can lead to 6x performance improvements."br/""br/"We also explore the upper limits of specialization if all dynamic inputs are constant at run-time. For instance, if uniform inputs are constant, up to 44% of instructions can be eliminated in some games, with a further 14% becoming constant-foldable at compile time. Analysis of run-time traces, reveals that 48-91% of uniform inputs are constant in real games, so values close to the upper limit may be achieved in practice

Customizing Experiences for Mobile Virtual Reality

A criação manual de conteúdo para um jogo é um processo demorado e trabalhoso que requer um conjunto de habilidades diversi cado (normalmente designers, artistas e programadores) e a gestão de diferentes recursos (hardware e software especializados). Dado que o orçamento, tempo e recursos são frequentemente muito limitados, os projetos poderiam bene ciar de uma solução que permitisse poupar e investir noutros aspectos do desenvolvimento. No contexto desta tese, abordamos este desa o sugerindo a criação de pacotes especí cos para a geração de conteúdo per sonalizável, focados em aplicações de Realidade Virtual (RV) móveis. Esta abordagem divide o problema numa solução com duas facetas: em primeiro lugar, a Geração Procedural de Conteúdo, alcançada através de métodos convencionais e pela utilização inovadora de Grandes Modelos de Lin guagem (normalmente conhecidos por Large Language Models). Em segundo lugar, a Co-Criação de Conteúdo, que enfatiza o desenvolvimento colaborativo de conteúdo. Adicionalmente, dado que este trabalho se foca na compatibilidade com RV móvel, as limitações de hardware associadas a capacetes de RV autónomos (standalone VR Headsets) e formas de as ultrapassar são também abordadas. O conteúdo será gerado utilizando métodos actuais em geração procedural e facilitando a co-criação de conteúdo pelo utilizador. A utilização de ambas estas abordagens resulta em ambi entes, objectivos e conteúdo geral mais re-jogáveis com muito menos desenho. Esta abordagem está actualmente a ser aplicada no desenvolvimento de duas aplicações de RV distintas. A primeira, AViR, destina-se a oferecer apoio psicológico a indivíduos após a perda de uma gravidez. A se gunda, EmotionalVRSystem, visa medir as variações nas respostas emocionais dos participantes induzidas por alterações no ambiente, utilizando tecnologia EEG para leituras precisas

Streaming and 3D mapping of agri-data on mobile devices

Farm monitoring and operations generate heterogeneous AGRI-data from a variety of different sources that have the potential to be delivered to users ‘on the go’ and in the field to inform farm decision making. A software framework capable of interfacing with existing web mapping services to deliver in-field farm data on commodity mobile hardware was developed and tested. This raised key research challenges related to: robustness of data steaming methods under typical farm connectivity scenarios, and mapping and 3D rendering of AGRI-data in an engaging and intuitive way. The presentation of AGRI-data in a 3D and interactive context was explored using different visualisation techniques; currently the 2D presentation of AGRI- data is the dominant practice, despite the fact that mobile devices can now support sophisticated 3D graphics via programmable pipelines. The testing found that WebSockets were the most reliable streaming method for high resolution image/texture data. From our focus groups there was no single visualisation technique that was preferred demonstrating that a range of methods is a good way to satisfy a large user base. Improved 3D experience on mobile phones is set to revolutionize the multimedia market and a key challenge is identifying useful 3D visualisation methods and navigation tools that support the exploration of data driven 3D interactive visualisation frameworks for AGRI-data

FromDaDy : spreading aircraft trajectories across views to support iterative queries

International audienceWhen displaying thousands of aircraft trajectories on a screen, the visualization is spoiled by a tangle of trails. The visual analysis is therefore difficult, especially if a specific class of trajectories in an erroneous dataset has to be studied. We designed FromDaDy, a trajectory visualization tool that tackles the difficulties of exploring the visualization of multiple trails. This multidimensional data exploration is based on scatterplots, brushing, pick and drop, juxtaposed views and rapid visual design. Users can organize the workspace composed of multiple juxtaposed views. They can define the visual configuration of the views by connecting data dimensions from the dataset to Bertin's visual variables. They can then brush trajectories, and with a pick and drop operation they can spread the brushed information across views. They can then repeat these interactions, until they extract a set of relevant data, thus formulating complex queries. Through two real-world scenarios, we show how FromDaDy supports iterative queries and the extraction of trajectories in a dataset that contains up to 5 million data

Reinforcement Learning for Generative Art

Reinforcement learning (RL) is an efficient class of sequential decision-making algorithms that have achieved remarkable success in a broad range of applications, such as robotic manipulations, strategic games, or autonomous driving. The most well-known example of reinforcement learning is AlphaGo, a computer program that plays the board game Go and outperforms top human Go players. Unlike other two major machine learning categories, supervised learning and unsupervised learning, in which media artists are actively engaged, reinforcement learning has yet to result in many creative applications. Generative art is usually driven, in whole or in part, by autonomous systems that are derived from a set of rules. Interestingly, an RL policy can be seen as an autonomous system where the rules are learned by interacting with its environment. Regardless of its initial purpose, reinforcement learning has the potential to expand the boundary of generative art. However, a formal process of applying reinforcement learning to generative art does not yet exist and the current RL tools require an in-depth understanding of RL concepts. To bridge the gap, the first part of the dissertation introduces a conceptual framework to adapt reinforcement learning for generative art. The framework proposes a term RL-based generative art to denote a novel form of generative art of which the use of RL agents is the key element. The creative process of RL-based generative art and possible emergent behaviors are discussed in the framework. This leads to a discussion of several author's related practices on generative art, deep-learning art, and reinforcement learning. Those practices are critical for understanding the conceptual and technical details of each component in order to construct the framework. The second part introduces RL5, a JavaScript library for rapidly prototyping RL environments and training RL policies in web browsers. The library combines RL algorithms and RL environments into one framework and is fully compatible with p5.js. RL5 is developed with a particular focus on simplicity to favor (re)usability of RL algorithms and development of RL environments. Specifically, the library implemented three RL algorithms, Tabular Q-learning, REINFORCE, and DDPG, to cover all the three families of model-free RL, and nine RL environments that six of them address autonomous agents in steering behaviors, which can be used as building blocks for complex systems. Finally, the author demonstrates four different use cases of how to apply RL5 for pedagogical and creative applications

Computational methods and software for the design of inertial microfluidic flow sculpting devices

The ability to sculpt inertially flowing fluid via bluff body obstacles has enormous promise for applications in bioengineering, chemistry, and manufacturing within microfluidic devices. However, the computational difficulty inherent to full scale 3-dimensional fluid flow simulations makes designing and optimizing such systems tedious, costly, and generally tasked to computational experts with access to high performance resources. The goal of this work is to construct efficient models for the design of inertial microfluidic flow sculpting devices, and implement these models in freely available, user-friendly software for the broader microfluidics community. Two software packages were developed to accomplish this: uFlow and FlowSculpt . uFlow solves the forward problem in flow sculpting, that of predicting the net deformation from an arbitrary sequence of obstacles (pillars), and includes estimations of transverse mass diffusion and particles formed by optical lithography. FlowSculpt solves the more difficult inverse problem in flow sculpting, which is to design a flow sculpting device which produces a target flow shape. Each piece of software uses efficient, experimentally validated forward models developed within this work, which are applied to deep learning techniques to explore other routes to solving the inverse problem. The models are also highly modular, capable of incorporating new microfluidic components and flow physics to the design process. It is anticipated that the microfluidics community will integrate the tools developed here into their own research, and bring new designs, components, and applications to the inertial flow sculpting platform

Navigating the Landscape for Real-time Localisation and Mapping for Robotics, Virtual and Augmented Reality

Visual understanding of 3D environments in real-time, at low power, is a huge computational challenge. Often referred to as SLAM (Simultaneous Localisation and Mapping), it is central to applications spanning domestic and industrial robotics, autonomous vehicles, virtual and augmented reality. This paper describes the results of a major research effort to assemble the algorithms, architectures, tools, and systems software needed to enable delivery of SLAM, by supporting applications specialists in selecting and configuring the appropriate algorithm and the appropriate hardware, and compilation pathway, to meet their performance, accuracy, and energy consumption goals. The major contributions we present are (1) tools and methodology for systematic quantitative evaluation of SLAM algorithms, (2) automated, machine-learning-guided exploration of the algorithmic and implementation design space with respect to multiple objectives, (3) end-to-end simulation tools to enable optimisation of heterogeneous, accelerated architectures for the specific algorithmic requirements of the various SLAM algorithmic approaches, and (4) tools for delivering, where appropriate, accelerated, adaptive SLAM solutions in a managed, JIT-compiled, adaptive runtime context.Comment: Proceedings of the IEEE 201

Visualisation of Large-Scale Call-Centre Data

The contact centre industry employs 4% of the entire United King-dom and United States’ working population and generates gigabytes of operational data that require analysis, to provide insight and to improve efficiency. This thesis is the result of a collaboration with QPC Limited who provide data collection and analysis products for call centres. They provided a large data-set featuring almost 5 million calls to be analysed. This thesis utilises novel visualisation techniques to create tools for the exploration of the large, complex call centre data-set and to facilitate unique observations into the data.A survey of information visualisation books is presented, provid-ing a thorough background of the field. Following this, a feature-rich application that visualises large call centre data sets using scatterplots that support millions of points is presented. The application utilises both the CPU and GPU acceleration for processing and filtering and is exhibited with millions of call events.This is expanded upon with the use of glyphs to depict agent behaviour in a call centre. A technique is developed to cluster over-lapping glyphs into a single parent glyph dependant on zoom level and a customizable distance metric. This hierarchical glyph repre-sents the mean value of all child agent glyphs, removing overlap and reducing visual clutter. A novel technique for visualising individually tailored glyphs using a Graphics Processing Unit is also presented, and demonstrated rendering over 100,000 glyphs at interactive frame rates. An open-source code example is provided for reproducibility.Finally, a novel interaction and layout method is introduced for improving the scalability of chord diagrams to visualise call transfers. An exploration of sketch-based methods for showing multiple links and direction is made, and a sketch-based brushing technique for filtering is proposed. Feedback from domain experts in the call centre industry is reported for all applications developed