556 research outputs found

    20th SC@RUG 2023 proceedings 2022-2023

    Get PDF

    Neural Residual Radiance Fields for Streamably Free-Viewpoint Videos

    Full text link
    The success of the Neural Radiance Fields (NeRFs) for modeling and free-view rendering static objects has inspired numerous attempts on dynamic scenes. Current techniques that utilize neural rendering for facilitating free-view videos (FVVs) are restricted to either offline rendering or are capable of processing only brief sequences with minimal motion. In this paper, we present a novel technique, Residual Radiance Field or ReRF, as a highly compact neural representation to achieve real-time FVV rendering on long-duration dynamic scenes. ReRF explicitly models the residual information between adjacent timestamps in the spatial-temporal feature space, with a global coordinate-based tiny MLP as the feature decoder. Specifically, ReRF employs a compact motion grid along with a residual feature grid to exploit inter-frame feature similarities. We show such a strategy can handle large motions without sacrificing quality. We further present a sequential training scheme to maintain the smoothness and the sparsity of the motion/residual grids. Based on ReRF, we design a special FVV codec that achieves three orders of magnitudes compression rate and provides a companion ReRF player to support online streaming of long-duration FVVs of dynamic scenes. Extensive experiments demonstrate the effectiveness of ReRF for compactly representing dynamic radiance fields, enabling an unprecedented free-viewpoint viewing experience in speed and quality.Comment: Accepted by CVPR 2023. Project page, see https://aoliao12138.github.io/ReRF

    20th SC@RUG 2023 proceedings 2022-2023

    Get PDF

    Improving low latency applications for reconfigurable devices

    Get PDF
    This thesis seeks to improve low latency application performance via architectural improvements in reconfigurable devices. This is achieved by improving resource utilisation and access, and by exploiting the different environments within which reconfigurable devices are deployed. Our first contribution leverages devices deployed at the network level to enable the low latency processing of financial market data feeds. Financial exchanges transmit messages via two identical data feeds to reduce the chance of message loss. We present an approach to arbitrate these redundant feeds at the network level using a Field-Programmable Gate Array (FPGA). With support for any messaging protocol, we evaluate our design using the NASDAQ TotalView-ITCH, OPRA, and ARCA data feed protocols, and provide two simultaneous outputs: one prioritising low latency, and one prioritising high reliability with three dynamically configurable windowing methods. Our second contribution is a new ring-based architecture for low latency, parallel access to FPGA memory. Traditional FPGA memory is formed by grouping block memories (BRAMs) together and accessing them as a single device. Our architecture accesses these BRAMs independently and in parallel. Targeting memory-based computing, which stores pre-computed function results in memory, we benefit low latency applications that rely on: highly-complex functions; iterative computation; or many parallel accesses to a shared resource. We assess square root, power, trigonometric, and hyperbolic functions within the FPGA, and provide a tool to convert Python functions to our new architecture. Our third contribution extends the ring-based architecture to support any FPGA processing element. We unify E heterogeneous processing elements within compute pools, with each element implementing the same function, and the pool serving D parallel function calls. Our implementation-agnostic approach supports processing elements with different latencies, implementations, and pipeline lengths, as well as non-deterministic latencies. Compute pools evenly balance access to processing elements across the entire application, and are evaluated by implementing eight different neural network activation functions within an FPGA.Open Acces

    Temporal multimodal video and lifelog retrieval

    Get PDF
    The past decades have seen exponential growth of both consumption and production of data, with multimedia such as images and videos contributing significantly to said growth. The widespread proliferation of smartphones has provided everyday users with the ability to consume and produce such content easily. As the complexity and diversity of multimedia data has grown, so has the need for more complex retrieval models which address the information needs of users. Finding relevant multimedia content is central in many scenarios, from internet search engines and medical retrieval to querying one's personal multimedia archive, also called lifelog. Traditional retrieval models have often focused on queries targeting small units of retrieval, yet users usually remember temporal context and expect results to include this. However, there is little research into enabling these information needs in interactive multimedia retrieval. In this thesis, we aim to close this research gap by making several contributions to multimedia retrieval with a focus on two scenarios, namely video and lifelog retrieval. We provide a retrieval model for complex information needs with temporal components, including a data model for multimedia retrieval, a query model for complex information needs, and a modular and adaptable query execution model which includes novel algorithms for result fusion. The concepts and models are implemented in vitrivr, an open-source multimodal multimedia retrieval system, which covers all aspects from extraction to query formulation and browsing. vitrivr has proven its usefulness in evaluation campaigns and is now used in two large-scale interdisciplinary research projects. We show the feasibility and effectiveness of our contributions in two ways: firstly, through results from user-centric evaluations which pit different user-system combinations against one another. Secondly, we perform a system-centric evaluation by creating a new dataset for temporal information needs in video and lifelog retrieval with which we quantitatively evaluate our models. The results show significant benefits for systems that enable users to specify more complex information needs with temporal components. Participation in interactive retrieval evaluation campaigns over multiple years provides insight into possible future developments and challenges of such campaigns

    Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques

    Full text link
    Video, as a key driver in the global explosion of digital information, can create tremendous benefits for human society. Governments and enterprises are deploying innumerable cameras for a variety of applications, e.g., law enforcement, emergency management, traffic control, and security surveillance, all facilitated by video analytics (VA). This trend is spurred by the rapid advancement of deep learning (DL), which enables more precise models for object classification, detection, and tracking. Meanwhile, with the proliferation of Internet-connected devices, massive amounts of data are generated daily, overwhelming the cloud. Edge computing, an emerging paradigm that moves workloads and services from the network core to the network edge, has been widely recognized as a promising solution. The resulting new intersection, edge video analytics (EVA), begins to attract widespread attention. Nevertheless, only a few loosely-related surveys exist on this topic. The basic concepts of EVA (e.g., definition, architectures) were not fully elucidated due to the rapid development of this domain. To fill these gaps, we provide a comprehensive survey of the recent efforts on EVA. In this paper, we first review the fundamentals of edge computing, followed by an overview of VA. The EVA system and its enabling techniques are discussed next. In addition, we introduce prevalent frameworks and datasets to aid future researchers in the development of EVA systems. Finally, we discuss existing challenges and foresee future research directions. We believe this survey will help readers comprehend the relationship between VA and edge computing, and spark new ideas on EVA.Comment: 31 pages, 13 figure

    Geographic information extraction from texts

    Get PDF
    A large volume of unstructured texts, containing valuable geographic information, is available online. This information – provided implicitly or explicitly – is useful not only for scientific studies (e.g., spatial humanities) but also for many practical applications (e.g., geographic information retrieval). Although large progress has been achieved in geographic information extraction from texts, there are still unsolved challenges and issues, ranging from methods, systems, and data, to applications and privacy. Therefore, this workshop will provide a timely opportunity to discuss the recent advances, new ideas, and concepts but also identify research gaps in geographic information extraction

    Development of traceability solution for furniture components

    Get PDF
    Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáIn the contemporary context, characterized by intensified global competition and the constant evolution of the globalization landscape, it becomes imperative for industries, including Small and Medium Enterprises (SMEs), to undertake efforts to enhance their operational processes, often through digital technological adaptation. The present study falls within the scope of the project named “Wood Work 4.0,” which aims to infuse innovation into the wood furniture manufacturing industry through process optimization and the adoption of digital technologies. This project received funding from the European Union Development Fund, in collaboration with the North 2020 Regional Program, and was carried out at the Carpintaria Mofreita company, located in Macedo de Cavaleiros, Portugal. In this regard, this study introduces a software architecture that supports the traceability of projects in the wood furniture industry and simultaneously employs a system to identify and manage material leftovers, aiming for more efficient waste management. For the development of this software architecture, an approach that integrates the Fiware platform, specialized in systems for the Internet of Things (IoT), with an Application Programming Interface (API) specifically created to manage information about users, projects, and associated media files, was adopted. The material leftovers identification system employs image processing techniques to extract geometric characteristics of the materials. Additionally, these data are integrated into the company’s database. In this way, it was possible to develop an architecture that allows not only the capturing of project information but also its effective management. In the case of material leftovers identification, the system was able to establish, with a satisfactory degree of accuracy, the dimensions of the materials, enabling the insertion of these data into the company’s database for resource management and optimization.No contexto contemporâneo, marcado por uma competição global intensificada e pela constante evolução do cenário de globalização, torna-se imperativo para as indústrias, incluindo as Pequenas e Médias Empresas (PMEs), empreender esforços para aprimorar seus processos operacionais, frequentemente pela via da adaptação tecnológica digital. O presente estudo insere-se dentro do escopo do projeto denominado “Wood Work 4.0”, cujo propósito é infundir inovação na indústria de fabricação de móveis de madeira por meio da otimização de processos e da adoção de tecnologias digitais. Este projeto obteve financiamento do Fundo de Desenvolvimento da União Europeia, em colaboração com o programa Regional do Norte 2020 e foi realizado na empresa Carpintaria Mofreita, localizada em Macedo de Cavaleiros, Portugal. Nesse sentido, este estudo introduz uma arquitetura de software que oferece suporte à rastreabilidade de projetos na indústria de móveis de madeira, e simultaneamente emprega um sistema para identificar e gerenciar sobras de material, objetivando uma gestão de resíduos mais eficiente. Para o desenvolvimento dessa arquitetura de software, adotou-se uma abordagem que integra a plataforma Fiware, especializada em sistemas para a Internet das Coisas (IoT), com uma Interface de Programação de Aplicações (API) criada especificamente para gerenciar informações de usuários, projetos, e arquivos de mídia associados. O sistema de identificação de sobras de material emprega técnicas de processamento de imagem para extrair características geométricas dos materiais. Adicionalmente, esses dados são integrados ao banco de dados da empresa. Desta forma, foi possível desenvolver uma arquitetura que permite não só capturar informações de projetos, mas também gerenciá-las de forma eficaz. No caso da identificação de sobras de material, o sistema foi capaz de estabelecer, com um grau de precisão satisfatório, as dimensões dos materiais, possibilitando a inserção desses dados no banco de dados da empresa para gestão e otimização do uso de recursos

    20th SC@RUG 2023 proceedings 2022-2023

    Get PDF

    Applications

    Get PDF
    Volume 3 describes how resource-aware machine learning methods and techniques are used to successfully solve real-world problems. The book provides numerous specific application examples: in health and medicine for risk modelling, diagnosis, and treatment selection for diseases in electronics, steel production and milling for quality control during manufacturing processes in traffic, logistics for smart cities and for mobile communications
    corecore