CHORUS Deliverable 2.2: Second report - identification of multi-disciplinary key issues for gap analysis toward EU multimedia search engines roadmap

After addressing the state-of-the-art during the first year of Chorus and establishing the existing landscape in multimedia search engines, we have identified and analyzed gaps within European research effort during our second year. In this period we focused on three directions, notably technological issues, user-centred issues and use-cases and socio- economic and legal aspects. These were assessed by two central studies: firstly, a concerted vision of functional breakdown of generic multimedia search engine, and secondly, a representative use-cases descriptions with the related discussion on requirement for technological challenges. Both studies have been carried out in cooperation and consultation with the community at large through EC concertation meetings (multimedia search engines cluster), several meetings with our Think-Tank, presentations in international conferences, and surveys addressed to EU projects coordinators as well as National initiatives coordinators. Based on the obtained feedback we identified two types of gaps, namely core technological gaps that involve research challenges, and “enablers”, which are not necessarily technical research challenges, but have impact on innovation progress. New socio-economic trends are presented as well as emerging legal challenges

Privacy Intelligence: A Survey on Image Sharing on Online Social Networks

Image sharing on online social networks (OSNs) has become an indispensable part of daily social activities, but it has also led to an increased risk of privacy invasion. The recent image leaks from popular OSN services and the abuse of personal photos using advanced algorithms (e.g. DeepFake) have prompted the public to rethink individual privacy needs when sharing images on OSNs. However, OSN image sharing itself is relatively complicated, and systems currently in place to manage privacy in practice are labor-intensive yet fail to provide personalized, accurate and flexible privacy protection. As a result, an more intelligent environment for privacy-friendly OSN image sharing is in demand. To fill the gap, we contribute a systematic survey of 'privacy intelligence' solutions that target modern privacy issues related to OSN image sharing. Specifically, we present a high-level analysis framework based on the entire lifecycle of OSN image sharing to address the various privacy issues and solutions facing this interdisciplinary field. The framework is divided into three main stages: local management, online management and social experience. At each stage, we identify typical sharing-related user behaviors, the privacy issues generated by those behaviors, and review representative intelligent solutions. The resulting analysis describes an intelligent privacy-enhancing chain for closed-loop privacy management. We also discuss the challenges and future directions existing at each stage, as well as in publicly available datasets.Comment: 32 pages, 9 figures. Under revie

Multiple-view product representation and development using augmented reality technology

Ph.DDOCTOR OF PHILOSOPH

Development of traceability solution for furniture components

Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáIn the contemporary context, characterized by intensified global competition and the constant evolution of the globalization landscape, it becomes imperative for industries, including Small and Medium Enterprises (SMEs), to undertake efforts to enhance their operational processes, often through digital technological adaptation. The present study falls within the scope of the project named “Wood Work 4.0,” which aims to infuse innovation into the wood furniture manufacturing industry through process optimization and the adoption of digital technologies. This project received funding from the European Union Development Fund, in collaboration with the North 2020 Regional Program, and was carried out at the Carpintaria Mofreita company, located in Macedo de Cavaleiros, Portugal. In this regard, this study introduces a software architecture that supports the traceability of projects in the wood furniture industry and simultaneously employs a system to identify and manage material leftovers, aiming for more efficient waste management. For the development of this software architecture, an approach that integrates the Fiware platform, specialized in systems for the Internet of Things (IoT), with an Application Programming Interface (API) specifically created to manage information about users, projects, and associated media files, was adopted. The material leftovers identification system employs image processing techniques to extract geometric characteristics of the materials. Additionally, these data are integrated into the company’s database. In this way, it was possible to develop an architecture that allows not only the capturing of project information but also its effective management. In the case of material leftovers identification, the system was able to establish, with a satisfactory degree of accuracy, the dimensions of the materials, enabling the insertion of these data into the company’s database for resource management and optimization.No contexto contemporâneo, marcado por uma competição global intensificada e pela constante evolução do cenário de globalização, torna-se imperativo para as indústrias, incluindo as Pequenas e Médias Empresas (PMEs), empreender esforços para aprimorar seus processos operacionais, frequentemente pela via da adaptação tecnológica digital. O presente estudo insere-se dentro do escopo do projeto denominado “Wood Work 4.0”, cujo propósito é infundir inovação na indústria de fabricação de móveis de madeira por meio da otimização de processos e da adoção de tecnologias digitais. Este projeto obteve financiamento do Fundo de Desenvolvimento da União Europeia, em colaboração com o programa Regional do Norte 2020 e foi realizado na empresa Carpintaria Mofreita, localizada em Macedo de Cavaleiros, Portugal. Nesse sentido, este estudo introduz uma arquitetura de software que oferece suporte à rastreabilidade de projetos na indústria de móveis de madeira, e simultaneamente emprega um sistema para identificar e gerenciar sobras de material, objetivando uma gestão de resíduos mais eficiente. Para o desenvolvimento dessa arquitetura de software, adotou-se uma abordagem que integra a plataforma Fiware, especializada em sistemas para a Internet das Coisas (IoT), com uma Interface de Programação de Aplicações (API) criada especificamente para gerenciar informações de usuários, projetos, e arquivos de mídia associados. O sistema de identificação de sobras de material emprega técnicas de processamento de imagem para extrair características geométricas dos materiais. Adicionalmente, esses dados são integrados ao banco de dados da empresa. Desta forma, foi possível desenvolver uma arquitetura que permite não só capturar informações de projetos, mas também gerenciá-las de forma eficaz. No caso da identificação de sobras de material, o sistema foi capaz de estabelecer, com um grau de precisão satisfatório, as dimensões dos materiais, possibilitando a inserção desses dados no banco de dados da empresa para gestão e otimização do uso de recursos

Holistic indoor scene understanding, modelling and reconstruction from single images.

3D indoor scene understanding in computer vision refers to perceiving the semantic and geometric information in a 3D indoor environment from partial observations (e.g. images or depth scans). Semantics in a scene generally involves the conceptual knowledge such as the room layout, object categories, and their interrelationships (e.g. support relationship). These scene semantics are usually coupled with object and room geometry for 3D scene understanding, for example, layout plan (i.e. location of walls, ceiling and floor), shape of in-room objects, and a camera pose of observer. This thesis focuses on the problem of holistic 3D scene understanding from single images to model or reconstruct the in- door geometry with enriched scene semantics. This challenging task requires computers to perform equivalently as human vision system to perceive and understand indoor contents from colour intensities. Existing works either focus on a sub-problem (e.g. layout estimation, 3D detection or object reconstruction), or ad- dressing this entire problem with independent subtasks, while this thesis aims to an integrated and unified solution toward semantic scene understanding and reconstruction. In this thesis, scene semantics and geometry are regarded inter- twined and complementary. Understanding each part (semantics or geometry) helps to perceive the other one, which enables joint scene understanding, modelling & reconstruction. We start by the problem of semantic scene modelling. To estimate the object semantics and shapes from a single image, a feasible scene modelling streamline is proposed. It is backboned with fully convolutional networks to learn 2D semantics and geometry, and powered by a top-down shape retrieval for object modelling. After this, We build a unified and more efficient visual system for semantic scene modelling. Scene semantics are divided into relational (i.e. support relationship) and non-relational (i.e. object segmentation & geometry, room layout) knowledge. A Relation Network is proposed to estimate the support relations between objects to guide the object modelling process. Afterwards, We focus on the problem of holistic and end-to-end scene understanding and reconstruction. Instead of modelling scenes by top-down shape retrieval, this method bridges the gap between scene understanding and object mesh reconstruction. It does not rely on any external CAD repositories. Camera poses, room lay- out, object bounding boxes and meshes are end-to-end predicted from an RGB image with a single network architecture. At the end, We extend our work by using a different input modality, single-view depth scan, to explore the object reconstruction performance. A skeleton-bridged approach is proposed to predict the meso-skeleton of shapes as an intermediate representation to guide surface reconstruction, which outperforms the prior-arts in shape completion. Overall, this thesis provides a series of novel approaches towards holistic 3D indoor scene understanding, modelling and reconstruction. It aims at automatic 3D scene perception that enables machines to understand and predict 3D contents as human vision, which we hope could advance the boundaries of 3D vision in machine perception, robotics and Artificial Intelligence

Excellentia Eminentia Effectio

"In these pages you will learn about the fascinating research endeavors that each of our faculty members is undertaking. We have divided their research into the broad categories of health, sustainability, information, and systems. While we recognize the imperfect nature of categorizing research that, by its very nature may be interdisciplinary or transdisciplinary, we nonetheless believe it will be helpful as a way to see the depth and breadth of our research endeavors within each grouping. As you read the profiles on these pages, I know you will begin to appreciate that, taken as a whole, the research spectrum at Columbia Engineering is exceptional and that, as our professors go about their work, they are at the cusp of making breakthroughs that will have a major impact on the way we live our lives today and tomorrow.

Graphics Technology in Space Applications (GTSA 1989)

This document represents the proceedings of the Graphics Technology in Space Applications, which was held at NASA Lyndon B. Johnson Space Center on April 12 to 14, 1989 in Houston, Texas. The papers included in these proceedings were published in general as received from the authors with minimum modifications and editing. Information contained in the individual papers is not to be construed as being officially endorsed by NASA