CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Vision for Social Robots: Human Perception and Pose Estimation

In order to extract the underlying meaning from a scene captured from the surrounding world in a single still image, social robots will need to learn the human ability to detect different objects, understand their arrangement and relationships relative both to their own parts and to each other, and infer the dynamics under which they are evolving. Furthermore, they will need to develop and hold a notion of context to allow assigning different meanings (semantics) to the same visual configuration (syntax) of a scene. The underlying thread of this Thesis is the investigation of new ways for enabling interactions between social robots and humans, by advancing the visual perception capabilities of robots when they process images and videos in which humans are the main focus of attention. First, we analyze the general problem of scene understanding, as social robots moving through the world need to be able to interpret scenes without having been assigned a specific preset goal. Throughout this line of research, i) we observe that human actions and interactions which can be visually discriminated from an image follow a very heavy-tailed distribution; ii) we develop an algorithm that can obtain a spatial understanding of a scene by only using cues arising from the effect of perspective on a picture of a person’s face; and iii) we define a novel taxonomy of errors for the task of estimating the 2D body pose of people in images to better explain the behavior of algorithms and highlight their underlying causes of error. Second, we focus on the specific task of 3D human pose and motion estimation from monocular 2D images using weakly supervised training data, as accurately predicting human pose will open up the possibility of richer interactions between humans and social robots. We show that when 3D ground-truth data is only available in small quantities, or not at all, it is possible to leverage knowledge about the physical properties of the human body, along with additional constraints related to alternative types of supervisory signals, to learn models that can regress the full 3D pose of the human body and predict its motions from monocular 2D images. Taken in its entirety, the intent of this Thesis is to highlight the importance of, and provide novel methodologies for, social robots' ability to interpret their surrounding environment, learn in a way that is robust to low data availability, and generalize previously observed behaviors to unknown situations in a similar way to humans.</p

Detection of fake images generated by deep learning

During the last few years, the amount of audiovisual content produced is continually increasing with technology development. Along with this growth comes the availability of the same information through numerous devices that any individual holds, including smartphones, laptops, tablets, and smart TVs, in an entirely free and open manner. These type of content are considered an authenticity element since they represent a reality record. For example, in court, photos frequently determine the jury's course of action since what is available is a recorded picture that validates a narrative and usually does not leave room for doubts. However, with the advancement of Deep Learning (DL) algorithms, a new and dangerous trend known as Deepfakes begins to emerge. For example, a deepfake can be a video or an image of a person on which their face or body is totally or partially modified to appear to be someone else. This technique is often used for manipulation, blackmailing, and spreading false information. After recognizing such a dangerous problem, this study aims to uncover patterns that deepfakes show to identify authenticity as accurately as possible, using machine learning and deep learning algorithms. To get the highest level of accuracy, these algorithms were trained on datasets that included both real and phony photos. The outcomes demonstrate that deepfakes can be accurately identified and that the optimal model may be selected based on the specific requirements of the application.RESUMO: Nos últimos anos, a quantidade de conteúdo audiovisual produzido tem vindo a aumentar continuamente com o desenvolvimento da tecnologia. Juntamente com este crescimento, surge a disponibilidade da mesma informação através de inúmeros dispositivos que qualquer indivíduo possui, incluindo telemóveis, computadores, tablets e smart TVs, de uma forma totalmente livre e aberta. Este tipo de conteúdo é considerado um elemento de autenticidade, uma vez que representa um registo da realidade. Por exemplo, em tribunal, as fotografias frequentemente determinam a linha de ação do júri, uma vez que o que está disponível é uma imagem registada que valida uma narrativa e geralmente não deixa espaço para dúvidas. No entanto, com o avanço dos algoritmos de Deep Learning (DL), começa a surgir uma nova e perigosa tendência conhecida como Deepfakes. Por exemplo, um deepfake pode ser um vídeo ou uma imagem de uma pessoa na qual o rosto ou o corpo é totalmente ou parcialmente modificado para parecer ser outra pessoa. Esta técnica é frequentemente utilizada para manipulação, chantagem e disseminação de informações falsas. Após reconhecer um problema tão perigoso, este estudo tem como objetivo descobrir padrões que os Deepfakes apresentam para identificar a autenticidade da forma mais precisa possível, utilizando algoritmos de Machine Learning e Deep Learning. Estes algoritmos foram treinados utilizando conjuntos de dados que contenham tanto fotografias autênticas quanto falsas, a fim de obter o melhor nível de precisão. Os resultados obtidos mostram bons resultados na identificação de deepfakes e que a escolha do melhor modelo pode ser ajustada às necessidades da aplicação em causa

Consulting report – Bigmond S.A.

Bigmond is a Peruvian company dedicated to the headhunting services and human resources consulting. In the last months, the company has been facing a decrease on its commercial activities due to the fierce competition and, more recently, due to the COVID-19 pandemic. This situation has aware Bigmond of the necessity of reinvent an offer a service that target individuals rather than companies, this is why an outplacement service was thought as a suitable solution. Bigond also looks at the service as a way to keep its reputations as a anti-discriminatory company and expects that this new service could be offered to low and middle management job positions. Bigmond wants to achieve its objective by launching the service in the short-term but need a clear path of how to do it. The present thesis is intented to give Bigmond a detailed study with the best practices of how to implement the service. The thesis starts with an analysis of the Porter’s five forces and an overview of the external and internal factors affecting the company. Then, a literature review is presented in order to give a clear understanding of what is outplacement and its implications. Next, a benchmark of international and national companies that are currently offering the service and a survey were developed as the qualitative and quantitative analysis respectively. As result, a new business unit with a fully digital service through a platform was defined as the best alternative to implement. The project was estimated to last 76 working days and to have an initial cost of S/. 181,000.00. Finally, as outcomes, the projections showed that the company can achieve positive results in the first year after launching the service and to get positive reputational and brand awareness outcomes.Bigmond es una empresa peruana dedicada a los servicios de headhunting y consultoría de recursos humanos. En los últimos meses, la empresa se ha enfrentado a una disminución de sus actividades comerciales debido a la feroz competencia y, más recientemente, a la pandemia de COVID-19. Esta situación ha hecho consciente a Bigmond de la necesidad de reinventar una oferta de servicio dirigida a particulares más que a empresas, por eso se pensó en un servicio de recolocación como una solución adecuada. Bigmond también considera el servicio como una forma de mantener su reputación como empresa antidiscriminatoria y espera que este nuevo servicio pueda ofrecerse a puestos de trabajo de baja y media dirección. Bigmond quiere lograr su objetivo lanzando el servicio a corto plazo, pero necesita un camino claro de cómo hacerlo. La presente tesis tiene como objetivo darle a Bigmond un estudio detallado con las mejores prácticas de cómo implementar el servicio. La tesis comienza con un análisis de las cinco fuerzas de Porter y una descripción general de los factores externos e internos que afectan a la empresa. Luego, se presenta una revisión de la literatura con el fin de dar una comprensión clara de qué es la recolocación y sus implicaciones. A continuación, se desarrolló un benchmark de empresas internacionales y nacionales que actualmente ofrecen el servicio y una encuesta como análisis cualitativo y cuantitativo respectivamente. Como resultado, se definió como la mejor alternativa a implementar una nueva unidad de negocio con un servicio totalmente digital a través de una plataforma. Se estimó que el proyecto tendría una duración de 76 días hábiles y un costo inicial de S /. 181.000,00. Finalmente, como resultados, las proyecciones mostraron que la empresa puede lograr resultados positivos en el primer año después del lanzamiento del servicio y obtener resultados positivos de reputación y reconocimiento de marca