CHORUS Deliverable 2.2: Second report - identification of multi-disciplinary key issues for gap analysis toward EU multimedia search engines roadmap

After addressing the state-of-the-art during the first year of Chorus and establishing the existing landscape in multimedia search engines, we have identified and analyzed gaps within European research effort during our second year. In this period we focused on three directions, notably technological issues, user-centred issues and use-cases and socio- economic and legal aspects. These were assessed by two central studies: firstly, a concerted vision of functional breakdown of generic multimedia search engine, and secondly, a representative use-cases descriptions with the related discussion on requirement for technological challenges. Both studies have been carried out in cooperation and consultation with the community at large through EC concertation meetings (multimedia search engines cluster), several meetings with our Think-Tank, presentations in international conferences, and surveys addressed to EU projects coordinators as well as National initiatives coordinators. Based on the obtained feedback we identified two types of gaps, namely core technological gaps that involve research challenges, and “enablers”, which are not necessarily technical research challenges, but have impact on innovation progress. New socio-economic trends are presented as well as emerging legal challenges

Privacy Intelligence: A Survey on Image Sharing on Online Social Networks

Image sharing on online social networks (OSNs) has become an indispensable part of daily social activities, but it has also led to an increased risk of privacy invasion. The recent image leaks from popular OSN services and the abuse of personal photos using advanced algorithms (e.g. DeepFake) have prompted the public to rethink individual privacy needs when sharing images on OSNs. However, OSN image sharing itself is relatively complicated, and systems currently in place to manage privacy in practice are labor-intensive yet fail to provide personalized, accurate and flexible privacy protection. As a result, an more intelligent environment for privacy-friendly OSN image sharing is in demand. To fill the gap, we contribute a systematic survey of 'privacy intelligence' solutions that target modern privacy issues related to OSN image sharing. Specifically, we present a high-level analysis framework based on the entire lifecycle of OSN image sharing to address the various privacy issues and solutions facing this interdisciplinary field. The framework is divided into three main stages: local management, online management and social experience. At each stage, we identify typical sharing-related user behaviors, the privacy issues generated by those behaviors, and review representative intelligent solutions. The resulting analysis describes an intelligent privacy-enhancing chain for closed-loop privacy management. We also discuss the challenges and future directions existing at each stage, as well as in publicly available datasets.Comment: 32 pages, 9 figures. Under revie

Security and Privacy Preservation in Mobile Social Networks

Social networking extending the social circle of people has already become an important integral part of our daily lives. As reported by ComScore, social networking sites such as Facebook and Twitter have reached 82 percent of the world's online population, representing 1.2 billion users around the world. In the meantime, fueled by the dramatic advancements of smartphones and the ubiquitous connections of Bluetooth/WiFi/3G/LTE networks, social networking further becomes available for mobile users and keeps them posted on the up-to-date worldwide news and messages from their friends and families anytime anywhere. The convergence of social networking, advanced smartphones, and stable network infrastructures brings us a pervasive and omnipotent communication platform, named mobile social network (MSN), helping us stay connected better than ever. In the MSN, multiple communication techniques help users to launch a variety of applications in multiple communication domains including single-user domain, two-user domain, user-chain domain, and user-star domain. Within different communication domains, promising mobile applications are fostered. For example, nearby friend search application can be launched in the two-user or user-chain domains to help a user find other physically-close peers who have similar interests and preferences; local service providers disseminate advertising information to nearby users in the user-star domain; and health monitoring enables users to check the physiological signals in the single-user domain. Despite the tremendous benefits brought by the MSN, it still faces many technique challenges among of which security and privacy protections are the most important ones as smartphones are vulnerable to security attacks, users easily neglect their privacy preservation, and mutual trust relationships are difficult to be established in the MSN. In this thesis, we explore the unique characteristics and study typical research issues of the MSN. We conduct our research with a focus on security and privacy preservation while considering human factors. Specifically, we consider the profile matching application in the two-user domain, the cooperative data forwarding in the user-chain domain, the trustworthy service evaluation application in the user-star domain, and the healthcare monitoring application in the single-user domain. The main contributions are, i) considering the human comparison behavior and privacy requirements, we first propose a novel family of comparison-based privacy-preserving profile matching (PPM) protocols. The proposed protocols enable two users to obtain comparison results of attribute values in their profiles, while the attribute values are not disclosed. Taking user anonymity requirement as an evaluation metric, we analyze the anonymity protection of the proposed protocols. From the analysis, we found that the more comparison results are disclosed, the less anonymity protection is achieved by the protocol. Further, we explore the pseudonym strategy and an anonymity enhancing technique where users could be self-aware of the anonymity risk level and take appropriate actions when needed; ii) considering the inherent MSN nature --- opportunistic networking, we propose a cooperative privacy-preserving data forwarding (PDF) protocol to help users forward data to other users. We indicate that privacy and effective data forwarding are two conflicting goals: the cooperative data forwarding could be severely interrupted or even disabled when the privacy preservation of users is applied, because without sharing personal information users become unrecognizable to each other and the social interactions are no longer traceable. We explore the morality model of users from classic social theory, and use game-theoretic approach to obtain the optimal data forwarding strategy. Through simulation results, we show that the proposed cooperative data strategy can achieve both the privacy preservation and the forwarding efficiency; iii) to establish the trust relationship in a distributed MSN is a challenging task. We propose a trustworthy service evaluation (TSE) system, to help users exchange their service reviews toward local vendors. However, vendors and users could be the potential attackers aiming to disrupt the TSE system. We then consider the review attacks, i.e., vendors rejecting and modifying the authentic reviews of users, and the Sybil attacks, i.e., users abusing their pseudonyms to generate fake reviews. To prevent these attacks, we explore the token technique, the aggregate signature, and the secret sharing techniques. Simulation results show the security and the effectiveness of the TSE system can be guaranteed; iv) to improve the efficiency and reliability of communications in the single-user domain, we propose a prediction-based secure and reliable routing framework (PSR). It can be integrated with any specific routing protocol to improve the latter's reliability and prevent data injection attacks during data communication. We show that the regularity of body gesture can be learned and applied by body sensors such that the route with the highest predicted link quality can always be chose for data forwarding. The security analysis and simulation results show that the PSR significantly increases routing efficiency and reliability with or without the data injection attacks

Electric vehicle as a service (EVaaS):applications, challenges and enablers

Under the vehicle-to-grid (V2G) concept, electric vehicles (EVs) can be deployed as loads to absorb excess production or as distributed energy resources to supply part of their stored energy back to the grid. This paper overviews the technologies, technical components and system requirements needed for EV deployment. Electric vehicle as a service (EVaaS) exploits V2G technology to develop a system where suitable EVs within the distribution network are chosen individually or in aggregate to exchange energy with the grid, individual customers or both. The EVaaS framework is introduced, and interactions among EVaaS subsystems such as EV batteries, charging stations, loads and advanced metering infrastructure are studied. The communication infrastructure and processing facilities that enable data and information exchange between EVs and the grid are reviewed. Different strategies for EV charging/discharging and their impact on the distribution grid are reviewed. Several market designs that incentivize energy trading in V2G environments are discussed. The benefits of V2G are studied from the perspectives of ancillary services, supporting of renewables and the environment. The challenges to V2G are studied with respect to battery degradation, energy conversion losses and effects on distribution system

Privacy-preserving recommendation system using federated learning

Federated Learning is a form of distributed learning which leverages edge devices for training. It aims to preserve privacy by communicating users’ learning parameters and gradient updates to the global server during the training while keeping the actual data on the users’ devices. The training on global server is performed on these parameters instead of user data directly while fine tuning of the model can be done on client’s devices locally. However, federated learning is not without its shortcomings and in this thesis, we present an overview of the learning paradigm and propose a new federated recommender system framework that utilizes homomorphic encryption. This results in a slight decrease in accuracy metrics but leads to greatly increased user-privacy. We also show that performing computations on encrypted gradients barely affects the recommendation performance while ensuring a more secure means of communicating user gradients to and from the global server

From Social Data Mining to Forecasting Socio-Economic Crisis

Socio-economic data mining has a great potential in terms of gaining a better understanding of problems that our economy and society are facing, such as financial instability, shortages of resources, or conflicts. Without large-scale data mining, progress in these areas seems hard or impossible. Therefore, a suitable, distributed data mining infrastructure and research centers should be built in Europe. It also appears appropriate to build a network of Crisis Observatories. They can be imagined as laboratories devoted to the gathering and processing of enormous volumes of data on both natural systems such as the Earth and its ecosystem, as well as on human techno-socio-economic systems, so as to gain early warnings of impending events. Reality mining provides the chance to adapt more quickly and more accurately to changing situations. Further opportunities arise by individually customized services, which however should be provided in a privacy-respecting way. This requires the development of novel ICT (such as a self- organizing Web), but most likely new legal regulations and suitable institutions as well. As long as such regulations are lacking on a world-wide scale, it is in the public interest that scientists explore what can be done with the huge data available. Big data do have the potential to change or even threaten democratic societies. The same applies to sudden and large-scale failures of ICT systems. Therefore, dealing with data must be done with a large degree of responsibility and care. Self-interests of individuals, companies or institutions have limits, where the public interest is affected, and public interest is not a sufficient justification to violate human rights of individuals. Privacy is a high good, as confidentiality is, and damaging it would have serious side effects for society.Comment: 65 pages, 1 figure, Visioneer White Paper, see http://www.visioneer.ethz.c

Towards privacy-compliant mobile computing

Sophisticated mobile computing, sensing and recording devices like smartphones, smartwatches, and wearable cameras are carried by their users virtually around the clock, blurring the distinction between the online and offline worlds. While these devices enable transformative new applications and services, they also introduce entirely new threats to users’ privacy because they can capture a complete record of the user’s location, online and offline activities, and social encounters, including an audiovisual record. Such a record of users’ personal information is highly sensitive and is subject to numerous privacy risks. In this thesis, we have investigated and built systems to mitigate two such privacy risks: 1) privacy risks due to ubiquitous digital capture, where bystanders may inadvertently be captured in photos and videos recorded by other nearby users, 2) privacy risks to users’ personal information introduced by a popular class of apps called ‘mobile social apps’. In this thesis, we present two systems, called I-Pic and EnCore, built to mitigate these two privacy risks. Both systems aim to put the users back in control of what personal information is being collected and shared, while still enabling innovative new applications. We built working prototypes of both systems and evaluated them through actual user deployments. Overall we demonstrate that it is possible to achieve privacy-compliant digital capture and it is possible to build privacy-compliant mobile social apps, while preserving their intended functionality and ease-of-use. Furthermore, we also explore how the two solutions can be merged into a powerful combination, one which could enable novel workflows for specifying privacy preferences in image capture that do not currently exist.Die heutigen Geräte zur mobilen Kommunikation, und Messdatenerfassung und - aufzeichnung, wie Smartphones, Smartwatches und Sport-Kameras werden in der Regel von ihren Besitzern rund um die Uhr getragen, so daß der Unterschied zwischen Online- und Offline-Zeiten zunehmend verschwimmt. Diese Geräte ermöglichen zwar völlig neue Applikationen und Dienste, gefährden aber gleichzeitig die Privatsphäre ihrer Nutzer, weil sie den Standort, die gesamten On-und Offline Aktivitäten, sowie die soziale Beziehungen protokollieren, bis hin zu audio-visuellen Aufzeichnungen. Solche persönlichen Nutzerdaten sind extrem schützenswert und sind verschiedenen Risiken in Bezug auf die Privatsphäre ausgesetzt. In dieser These haben wir Systeme untersucht und gebaut, die zwei dieser Risiken für die Privatsphäre minimieren: 1) Risiko der Privatssphäre wegen omnipräsenter digitaler Aufzeichnungen Dritter, bei denen Unbeteiligte unbeabsichtigt (oder gegen ihren Wunsch) in Fotos und Videos festgehalten werden 2) Risiko für die persönlichen Informationen der Nutzer welche durch die bekannte Kategorie der sozialen Applikationen herbeigeführt werden. In dieser These stellen wir zwei Systeme, namens I-Pic und EnCore vor, welche die zwei Privatssphäre-Risiken minimieren. Beide System wollen dem Benutzer die Kontrolle zurückgeben, zu entscheiden welche seiner persönlichen Daten gesammelt und geteilt werden, während weiterhin neue innovative Applikationen ermöglicht werden. Wir haben für beide Systeme funktionsfähige Prototypen gebaut und diese mit echten Nutzerdaten evaluiert. Wir können generell zeigen dass es möglich ist, digitale Aufzeichnung zu machen, und soziale Applikationen zu bauen, welche nicht die Privatsphäre verletzen, ohne dabei die beabsichtige Funktionalität zu verlieren oder die Bedienbarkeit zu mindern. Des weiteren erforschen wir, wie diese zwei Systeme zu einem leistungsfähigeren Ansatz zusammengeführt werden können, welcher neuartige Workflows ermöglicht, um Einstellungen zur Privatsphäre für digitale Aufzeichnungen vorzunehmen, die es heute noch nicht gibt

Internet of Things Strategic Research Roadmap

Internet of Things (IoT) is an integrated part of Future Internet including existing and evolving Internet and network developments and could be conceptually defined as a dynamic global network infrastructure with self configuring capabilities based on standard and interoperable communication protocols where physical and virtual “things” have identities, physical attributes, and virtual personalities, use intelligent interfaces, and are seamlessly integrated into the information network

Privacy-Preserved Linkable Social-Physical Data Publication

In this dissertation, we investigate the privacy-preserved data publication problems towards pervasively existing linkable social-physical contents. On the one hand, data publication has been considered as a critical approach to facilitate numerous utilities for individuals, populations, platform owners, and all third-party service providers. On the other hand, the unprecedented adoption of mobile devices and the dramatic development of Internet-of-Thing (IoT) systems have pushed the collection of surrounding physical information among populations to a totally novel stage. The collected contents can provide a fine-grained access to both physical and social aspects of the crowds, which introduces a comprehensively linkable and potentially sensitive information domain. The linkage includes the related index like privacy, utility, and efficiency for sophisticated applications, the inherent correlations among multiple data sources or information dimensions, and the connections among individuals. As the linkage leads to various novel challenges for privacy preservation, there should be a body of novel mechanisms for linkable social-physical data publications. As a result, this dissertation proposes a series of mechanisms for privacy-preserved linkable social-physical data publication. Firstly, we study the publication of physical data where the co-existing useful social proles and the sensitive physical proles of the data should be carefully maintained. Secondly, we investigate the data publication problem jointly considering the privacy preservation, data utility, and resource efficiency for task completion in crowd-sensing systems. Thirdly, we investigate the publication of private contents used for the recommendation, where contents of a user contribute to the recommendation results for others. Fourthly, we study the publications of reviews in local business service systems, where users expect to conceal their frequently visited locations while cooperatively maintain the utility of the whole system. Fifthly, we study the acquisition of privacy-preserved knowledge on cyber-physical social networks, where third-party service providers can derive the community structure without accessing the sensitive social links. We also provide detailed analysis and discussion for proposed mechanisms, and extensively validate their performance via real-world datasets. Both results demonstrate that the proposed mechanisms can properly preserve the privacy while maintaining the data utility. At last, we also propose the future research topics to complete the whole dissertation. The first topic focuses on the privacy preservation towards correlations beneath multiple data sources. The second topic studies more privacy issues for the whole population during data publication, including both the novel threats for related communities, and the disclosure of trends within crowds