Conversational Agent Experience: How to Create Good Alexa Skill

Conversational Design guidelines offer recommendations on how to lead the user-agent conversation, how to help customers achieve their goals, and how to handle the mistakes caused by each side. However, the effective methodology to evaluate the experience of user-agent conversation is unclear. Here we show a data pipeline that evaluates the user-agent experience on a variety of scenarios. We found that the coherence of Alexa's response has a positive impact on user's experience, which is based on the categories of skills, the number of slots in utterances, and the goals that users are trying to achieve. Furthermore, our study shows a gap between the theoretical conversational design guideline and the needs for practical testing for CA. Our data pipeline demonstrates the importance of testing experience by measurements that cast positive or negative affect on conversational experience. We anticipate our study to be a starting point for a more robust user experience evaluating system for CA and related applications.Master of Science in InformationSchool of Informationhttp://deepblue.lib.umich.edu/bitstream/2027.42/168558/1/20210104_Zhou,Xunan[Andy]_Final_MTOP_Thesis.pd

Ideas Matchmaking for Supporting Innovators and Entrepreneurs

Käesolevas töös esitletakse süsteemi, mis on võimeline sirvima veebist ettevõtluse ja tehnoloogiaga seotud andmeid, mida saab siduda kasutajate poolt Innovvoice platvormil välja pakutud ideedega. Selline teenus on ideabator platvormi väärtuslik osa, mis toetab ettevõtluse uuendajaid ja potentsiaalseid ettevõtjaid.In this paper we show a system able to crawl content from the Web related to entrepreneurship and technology, to be matched with ideas proposed by users in the Innovvoice platform. We argue that such a service is a valuable component of an ideabator platform, supporting innovators and possible entrepreneurs

Finding the Needle in a Haystack: On the Automatic Identification of Accessibility User Reviews

In recent years, mobile accessibility has become an important trend with the goal of allowing all users the possibility of using any app without many limitations. User reviews include insights that are useful for app evolution. However, with the increase in the amount of received reviews, manually analyzing them is tedious and time-consuming, especially when searching for accessibility reviews. The goal of this paper is to support the automated identification of accessibility in user reviews, to help technology professionals in prioritizing their handling, and thus, creating more inclusive apps. Particularly, we design a model that takes as input accessibility user reviews, learns their keyword-based features, in order to make a binary decision, for a given review, on whether it is about accessibility or not. The model is evaluated using a total of 5,326 mobile app reviews. The findings show that (1) our model can accurately identify accessibility reviews, outperforming two baselines, namely keyword-based detector and a random classifier; (2) our model achieves an accuracy of 85% with relatively small training dataset; however, the accuracy improves as we increase the size of the training dataset

CHORUS Deliverable 2.2: Second report - identification of multi-disciplinary key issues for gap analysis toward EU multimedia search engines roadmap

After addressing the state-of-the-art during the first year of Chorus and establishing the existing landscape in multimedia search engines, we have identified and analyzed gaps within European research effort during our second year. In this period we focused on three directions, notably technological issues, user-centred issues and use-cases and socio- economic and legal aspects. These were assessed by two central studies: firstly, a concerted vision of functional breakdown of generic multimedia search engine, and secondly, a representative use-cases descriptions with the related discussion on requirement for technological challenges. Both studies have been carried out in cooperation and consultation with the community at large through EC concertation meetings (multimedia search engines cluster), several meetings with our Think-Tank, presentations in international conferences, and surveys addressed to EU projects coordinators as well as National initiatives coordinators. Based on the obtained feedback we identified two types of gaps, namely core technological gaps that involve research challenges, and “enablers”, which are not necessarily technical research challenges, but have impact on innovation progress. New socio-economic trends are presented as well as emerging legal challenges

On the Security and Privacy Challenges in Android-based Environments

In the last decade, we have faced the rise of mobile devices as a fundamental tool in our everyday life. Currently, there are above 6 billion smartphones, and 72% of them are Android devices. The functionalities of smartphones are enriched by mobile apps through which users can perform operations that in the past have been made possible only on desktop/laptop computing. Besides, users heavily rely on them for storing even the most sensitive information from a privacy point of view. However, apps often do not satisfy all minimum security requirements and can be targeted to indirectly attack other devices managed or connected to them (e.g., IoT nodes) that may perform sensitive operations such as health checks, control a smart car or open a smart lock. This thesis discusses some research activities carried out to enhance the security and privacy of mobile apps by i) proposing novel techniques to detect and mitigate security vulnerabilities and privacy issues, and ii) defining techniques devoted to the security evaluation of apps interacting with complex environments (e.g., mobile-IoT-Cloud). In the first part of this thesis, I focused on the security and privacy of Mobile Apps. Due to the widespread adoption of mobile apps, it is relatively straightforward for researchers or users to quickly retrieve the app that matches their tastes, as Google provides a reliable search engine. However, it is likewise almost impossible to select apps according to a security footprint (e.g., all apps that enforce SSL pinning). To overcome this limitation, I present APPregator, a platform that allows users to select apps according to a specific security footprint. This tool aims to implement state-of-the-art static and dynamic analysis techniques for mobile apps and provide security researchers and analysts with a tool that makes it possible to search for mobile applications under specific functional or security requirements. Regarding the security status of apps, I studied a particular context of mobile apps: hybrid apps composed of web technologies and native technologies (i.e., Java or Kotlin). In this context, I studied a vulnerability that affected only hybrid apps: the Frame Confusion. This vulnerability, despite being discovered several years ago, it is still very widespread. I proposed a methodology implemented in FCDroid that exploits static and dynamic analysis techniques to detect and trigger the vulnerability automatically. The results of an extensive analysis carried out through FCDroid on a set of the most downloaded apps from the Google Play Store prove that 6.63% (i.e., 1637/24675) of hybrid apps are potentially vulnerable to Frame Confusion. A side effect of the analysis I carried out through APPregator was suggesting that very few apps may have a privacy policy, despite Google Play Store imposes some strict rules about it and contained in the Google Play Privacy Guidelines. To empirically verify if that was the case, I proposed a methodology based on the combination of static analysis, dynamic analysis, and machine learning techniques. The proposed methodology verifies whether each app contains a privacy policy compliant with the Google Play Privacy Guidelines, and if the app accesses privacy-sensitive information only upon the acceptance of the policy by the user. I then implemented the methodology in a tool, 3PDroid, and evaluated a number of recent and most downloaded Android apps in the Google Play Store. Experimental results suggest that over 95% of apps access sensitive user privacy information, but only a negligible subset of it (~ 1%) fully complies with the Google Play Privacy Guidelines. Furthermore, the obtained results have also suggested that the user privacy could be put at risk by mobile apps that keep collecting a plethora of information regarding the user's and the device behavior by relying on third-party analytics libraries. However, collecting and using such data raised several privacy concerns, mainly because the end-user - i.e., the actual data owner - is out of the loop in this collection process. The existing privacy-enhanced solutions that emerged in the last years follow an ``all or nothing" approach, leaving to the user the sole option to accept or completely deny access to privacy-related data. To overcome the current state-of-the-art limitations, I proposed a data anonymization methodology, called MobHide, that provides a compromise between the usefulness and privacy of the data collected and gives the user complete control over the sharing process. For evaluating the methodology, I implemented it in a prototype called HideDroid and tested it on 4500 most-used Android apps of the Google Play Store between November 2020 and January 2021. In the second part of this thesis, I extended privacy and security considerations outside the boundary of the single mobile device. In particular, I focused on two scenarios. The first is composed of an IoT device and a mobile app that have a fruitful integration to resolve and perform specific actions. From a security standpoint, this leads to a novel and unprecedented attack surface. To deal with such threats, applying state-of-the-art security analysis techniques on each paradigm can be insufficient. I claimed that novel analysis methodologies able to systematically analyze the ecosystem as a whole must be put forward. To this aim, I introduced the idea of APPIoTTe, a novel approach to the security testing of Mobile-IoT hybrid ecosystems, as well as some notes on its implementation working on Android (Mobile) and Android Things (IoT) applications. The second scenario is composed of an IoT device widespread in the Smart Home environment: the Smart Speaker. Smart speakers are used to retrieving information, interacting with other devices, and commanding various IoT nodes. To this aim, smart speakers typically take advantage of cloud architectures: vocal commands of the user are sampled, sent through the Internet to be processed, and transmitted back for local execution, e.g., to activate an IoT device. Unfortunately, even if privacy and security are enforced through state-of-the-art encryption mechanisms, the features of the encrypted traffic, such as the throughput, the size of protocol data units, or the IP addresses, can leak critical information about the users' habits. In this perspective, I showcase this kind of risk by exploiting machine learning techniques to develop black-box models to classify traffic and implement privacy leaking attacks automatically

Scalable and Declarative Information Extraction in a Parallel Data Analytics System

Informationsextraktions (IE) auf sehr großen Datenmengen erfordert hochkomplexe, skalierbare und anpassungsfähige Systeme. Obwohl zahlreiche IE-Algorithmen existieren, ist die nahtlose und erweiterbare Kombination dieser Werkzeuge in einem skalierbaren System immer noch eine große Herausforderung. In dieser Arbeit wird ein anfragebasiertes IE-System für eine parallelen Datenanalyseplattform vorgestellt, das für konkrete Anwendungsdomänen konfigurierbar ist und für Textsammlungen im Terabyte-Bereich skaliert. Zunächst werden konfigurierbare Operatoren für grundlegende IE- und Web-Analytics-Aufgaben definiert, mit denen komplexe IE-Aufgaben in Form von deklarativen Anfragen ausgedrückt werden können. Alle Operatoren werden hinsichtlich ihrer Eigenschaften charakterisiert um das Potenzial und die Bedeutung der Optimierung nicht-relationaler, benutzerdefinierter Operatoren (UDFs) für Data Flows hervorzuheben. Anschließend wird der Stand der Technik in der Optimierung nicht-relationaler Data Flows untersucht und herausgearbeitet, dass eine umfassende Optimierung von UDFs immer noch eine Herausforderung ist. Darauf aufbauend wird ein erweiterbarer, logischer Optimierer (SOFA) vorgestellt, der die Semantik von UDFs mit in die Optimierung mit einbezieht. SOFA analysiert eine kompakte Menge von Operator-Eigenschaften und kombiniert eine automatisierte Analyse mit manuellen UDF-Annotationen, um die umfassende Optimierung von Data Flows zu ermöglichen. SOFA ist in der Lage, beliebige Data Flows aus unterschiedlichen Anwendungsbereichen logisch zu optimieren, was zu erheblichen Laufzeitverbesserungen im Vergleich mit anderen Techniken führt. Als Viertes wird die Anwendbarkeit des vorgestellten Systems auf Korpora im Terabyte-Bereich untersucht und systematisch die Skalierbarkeit und Robustheit der eingesetzten Methoden und Werkzeuge beurteilt um schließlich die kritischsten Herausforderungen beim Aufbau eines IE-Systems für sehr große Datenmenge zu charakterisieren.Information extraction (IE) on very large data sets requires highly complex, scalable, and adaptive systems. Although numerous IE algorithms exist, their seamless and extensible combination in a scalable system still is a major challenge. This work presents a query-based IE system for a parallel data analysis platform, which is configurable for specific application domains and scales for terabyte-sized text collections. First, configurable operators are defined for basic IE and Web Analytics tasks, which can be used to express complex IE tasks in the form of declarative queries. All operators are characterized in terms of their properties to highlight the potential and importance of optimizing non-relational, user-defined operators (UDFs) for dataflows. Subsequently, we survey the state of the art in optimizing non-relational dataflows and highlight that a comprehensive optimization of UDFs is still a challenge. Based on this observation, an extensible, logical optimizer (SOFA) is introduced, which incorporates the semantics of UDFs into the optimization process. SOFA analyzes a compact set of operator properties and combines automated analysis with manual UDF annotations to enable a comprehensive optimization of data flows. SOFA is able to logically optimize arbitrary data flows from different application areas, resulting in significant runtime improvements compared to other techniques. Finally, the applicability of the presented system to terabyte-sized corpora is investigated. Hereby, we systematically evaluate scalability and robustness of the employed methods and tools in order to pinpoint the most critical challenges in building an IE system for very large data sets

A comparison of mobile search interfaces for isiXhosa speakers

Search interfaces have for a long time been targeted at the resource-rich languages, such as English. There has been little effort to support African (Bantu) languages in search interfaces when compared to languages such as English, particularly the isiXhosa language. However, due to the increase in use of mobile phones in developing countries, these interfaces can now be adapted to languages in these settings to support information access on the Web. This study proposes mobile search interfaces to support isiXhosa speakers to search for information on the Web using isiXhosa as a discovery language. The isiXhosa language is considered a low-resourced African (Bantu) language spoken in resource-constrained environments in South Africa. The language is spoken by over eight million people. Yet, there has been no search interface specifically targeted at supporting isiXhosa speakers. Two mobile search interfaces were developed on an Android application. The interfaces were text based and voice based. The design of the interfaces was based on feedback from 4 native isiXhosa speakers in a design focus group, and guidelines from the literature. Using the developed interfaces, an experiment was conducted with 34 native isiXhosa speaking students at the University of Cape Town, South Africa. This was done to investigate, which interface could better support isiXhosa speakers to search for information on the Web using mobile phones. Quantitative data was collected using application log files. Additionally, user feedback was then obtained using the standard Software Usability Measurement Inventory (SUMI) instrument, and both interfaces were confirmed as usable. In contrast to what was expected, users preferred the text interface in general, and according to most SUMI subscales. This could be because of greater familiarity with text search interfaces or because of the relative scarcity of voice interfaces in African (Bantu) languages. Where users are not literate, the voice interface may be the only option, so the fact that it was deemed usable is an important independent finding. Search in African (Bantu) language collections is still a largely unexplored field, and more work needs to be done on the interfaces as the algorithms and collections are developed in parallel