15 research outputs found
Navigating Complex Search Tasks with AI Copilots
As many of us in the information retrieval (IR) research community know and
appreciate, search is far from being a solved problem. Millions of people
struggle with tasks on search engines every day. Often, their struggles relate
to the intrinsic complexity of their task and the failure of search systems to
fully understand the task and serve relevant results. The task motivates the
search, creating the gap/problematic situation that searchers attempt to
bridge/resolve and drives search behavior as they work through different task
facets. Complex search tasks require more than support for rudimentary fact
finding or re-finding. Research on methods to support complex tasks includes
work on generating query and website suggestions, personalizing and
contextualizing search, and developing new search experiences, including those
that span time and space. The recent emergence of generative artificial
intelligence (AI) and the arrival of assistive agents, or copilots, based on
this technology, has the potential to offer further assistance to searchers,
especially those engaged in complex tasks. There are profound implications from
these advances for the design of intelligent systems and for the future of
search itself. This article, based on a keynote by the author at the 2023 ACM
SIGIR Conference, explores these issues and charts a course toward new horizons
in information access guided by AI copilots.Comment: 10 pages, 6 figure
Embedding Intelligence. Designerly reflections on AI-infused products
Artificial intelligence is more-or-less covertly entering our lives and houses, embedded into products and services that are acquiring novel roles and agency on users.
Products such as virtual assistants represent the first wave of materializa- tion of artificial intelligence in the domestic realm and beyond. They are new interlocutors in an emerging redefined relationship between humans and computers. They are agents, with miscommunicated or unclear proper- ties, performing actions to reach human-set goals.
They embed capabilities that industrial products never had. They can learn users’ preferences and accordingly adapt their responses, but they are also powerful means to shape people’s behavior and build new practices and habits. Nevertheless, the way these products are used is not fully exploiting their potential, and frequently they entail poor user experiences, relegating their role to gadgets or toys.
Furthermore, AI-infused products need vast amounts of personal data to work accurately, and the gathering and processing of this data are often obscure to end-users. As well, how, whether, and when it is preferable to implement AI in products and services is still an open debate. This condition raises critical ethical issues about their usage and may dramatically impact users’ trust and, ultimately, the quality of user experience.
The design discipline and the Human-Computer Interaction (HCI) field are just beginning to explore the wicked relationship between Design and AI, looking for a definition of its borders, still blurred and ever-changing. The book approaches this issue from a human-centered standpoint, proposing designerly reflections on AI-infused products. It addresses one main guiding question: what are the design implications of embedding intelligence into everyday objects
End-User Development of Voice User Interfaces based on Web content
Voice Assistants, and particularly the latest gadgets called smart speakers, allow end users to interact with applications by means of voice commands. As usual, end users are able to install applications (also called skills) that are available in repositories and fulfill multiple purposes. In this work we present an end-user environment to define skills for voice assistants based on the extraction of Web content and their organization into different voice navigation patterns. We describe the approach, the end-user development environment, and finally we present some case studies based on Alexa and Amazon Echo
End-User Development of Voice User Interfaces based on Web content
Voice Assistants, and particularly the latest gadgets called smart speakers, allow end users to interact with applications by means of voice commands. As usual, end users are able to install applications (also called skills) that are available in repositories and fulfill multiple purposes. In this work we present an end-user environment to define skills for voice assistants based on the extraction of Web content and their organization into different voice navigation patterns. We describe the approach, the end-user development environment, and finally we present some case studies based on Alexa and Amazon Echo.Publicado en Lecture Notes in Computer Science book series (LNCS, volume 11553).Laboratorio de Investigación y Formación en Informática Avanzad
SkillVet: Automated Traceability Analysis of Amazon Alexa Skills
Third-party software, or skills, are essential components in Smart Personal
Assistants (SPA). The number of skills has grown rapidly, dominated by a
changing environment that has no clear business model. Skills can access
personal information and this may pose a risk to users. However, there is
little information about how this ecosystem works, let alone the tools that can
facilitate its study. In this paper, we present the largest systematic
measurement of the Amazon Alexa skill ecosystem to date. We study developers'
practices in this ecosystem, including how they collect and justify the need
for sensitive information, by designing a methodology to identify
over-privileged skills with broken privacy policies. We collect 199,295 Alexa
skills and uncover that around 43% of the skills (and 50% of the developers)
that request these permissions follow bad privacy practices, including
(partially) broken data permissions traceability. In order to perform this kind
of analysis at scale, we present SkillVet that leverages machine learning and
natural language processing techniques, and generates high-accuracy prediction
sets. We report a number of concerning practices including how developers can
bypass Alexa's permission system through account linking and conversational
skills, and offer recommendations on how to improve transparency, privacy and
security. Resulting from the responsible disclosure we have conducted,13% of
the reported issues no longer pose a threat at submission time.Comment: 17pages, 8 figure
Smart Home Personal Assistants: A Security and Privacy Review
Smart Home Personal Assistants (SPA) are an emerging innovation that is
changing the way in which home users interact with the technology. However,
there are a number of elements that expose these systems to various risks: i)
the open nature of the voice channel they use, ii) the complexity of their
architecture, iii) the AI features they rely on, and iv) their use of a
wide-range of underlying technologies. This paper presents an in-depth review
of the security and privacy issues in SPA, categorizing the most important
attack vectors and their countermeasures. Based on this, we discuss open
research challenges that can help steer the community to tackle and address
current security and privacy issues in SPA. One of our key findings is that
even though the attack surface of SPA is conspicuously broad and there has been
a significant amount of recent research efforts in this area, research has so
far focused on a small part of the attack surface, particularly on issues
related to the interaction between the user and the SPA devices. We also point
out that further research is needed to tackle issues related to authorization,
speech recognition or profiling, to name a few. To the best of our knowledge,
this is the first article to conduct such a comprehensive review and
characterization of the security and privacy issues and countermeasures of SPA.Comment: Accepted for publication in ACM Computing Survey
User Perceptions and Privacy Information in the Smart Speaker Onboarding Process
A smart speaker's onboarding process is extremely important because it is the user's first touchpoint with the device. An effective onboarding process will communicate the necessary information a user needs in order to understand the smart speaker's functionalities as well as how to interact with it. Despite this, the onboarding process has not been a properly considered channel for conveying privacy information to users. This is surprising given that recommendations for communicating privacy information and related privacy controls often include making this information salient to the user and providing instructional use for controls. In this thesis, I explore the onboarding process for smart speakers as a potentially effective medium for which to convey privacy information. I conducted an empirical assessment of smart speakers current privacy practices and their onboarding flows in order to determine where privacy information and the communication of this information may be improved. I used my findings from this analysis to develop a smart speaker prototype to test the effectiveness of the speaker's onboarding process in helping users understand the speaker's functionalities. I also designed privacy-oriented voice commands to test within the onboarding process to evaluate if this type of privacy control influences user comprehension of privacy information. The results of this thesis show that the smart speaker's onboarding process can be improved to help users understand the device's privacy practices. Furthermore, they demonstrate that privacy-oriented voice commands show potential for future research despite being ineffective in this study.Master of Science in InformationSchool of Informationhttp://deepblue.lib.umich.edu/bitstream/2027.42/168550/1/20210506_Herakovic,Gina_Final_MTOP_Thesis.pd