Would Archimedes Shout “Eureka” If He Had Google? Innovating with Search Algorithms

In this paper we investigate the relationship between algorithmic search tools and the innovation process. Today, search algorithms are used for all tasks, yet we know little about their impact on the well-studied innovation process. We suggest a theoretical framework based on centripetal and centrifugal forces that conceptualizes the relationship between the algorithmic design logics of search tools and the innovation process. We use it to illustrate the current challenges with the use of informational search tools based on design principles of popularity and personalization, for innovation. We propose the need to develop and use exploratory search models and tools for innovation

To engage or not to engage with AI for critical judgments : how professionals deal with opacity when using AI for medical diagnosis

rtificial intelligence (AI) technologies promise to transform how professionals conduct knowledge work by augmenting their capabilities for making professional judgments. We know little, however, about how human-AI augmentation takes place in practice. Yet, gaining this understanding is particularly important when professionals use AI tools to form judgments on critical decisions. We conducted an in-depth field study in a major U.S. hospital where AI tools were used in three departments by diagnostic radiologists making breast cancer, lung cancer, and bone age determinations. The study illustrates the hindering effects of opacity that professionals experienced when using AI tools and explores how these professionals grappled with it in practice. In all three departments, this opacity resulted in professionals experiencing increased uncertainty because AI tool results often diverged from their initial judgment without providing underlying reasoning. Only in one department (of the three) did professionals consistently incorporate AI results into their final judgments, achieving what we call engaged augmentation. These professionals invested in AI interrogation practices—practices enacted by human experts to relate their own knowledge claims to AI knowledge claims. Professionals in the other two departments did not enact such practices and did not incorporate AI inputs into their final decisions, which we call unengaged “augmentation.” Our study unpacks the challenges involved in augmenting professional judgment with powerful, yet opaque, technologies and contributes to literature on AI adoption in knowledge work

Intelligence Augmentation: Human Factors in AI and Future of Work

The availability of parallel and distributed processing at a reasonable cost and the diversity of data sources have contributed to advanced developments in artificial intelligence (AI). These developments in the AI computing environment are not concomitant with changes in the social, legal, and political environment. While considering deploying AI, the deployment context and the end goal of human intelligence augmentation for that specific context have surfaced as significant factors for professionals, organizations, and society. In this research commentary, we highlight some important socio-technical aspects associated with recent growth in AI systems. We elaborate on the intricacies of human-machine interaction that form the foundation of augmented intelligence. We also highlight the ethical considerations that relate to these interactions and explain how augmented intelligence can play a key role in shaping the future of human work

How maker tools can accelerate ideation

Innovators face tremendous time pressures today, whether they are tackling urgent issues such as public health and climate change or designing new products to stay ahead in a fast-moving competitive market. To meet the challenge, companies are investing in a number of technologies that accelerate innovation, but for many, the process is still frustratingly slow. What should organizations do? Our short answer: Do not use accelerating technologies only for rapid prototyping; use them much earlier and differently, for rapid ideating

The no.1 question to ask when evaluating AI tools

In the fast-moving and highly competitive artificial intelligence sector, developers’ claims that their AI tools can make critical predictions with a high degree of accuracy are key to selling prospective customers on their value. Because it can be daunting for people who are not AI experts to evaluate these tools, leaders may be tempted to rely on the high-level performance metrics published in sales materials. But doing so often leads to disappointing or even risky implementations

Minimal and adaptive coordination : how hackathons’ projects accelerate innovation without killing it

The innovation journey of new product development processes often spans weeks or months. Recently, though, hackathons have turned the journey into an ad hoc sprint of only a couple of days using new tools and technologies. Existing research predicts that such conditions will result in a failure to produce new working products, yet hackathons often lead to functioning innovative products. To investigate this puzzle, we closely studied the product development process of 13 comparable projects in assistive technology hackathons. We found that accelerating innovation created temporal ambiguity, as it was unclear how to coordinate the challenging work within such an extremely limited and ad hoc time frame. Multiple projects worked to reduce this ambiguity, importing temporal structures from organizational innovation processes and compressing them to fit the extremely limited and ad hoc time frame. They worked in full coordination to build a new product. They all failed. Only projects that sustained the temporal ambiguity—by working with a minimal basis for coordination and allowing new temporal structures to emerge—were able to produce functioning new products under the intense time pressure. This study contributes to theories on innovation processes, coordination, and temporality

Is AI Ground Truth Really True? The Dangers of Training and Evaluating AI Tools Based on Experts’ Know-What

Organizational decision-makers need to evaluate AI tools in light of increasing claims that such tools out-perform human experts. Yet, measuring the quality of knowledge work is challenging, raising the question of how to evaluate AI performance in such contexts. We investigate this question through a field study of a major U.S. hospital, observing how managers evaluated five different machine-learning (ML) based AI tools. Each tool reported high performance according to standard AI accuracy measures, which were based on ground truth labels provided by qualified experts. Trying these tools out in practice, however, revealed that none of them met expectations. Searching for explanations, managers began confronting the high uncertainty of experts’ know-what knowledge captured in ground truth labels used to train and validate ML models. In practice, experts address this uncertainty by drawing on rich know-how practices, which were not incorporated into these ML-based tools. Discovering the disconnect between AI’s know-what and experts’ know-how enabled managers to better understand the risks and benefits of each tool. This study shows dangers of treating ground truth labels used in ML models objectively when the underlying knowledge is uncertain. We outline implications of our study for developing, training, and evaluating AI for knowledge work

Neither a Bazaar nor a cathedral : the interplay between structure and agency in Wikipedia's role system

Roles provide a key coordination mechanism in peer-production. Whereas one stream in the literature has focused on the structural responsibilities associated with roles, another has stressed the emergent nature of work. To date, these streams have proceeded largely in parallel. In seeking to enhance our understanding of the tension between structure and agency in peer-production, we investigated the interplay between structural and emergent roles. Our study explored the breadth of structural roles in Wikipedia (English version) and their linkage to various forms of activities. Our analyses show that despite the latitude in selecting their mode of participation, participants' structural and emergent roles are tightly coupled. Our discussion highlights that: (a) participants often stay close to the “production ground floor” despite the assignment into structural roles; and (b) there are typical modifications in activity patterns associated with role-assignment, namely: functional specialization, multispecialization, defunctionalization, changes in communication patterns, management of identity, and role definition. We contribute to theory of coordination and roles, as well as provide some practical implications