ChatTL;DR – You Really Ought to Check What the LLM Said on Your Behalf

Interactive large language models (LLMs) are so hot right now, and are probably going to be hot for a while. There are lots of problems exciting challenges created by mass use of LLMs. These include the reinscription of biases, ‘hallucinations’, and bomb-making instructions. Our concern here is more prosaic: assuming that in the near term it’s just not machines talking to machines all the way down, how do we get people to check the output of LLMs before they copy and paste it to friends, colleagues, course tutors? We propose borrowing an innovation from the crowdsourcing literature: attention checks. These checks (e.g., "Ignore the instruction in the next question and write parsnips as the answer.") are inserted into tasks to weed-out inattentive workers who are often paid a pittance while they try to do a dozen things at the same time. We propose ChatTL;DR1, an interactive LLM that inserts attention checks into its outputs. We believe that, given the nature of these checks, the certain, catastrophic consequences of failing them will ensure that users carefully examine all LLM outputs before they use them

The Paradox of spreadsheet self-efficacy: Social incentives for informal knowledge sharing in end-user programming

Informal Knowledge Sharing (KS) is vital for end- user programmers to gain expertise. To better understand how personal (self-efficacy), social (reputational gains, trust between colleagues), and software-related (codification effort) variables influence spreadsheet KS intention, we conducted a multiple regressions analysis based on survey data from spreadsheet users (n=100) in administrative and finance roles. We found that high levels of spreadsheet self-efficacy and a perception that sharing would result in reputational gains predicted higher KS intention, but individuals who found knowledge codification effortful showed lower KS intention. We also observed that regardless of occupation, users tended to report a lower sense of self-efficacy in their general spreadsheet proficiency, despite also reporting high self-efficacy in spreadsheet use for job-related contexts. Our findings suggest that acknowledging and designing for these social and personal variables can help avoid situations where experienced individuals refrain unnecessarily from sharing, with implications for spreadsheet design

Crowdworkers' temporal flexibility is being traded for the convenience of requesters through 19 'invisible mechanisms' employed by crowdworking platforms

Crowdworking platforms are a prime example of a product that sells flexibility to its consumers. In this paper, we argue that crowdworking platforms sell temporal flexibility to requesters to the detriment of workers. We begin by identifying a list of 19 features employed by crowdworking platforms that facilitate the trade of temporal flexibility from crowdworkers to requesters. Using the list of features, we conduct a comparative analysis of nine crowdworking platforms available to U.S.-based workers, in which we describe key differences and similarities between the platforms. We find that crowdworking platforms strongly favour features that promote requesters’ temporal flexibility over workers’ by limiting the predictability of workers’ working hours and restricting paid time. Further, we identify which platforms employ the highest number of features that facilitate the trade of temporal flexibility from workers to requesters, consequently increasing workers’ temporal precarity. We conclude the paper by discussing the implications of the results

Visualisations with semantic icons: Assessing engagement with distracting elements

As visualisations reach a broad range of audiences, designing visualisations that attract and engage becomes more critical. Prior work suggests that semantic icons entice and immerse the reader; however, little is known about their impact with informational tasks and when the viewer’s attention is divided because of a distracting element. To address this gap, we first explored a variety of semantic icons with various visualisation attributes. The findings of this exploration shaped the design of our primary comparative online user studies, whereparticipants saw a target visualisation with a distracting visualisation on a web page and were asked to extract insights. Their engagement was measured through three dependent variables: (1) visual attention, (2) effort to write insights, and (3) self-reported engagement. In Study 1, we discovered that visualisations with semantic icons were consistently perceived to be more engaging than the plain version. However, we found no differences in visual attention and effort between the two versions. Thus, we ran Study 2 using visualisations with more salient semantic icons to achieve maximum contrast. The results were consistent with our first Study. Furthermore, we found that semantic icons elevated engagement with visualisations depicting less interestingand engaging topics from the participant’s perspective. We extended prior work by demonstrating the semantic value after performing an informational task (extracting insights) and reflecting on the visualisation, besides its value to the first impression. Our findings may be helpful to visualisation designers and storytellers keen on designing engaging visualisations with limited resources. We also contribute reflections on engagement measurements with visualisations and provide future directions

Short links and tiny keyboards::A systematic exploration of design trade-offs in link shortening services

AbstractLink-shortening services save space and make the manual entry of URLs less onerous. Short links are often included on printed materials so that people using mobile devices can quickly enter URLs. Although mobile transcription is a common use-case, link-shortening services generate output that is poorly suited to entry on mobile devices: links often contain numbers and capital letters that require time consuming mode switches on touch screen keyboards. With the aid of computational modeling, we identified problems with the output of a link-shortening service, bit.ly. Based on the results of this modeling, we hypothesized that longer links that are optimized for input on mobile keyboards would improve link entry speeds compared to shorter links that required keyboard mode switches. We conducted a human performance study that confirmed this hypothesis. Finally, we applied our method to a selection of different non-word mobile data-entry tasks. This work illustrates the need for service design to fit the constraints of the devices people use to consume services

The Emergence of Interactive Behaviour: A Model of Rational Menu Search

ABSTRACT One reason that human interaction with technology is difficult to understand is because the way in which people perform interactive tasks is highly adaptive. One such interactive task is menu search. In the current article we test the hypothesis that menu search is rationally adapted to (1) the ecological structure of interaction, (2) cognitive and perceptual limits, and (3) the goal to maximise the trade-off between speed and accuracy. Unlike in previous models, no assumptions are made about the strategies available to or adopted by users, rather the menu search problem is specified as a reinforcement learning problem and behaviour emerges by finding the optimal policy. The model is tested against existing empirical findings concerning the effect of menu organisation and menu length. The model predicts the effect of these variables on task completion time and eye movements. The discussion considers the pros and cons of the modelling approach relative to other well-known modelling approaches

The Emergence of Interactive Behaviour: A Model of Rational Menu Search

ABSTRACT One reason that human interaction with technology is difficult to understand is because the way in which people perform interactive tasks is highly adaptive. One such interactive task is menu search. In the current article we test the hypothesis that menu search is rationally adapted to (1) the ecological structure of interaction, (2) cognitive and perceptual limits, and (3) the goal to maximise the trade-off between speed and accuracy. Unlike in previous models, no assumptions are made about the strategies available to or adopted by users, rather the menu search problem is specified as a reinforcement learning problem and behaviour emerges by finding the optimal policy. The model is tested against existing empirical findings concerning the effect of menu organisation and menu length. The model predicts the effect of these variables on task completion time and eye movements. The discussion considers the pros and cons of the modelling approach relative to other well-known modelling approaches

"Sometimes it's like putting the track in front of the rushing train": Having to be 'on call' for work limits the temporal flexibility of crowdworkers

Research suggests that the temporal flexibility advertised to crowdworkers by crowdsourcing platforms is limited by both client-imposed constraints (e.g., strict completion times) and crowdworkers' tooling practices (e.g., multitasking). In this paper, we explore an additional contributor to workers' limited temporal flexibility: the design of crowdsourcing platforms, namely requiring crowdworkers to be `on call' for work. We conducted two studies to investigate the impact of having to be `on call' on workers' schedule control and job control. We find that being `on call' impacted: (1) participants' ability to schedule their time and stick to planned work hours, and (2) the pace at which participants worked and took breaks. The results of the two studies suggest that the `on-demand' nature of crowdsourcing platforms can limit workers' temporal flexibility by reducing schedule control and job control. We conclude the paper by discussing the implications of the results for: (a) crowdworkers, (b) crowdsourcing platforms, and (c) the wider platform economy

Exploring the effects of non-monetary reimbursement for participants in HCI research

When running experiments within the field of Human Computer Interaction (HCI) it is common practice to ask participants to come to a specified lab location, and reimburse them monetarily for their time and travel costs. This, however, is not the only means by which to encourage participation in scientific study. Citizen science projects, which encourage the public to become involved in scientific research, have had great success in getting people to act as sensors to collect data or to volunteer their idling computer or brain power to classify large data sets across a broad range of fields including biology, cosmology and physical and environmental science. This is often done without the expectation of payment. Additionally, data collection need not be done on behalf of an external researcher; the Quantified Self (QS) movement allows people to reflect on data they have collected about themselves. This too, then, is a form of non-reimbursed data collection. Here we investigate whether citizen HCI scientists and those interested in personal data produce reliable results compared to participants in more traditional lab-based studies. Through six studies, we explore how participation rates and data quality are affected by recruiting participants without monetary reimbursement: either by providing participants with data about themselves as reward (a QS approach), or by simply requesting help with no extrinsic reward (as in citizen science projects). We show that people are indeed willing to take part in online HCI research in the absence of extrinsic monetary reward, and that the data generated by participants who take part for selfless reasons, rather than for monetary reward, can be as high quality as data gathered in the lab and in addition may be of higher quality than data generated by participants given monetary reimbursement online. This suggests that large HCI experiments could be run online in the future, without having to incur the equally large reimbursement costs alongside the possibility of running experiments in environments outside of the lab