Large Language Models as General Pattern Machines

We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art. Surprisingly, pattern completion proficiency can be partially retained even when the sequences are expressed using tokens randomly sampled from the vocabulary. These results suggest that without any additional training, LLMs can serve as general sequence modelers, driven by in-context learning. In this work, we investigate how these zero-shot capabilities may be applied to problems in robotics -- from extrapolating sequences of numbers that represent states over time to complete simple motions, to least-to-most prompting of reward-conditioned trajectories that can discover and represent closed-loop policies (e.g., a stabilizing controller for CartPole). While difficult to deploy today for real systems due to latency, context size limitations, and compute costs, the approach of using LLMs to drive low-level control may provide an exciting glimpse into how the patterns among words could be transferred to actions.Comment: 21 pages, 25 figures. To appear at Conference on Robot Learning (CoRL) 202

Chickens play to the crowd

The time was ripe for Marino’s review of chickens’ cognitive capacities. The research community, apart from expressing gratitude for Marino’s work, should now use it to increase public awareness of chickens’ abilities. People’s views on many animals are ill-informed. Scientists need to communicate and engage with the public about the relevance and societal implications of their findings

Independence in the Home: A Wearable Interface for a Person with Quadriplegia to Teleoperate a Mobile Manipulator

Teleoperation of mobile manipulators within a home environment can significantly enhance the independence of individuals with severe motor impairments, allowing them to regain the ability to perform self-care and household tasks. There is a critical need for novel teleoperation interfaces to offer effective alternatives for individuals with impairments who may encounter challenges in using existing interfaces due to physical limitations. In this work, we iterate on one such interface, HAT (Head-Worn Assistive Teleoperation), an inertial-based wearable integrated into any head-worn garment. We evaluate HAT through a 7-day in-home study with Henry Evans, a non-speaking individual with quadriplegia who has participated extensively in assistive robotics studies. We additionally evaluate HAT with a proposed shared control method for mobile manipulators termed Driver Assistance and demonstrate how the interface generalizes to other physical devices and contexts. Our results show that HAT is a strong teleoperation interface across key metrics including efficiency, errors, learning curve, and workload. Code and videos are located on our project website

Training a New Trick Using No-Reward Markers: Effects on Dogs’ Performance and Stress Behaviors

This study explored using no-reward markers (NRMs). Dogs were taught a novel trick. In the IG group dogs’ errors were ignored; in the NRM group they elicited a tone. Performance and stress were evaluated. IG dogs reached higher levels of performance, with no difference in the frequency of stress behaviors

Task Dynamics of Prior Training Influence Visual Force Estimation Ability During Teleoperation

The lack of haptic feedback in Robot-assisted Minimally Invasive Surgery (RMIS) is a potential barrier to safe tissue handling during surgery. Bayesian modeling theory suggests that surgeons with experience in open or laparoscopic surgery can develop priors of tissue stiffness that translate to better force estimation abilities during RMIS compared to surgeons with no experience. To test if prior haptic experience leads to improved force estimation ability in teleoperation, 33 participants were assigned to one of three training conditions: manual manipulation, teleoperation with force feedback, or teleoperation without force feedback, and learned to tension a silicone sample to a set of force values. They were then asked to perform the tension task, and a previously unencountered palpation task, to a different set of force values under teleoperation without force feedback. Compared to the teleoperation groups, the manual group had higher force error in the tension task outside the range of forces they had trained on, but showed better speed-accuracy functions in the palpation task at low force levels. This suggests that the dynamics of the training modality affect force estimation ability during teleoperation, with the prior haptic experience accessible if formed under the same dynamics as the task.Comment: 12 pages, 8 figure

Using ICT Programs to Support Students with Dyslexia in Aquiring Literacy

Mastering reading and writing skills as the key priority for students who experience developmental disorders reading and writing or dyslexia. Learning language for dyslexic students is not easy, dyslexic students have difficulty processing language components, especially in reading and writing. Reading and writing begin at an early age, and continue into elementary school. Students learn to read and write by memorizing and repeating letters and words. However, the fact is that this method cannot always be taught for dyslexic students. Students with dyslexia are very different from students in general, they learn differently at very different levels. Some students need more support from the people around them. Dyslexic students are easily saturated if they are invited to learn to read and write. To overcome this problem it is necessary to use the Information and Communication Technology (ICT) program in learning, this program is based on digital, so that it can help dyslexic students in the field of literacy. The use of ICT programs is very supportive of literacy skills and can provide benefits to them, they can learn independently in education, work, and home environment. These programs allow dyslexic students to have the opportunity to access almost all texts. Using the program  provide opportunities for dyleksia students to continue learning, and practice so that it is hoped that dyslexic students are able to succeed in writing and reading activities

Constraining the Size Growth of the Task Space with Socially Guided Intrinsic Motivation using Demonstrations

This paper presents an algorithm for learning a highly redundant inverse model in continuous and non-preset environments. Our Socially Guided Intrinsic Motivation by Demonstrations (SGIM-D) algorithm combines the advantages of both social learning and intrinsic motivation, to specialise in a wide range of skills, while lessening its dependence on the teacher. SGIM-D is evaluated on a fishing skill learning experiment.Comment: JCAI Workshop on Agents Learning Interactively from Human Teachers (ALIHT), Barcelona : Spain (2011