Mobile to cloud co-processing of ASL finger spelling to text conversion

Computer recognition of American Sign Language (ASL) is a computationally intensive task. Although it has generally been performed using powerful lab workstations, this research investigates transcription of static ASL signs using an application on a consumer-level mobile device. The application provides real-time sign to text translation by processing a live video stream to detect the ASL alphabet as well as custom signs to perform tasks on the device. In this work several avenues for classification and processing were ex-plored to evaluate performance for mobile ASL transcription. The cho-sen classification algorithm uses locality preserving projections (LPP) with trained support vector machines (SVMs). Processing was investigated using either the mobile device only or with cloud assistance. In comparison to the native mobile application, the cloud-assisted application increased classification speed, reduced memory usage, and kept the network usage low while barely increasing the power required. A distributed solution has been created that will provide a new way of interacting with the mobile device in a native way to a hard-of-hearing person while also considering the network, power and processing constraints of the mobile device

Reading Bodies: Disability and American Literary History, 1789-1889

This dissertation brings the field of critical disability studies to bear on organizational paradigms of nineteenth-century American literature. “Reading Bodies” intervenes in these fields with the claim that the book in a variety of formats, publications, and circulations acts as a disciplinary tool that seeks to arrange physical and mental characteristics and capacities into the category of disability. This project moves beyond examining representations of disability to demonstrate that the same social, cultural, and political forces that generated literary movements and outpourings – such as nationalism, displacement of Native peoples, slavery, and state-sanctioned violence – also generated material conditions of impairment that formal literary conventions sought to consolidate as “disability.” Individuals and communities reading, writing, and responding to the genres of seduction, historical fiction, slave narrative, Civil War poetry, and children’s literature both deployed and challenged formal literary conventions to model or defy normative and deviant behaviors. The formal characteristics and aesthetic concerns of the field of American literature, I find, are products of larger social processes that both cause impairment and that communicate and mark constructions of disability into and onto reading and non-reading publics. as social and literary forces coalesced the category “disability,” often those populations most vulnerable to impairment responded by challenging, resisting, or completely renovating the conventions and categories of textual and bodily behavior. In a variety of interactions with the book, nineteenth-century women, Native Americans, African Americans, wounded soldiers, and children offer alternative intersectional perspectives and possibilities for what it means to produce literature and for what it means to inhabit a body. Those works considered literary outliers both in their day and in contemporary critical assessments, such as Leonora Sansay’s Secret History (1808), the Life of Black Hawk (1833), and midcentury children’s books printed for sight-impaired readers, reveal the normative underpinnings of literary and bodily taxonomies

Proceedings of the 3rd IUI Workshop on Interacting with Smart Objects

These are the Proceedings of the 3rd IUI Workshop on Interacting with Smart Objects. Objects that we use in our everyday life are expanding their restricted interaction capabilities and provide functionalities that go far beyond their original functionality. They feature computing capabilities and are thus able to capture information, process and store it and interact with their environments, turning them into smart objects

A Grammar of Italian Sign Language (LIS)

A Grammar of Italian Sign Language (LIS) is a comprehensive presentation of the grammatical properties of LIS. It has been conceived as a tool for students, teachers, interpreters, the Deaf community, researchers, linguists and whoever is interested in the study of LIS. It is one output of the Horizon 2020 SIGN-HUB project. It is composed of six Parts: Part 1 devoted to the social and historical background in which the language has developed, and five Parts covering the main properties of Phonology, Lexicon, Morphology, Syntax and Pragmatics. Thanks to the electronic format of the grammar, text and videos are highly interconnected and are designed to fit the description of a visual language

Bridging the Gap Between People, Mobile Devices, and the Physical World

Human-computer interaction (HCI) is being revolutionized by computational design and artificial intelligence. As the diversity of user interfaces shifts from personal desktops to mobile and wearable devices, yesterday’s tools and interfaces are insufficient to meet the demands of tomorrow’s devices. This dissertation describes my research on leveraging different physical channels (e.g., vibration, light, capacitance) to enable novel interaction opportunities. We first introduce FontCode, an information embedding technique for text documents. Given a text document with specific fonts, our method can embed user-specified information (e.g., URLs, meta data, etc) in the text by perturbing the glyphs of text characters while preserving the text content. The embedded information can later be retrieved using a smartphone in real time. Then, we present Vidgets, a family of mechanical widgets, specifically push buttons and rotary knobs that augment mobile devices with tangible user interfaces. When these widgets are attached to a mobile device and a user interacts with them, the nonlinear mechanical response of the widgets shifts the device slightly and quickly. Subsequently, this subtle motion can be detected by the Inertial Measurement Units (IMUs), which is commonly installed on mobile devices. Next, we propose BackTrack, a trackpad placed on the back of a smartphone to track finegrained finger motions. Our system has a small form factor, with all the circuits encapsulated in a thin layer attached to a phone case. It can be used with any off-the-shelf smartphone, requiring no power supply or modification of the operating systems. BackTrack simply extends the finger tracking area of the front screen, without interrupting the use of the front screen. Lastly, we demonstrate MoiréBoard, a new camera tracking method that leverages a seemingly irrelevant visual phenomenon, the moiré effect. Based on a systematic analysis of the moiré effect under camera projection, MoiréBoard requires no power nor camera calibration. It can easily be made at a low cost (e.g., through 3D printing) and ready to use with any stock mobile device with a camera. Its tracking algorithm is computationally efficient and can run at a high frame rate. It is not only simple to implement, but also tracks devices at a high accuracy, comparable to the state-of-the-art commercial VR tracking systems

An Introspective Enquiry Mutually Emplacing Teacher and Non-literate Former Refugee Students in Pedagogical Landscapes

Searching to discover what it is that is in the self that is brought to the teaching role this study transports the teacher along a lived experience trajectory. Assuming an interface stance as teacher-learner, researcher-interpreter, narrator-writer, to find insightful meaning the inquiry glances back as it moves forward, emplacing the teacher with adult former refugee students in an empowering landscape that engenders self-learning and respect for diversity within a culture of education

20.2 Scientific Wonders

Rampike Vol. 20 / No. 2 (Scientific Wonders issue): Carol Stetser, Memoriam of Robert Kroetsch, Guy Laramée, Norman Cornett, John Oughton, Ryosuke Cohen, N.E.Thing Co., Iain Baxter, Adam Lauder, Richard Kostelanetz, Rosemary Nixon, Gordon W.F. Drake, Daniel King, Gail Scott, Karl Jirgens, Antanas Sileika, SS Prasad, Mike Marcon, Murali Sivaramakrishnan, Eric Zboya, Alan Lord, Ruggero Maggi, Daniel David Moses, Lorenzo Menoud, Reed Altemus, Stephen Humphrey, J.R. Carpenter, John Robert Colombo, Christen Thomas, Kevin McPherson, Francine P. Lewis, Joe Davies, Monica Radulescu, Norman Lock, Gustav Morin, Kim Goldberg, Edward Nixon, Stan Rogal, Scott Bentley, Aaron Tucker, Richard Truhlar, Fausto Bedoya, Christian Burgaud. Cover art: Guy Laramée

TouchEditor: Interaction design and evaluation of a flexible touchpad for text editing of head-mounted displays in speech-unfriendly environments

A text editing solution that adapts to speech-unfriendly (inconvenient to speak or difficult to recognize speech) environments is essential for head-mounted displays (HMDs) to work universally. For existing schemes, e.g., touch bar, virtual keyboard and physical keyboard, there are shortcomings such as insufficient speed, uncomfortable experience or restrictions on user location and posture. To mitigate these restrictions, we propose TouchEditor, a novel text editing system for HMDs based on a flexible piezoresistive film sensor, supporting cursor positioning, text selection, text retyping and editing commands (i.e., Copy, Paste, Delete, etc.). Through literature overview and heuristic study, we design a pressure-controlled menu and a shortcut gesture set for entering editing commands, and propose an area-and-pressure-based method for cursor positioning and text selection that skillfully maps gestures in different areas and with different strengths to cursor movements with different directions and granularities. The evaluation results show that TouchEditor i) adapts to various contents and scenes well with a stable correction speed of 0.075 corrections per second; ii) achieves 95.4% gesture recognition accuracy; iii) reaches a considerable level with a mobile phone in text selection tasks. The comparison results with the speech-dependent EYEditor and the built-in touch bar further prove the flexibility and robustness of TouchEditor in speech-unfriendly environments

Qur'anic Matters: Media and Materiality

Mushaf (read with "s" and "h" separately) is what Muslims call the physical body of the Quran, its pages, binding, and print. In contrast to earlier works on the holy book of Muslims, in my dissertation I shift from studying the Quranic text exclusively as message to its corporeal existence as an object in the hands of its manufacturers and users in Egypt, treating its production and consumption as meaning-making processes. I approach materiality of the Quran -- the corporeal and electronic medium of the text -- as a place for tracking transformations of religious authority, agency, knowledge and Muslim practice in the context of changing technologies of text production. By looking at different media forms (manuscript, print, and digital devices) that disseminate the text of the Quran, I follow tensions existing between a manufactured object, tradition, and practical religious experience among Muslims in Egypt. I treat material carrier of the Quran as an object that mediates between institutionalized authority epitomized by the University of al-Azhar (the oldest religious institution in the Middle East) that controls Quranic production, markets of publishing houses that manufacture printed and electronic "Qurans," and socially and economically diversified public consumers of the holy word in Egypt and beyond. Egypt is not the only potentate on the Muslim religious publishing market; highlighting the materiality of the Quranic text allows me to track the connections and tensions that arise between Egypt and Saudi Arabia, a new competitor in the production of Qurans. Through this project, I show how the Quran as an object participates in configuring social ties both locally and transnationally.Doctor of Philosoph