Finger Tracking Untuk Interaksi Pada Virtual Keyboard

Secara konservatif, interaksi antara manusia dengan komputer saat ini masih menggunakan mouse, keyboard, dan layar monitor. Melalui Proyek Akhir ini, akan diberikan suatu alternatif substitusi keyboard konvensional dengan kamera video (webcam). Kamera digunakan sebagai sensor untuk menelusuri pergerakan atau perilaku jari tangan. Selanjutnya, perilaku jari tangan ini diterjemahkan dalam aksi keyboard. Sistem ini kemudian dinamakan dengan virtual keyboard. Pendeteksian jari tangan menggunakan haar cascade pada library OpenCV, selanjutnya tracking pergerakan jari tangan diterapkan dengan metode Kalman Filter, yaitu digunakan untuk memprediksi posisi jari tangan pada frame selanjutnya

Pengolahan Citra Digital untuk Keyboard Virtual Sebagai Antarmuka pada Aplikasi Berbasis Web

Since the first computer was founded, keyboard is always been a primary tool for interaction between humans and computers. Today, many computers use image processing technology to make interaction between computers and humans.The author try to apply image processing technology that implemented to virtual keyboard on web application. Using a webcam to capture the tip of index finger and the results will be sent to the localhost server for processing with image processing. Using Haar Cascade Classifier method to detect the tip of index finger, it will produce coordinates that sent to the web application and it used as a reference for determining button positions on virtual keyboard. Virtual keyboard characters will display after appointed by the tip of  index finger.From testing results, optimal distance from index finger to webcam is 20 – 35 cm. System can recognize the tip of index finger on white background and room with few furnitures. Average response time for displaying virtual keyboard sentences is 3 minutes and 28.838 seconds. So the virtual keyboard on this system was not able to be used as interface on web application, because it difficult to use in directing the tip of index finger to the character keys

Synesthesia: Detecting Screen Content via Remote Acoustic Side Channels

We show that subtle acoustic noises emanating from within computer screens can be used to detect the content displayed on the screens. This sound can be picked up by ordinary microphones built into webcams or screens, and is inadvertently transmitted to other parties, e.g., during a videoconference call or archived recordings. It can also be recorded by a smartphone or "smart speaker" placed on a desk next to the screen, or from as far as 10 meters away using a parabolic microphone. Empirically demonstrating various attack scenarios, we show how this channel can be used for real-time detection of on-screen text, or users' input into on-screen virtual keyboards. We also demonstrate how an attacker can analyze the audio received during video call (e.g., on Google Hangout) to infer whether the other side is browsing the web in lieu of watching the video call, and which web site is displayed on their screen

Technology - Empowering the Educational Researcher through Remote Observation

Observing students using computers often occurs through three methods: user-lab, on-site and remote data logging. Whilst each of these have their advantages with the new type of students such as elearners, an alternative method called web-conferencing remote observation is presented for observing students at a distance. This method collects both audio and video data of the observer through webcams and voice/video conversations. Students are able to interact with the software through application sharing facilities. Further, it allows both quantitative and qualitative data to be collected. This proof-of-concept method is presented here where it has been used in two previous studies using Windows Messenger and Netviewer. Although, video quality is not high the quality is sufficient for observational data

A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays

We identify usability challenges facing consumers adopting Virtual Reality (VR) head-mounted displays (HMDs) in a survey of 108 VR HMD users. Users reported significant issues in interacting with, and being aware of their real-world context when using a HMD. Building upon existing work on blending real and virtual environments, we performed three design studies to address these usability concerns. In a typing study, we show that augmenting VR with a view of reality significantly corrected the performance impairment of typing in VR. We then investigated how much reality should be incorporated and when, so as to preserve users’ sense of presence in VR. For interaction with objects and peripherals, we found that selectively presenting reality as users engaged with it was optimal in terms of performance and users’ sense of presence. Finally, we investigated how this selective, engagement-dependent approach could be applied in social environments, to support the user’s awareness of the proximity and presence of others

Human-display interaction technology: Emerging remote interfaces for pervasive display environments

This is the author's accepted manuscript. The final published article is available from the link below. Copyright @ 2010 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.We're living in a world where information processing isn't confined to desktop computers - it's being integrated into everyday objects and activities. Pervasive computation is human centered: it permeates our physical world, helping us achieve goals and fulfill our needs with minimum effort by exploiting natural interaction styles. Remote interaction with screen displays requires a sensor-based, multimodal, touchless approach. For example, by processing user hand gestures, this paradigm removes constraints requiring physical contact and permits natural interaction with tangible digital information. Such touchless interaction can be multimodal, exploiting the visual, auditory, and olfactory senses.Ministerio de Educación y Ciencia and Amper Sistemas, SA