4 research outputs found
Crowdsourcing the Perception of Machine Teaching
Teachable interfaces can empower end-users to attune machine learning systems
to their idiosyncratic characteristics and environment by explicitly providing
pertinent training examples. While facilitating control, their effectiveness
can be hindered by the lack of expertise or misconceptions. We investigate how
users may conceptualize, experience, and reflect on their engagement in machine
teaching by deploying a mobile teachable testbed in Amazon Mechanical Turk.
Using a performance-based payment scheme, Mechanical Turkers (N = 100) are
called to train, test, and re-train a robust recognition model in real-time
with a few snapshots taken in their environment. We find that participants
incorporate diversity in their examples drawing from parallels to how humans
recognize objects independent of size, viewpoint, location, and illumination.
Many of their misconceptions relate to consistency and model capabilities for
reasoning. With limited variation and edge cases in testing, the majority of
them do not change strategies on a second training attempt.Comment: 10 pages, 8 figures, 5 tables, CHI2020 conferenc
Hand–object interaction recognition based on visual attention using multiscopic cyber-physical-social system
Computer vision-based cyber-physical-social systems (CPSS) are predicted to be the future of independent hand rehabilitation. However, there is a link between hand function and cognition in the elderly that this technology has not adequately supported. To investigate this issue, this paper proposes a multiscopic CPSS framework by developing hand–object interaction (HOI) based on visual attention. First, we use egocentric vision to extract features from hand posture at the microscopic level. With 94.87% testing accuracy, we use three layers of graph neural network (GNN) based on hand skeletal features to categorize 16 grasp postures. Second, we use a mesoscopic active perception ability to validate the HOI with eye tracking in the task-specific reach-to-grasp cycle. With 90.75% testing accuracy, the distance between the fingertips and the center of an object is used as input to a multi-layer gated recurrent unit based on recurrent neural network architecture. Third, we incorporate visual attention into the cognitive ability for classifying multiple objects at the macroscopic level. In two scenarios with four activities, we use GNN with three convolutional layers to categorize some objects. The outcome demonstrates that the system can successfully separate objects based on related activities. Further research and development are expected to support the CPSS application in independent rehabilitation