1 research outputs found
Multimodal Systems: Taxonomy, Methods, and Challenges
Naturally, humans use multiple modalities to convey information. The
modalities are processed both sequentially and in parallel for communication in
the human brain, this changes when humans interact with computers. Empowering
computers with the capability to process input multimodally is a major domain
of investigation in Human-Computer Interaction (HCI). The advancement in
technology (powerful mobile devices, advanced sensors, new ways of output,
etc.) has opened up new gateways for researchers to design systems that allow
multimodal interaction. It is a matter of time when the multimodal inputs will
overtake the traditional ways of interactions. The paper provides an
introduction to the domain of multimodal systems, explains a brief history,
describes advantages of multimodal systems over unimodal systems, and discusses
various modalities. The input modeling, fusion, and data collection were
discussed. Finally, the challenges in the multimodal systems research were
listed. The analysis of the literature showed that multimodal interface systems
improve the task completion rate and reduce the errors compared to unimodal
systems. The commonly used inputs for multimodal interaction are speech and
gestures. In the case of multimodal inputs, late integration of input
modalities is preferred by researchers because it allows easy update of
modalities and corresponding vocabularies