Taking a Call Is Facilitated by the Multisensory Processing of Smartphone Vibrations, Sounds, and Flashes

Many electronic devices that we use in our daily lives provide inputs that need to be processed and integrated by our senses. For instance, ringing, vibrating, and flashing indicate incoming calls and messages in smartphones. Whether the presentation of multiple smartphone stimuli simultaneously provides an advantage over the processing of the same stimuli presented in isolation has not yet been investigated. In this behavioral study we examined multisensory processing between visual (V), tactile (T), and auditory (A) stimuli produced by a smartphone. Unisensory V, T, and A stimuli as well as VA, AT, VT, and trisensory VAT stimuli were presented in random order. Participants responded to any stimulus appearance by touching the smartphone screen using the stimulated hand (Experiment 1), or the non-stimulated hand (Experiment 2). We examined violations of the race model to test whether shorter response times to multisensory stimuli exceed probability summations of unisensory stimuli. Significant violations of the race model, indicative of multisensory processing, were found for VA stimuli in both experiments and for VT stimuli in Experiment 1. Across participants, the strength of this effect was not associated with prior learning experience and daily use of smartphones. This indicates that this integration effect, similar to what has been previously reported for the integration of semantically meaningless stimuli, could involve bottom-up driven multisensory processes. Our study demonstrates for the first time that multisensory processing of smartphone stimuli facilitates taking a call. Thus, research on multisensory integration should be taken into consideration when designing electronic devices such as smartphones

Illustration of the stimulation sequence.

<p>A continuous stream of unisensory visual (V), auditory (A), and tactile (T), bisensory VA, VT, VA, and trisensory VAT stimuli was presented in random order.</p

Miller's inequality: statistical outcome of significant t-tests between cumulative probabilities from empirical data for bimodal stimulation and the corresponding race model.

<p>Miller's inequality: statistical outcome of significant t-tests between cumulative probabilities from empirical data for bimodal stimulation and the corresponding race model.</p

Redundant target effects: statistical outcome of post-hoc t-tests between RTs to bimodal and unimodal stimulation as well as trimodal and bimodal stimulation.

<p>Redundant target effects: statistical outcome of post-hoc t-tests between RTs to bimodal and unimodal stimulation as well as trimodal and bimodal stimulation.</p

Response times (RTs) for all stimuli, and cumulative probability distributions and Miller's inequality for unisensory and bisensory stimuli.

<p>a) RTs to unisensory, bisensory, and trisensory stimuli in Experiment 1 (left panel) and Experiment 2 (right panel). In both experiments RTs were shortest for tactile stimuli and shorter for auditory than for visual stimuli. Moreover, RTs to bisensory stimuli were shorter than the RTs to the respective unisensory constituents. However, RTs to trisensory stimuli did not differ from the responses of the fastest bisensory stimulus combination (i.e. AT stimuli). b) Cumulative probability distributions and Miller's inequality for unisensory stimuli and bisensory stimulus combinations in Experiment 1 (left panel) and Experiment 2 (right panel). Following the criterion by Kiesel et al. (2007), violations of Miller's inequality were found for VA stimuli (upper column) in both experiments and for VT stimuli in Experiment 1 (middle column). No significant violations were observed for AT stimuli.</p