The impact of temporal and spatial consistency of audiovisual stimuli on Pip-and-Pop effect

Subjects: Psychology >> Cognitive Psychology submitted time 2022-06-14

唐晓雨 崔鑫忠 高敏 袁梦莹

Abstract：

Previous studies usually used the dynamic visual search paradigm to explore the Pip-and-Pop effect, which suggests that the Pip-and-Pop effect is influenced by multiple factors. But the impact of temporal and spatial consistency of audiovisual stimuli on the Pip-and-Pop effect is still controversial. In present study, we applied the eye movement technology to the dynamic visual search paradigm to systematically investigate the impact of temporal and spatial consistency of audiovisual stimuli on the Pip-and-Pop effect. Based on the results of previous studies, we expected that the spatial and temporal consistency of audiovisual modulated the Pip-and-Pop effect. The more consistent the spatial location of audiovisual stimuli are, the greater the Pip-and-Pop effect will be. The more consistent the temporal of audiovisual stimuli are, the greater the Pip-and-Pop effect will be.

Experiment 1 was a 3 (set sizes: 36, 48, 60) × 4 (spatial consistency conditions: ipsilateral, contralateral, bilateral, no sound) within-subjects design, recruited 24 participants(5 males; age range: 19–28 years; mean age: 22.5±2.3 years). The visual search displays consisted of 24, 36, or 48 red (13.9 cd/m²) or green (46.4 cd/m²) line segments (0.57°×0.17°) on a black (0.4cd/m²) background. Auditory stimulus was 1000Hz pure tone (65 dB, 60ms, including 5ms fade-in and 5ms fade-out time), presented through speakers on the left and right sides behind the screen.  Each trial started with a drift calibration point. The drift calibration point would not disappear until the participant gazed upon this point. This was followed by a central fixation point of 1000ms, and finally a search screen. The participants were required to find the target line segment and judge whether the target line segment was vertical or horizontal by pressing the key (Z key or M key) as quickly and accurately as possible. Experiment 2 was a 2 (set sizes: 36, 60) × 6 (temporal consistency conditions: -200ms, -100ms, 0ms, 100ms, 200ms, no sound) within-subjects design, recruited 27 participants(5 males; age range: 18–25 years; mean age: 20.7±2.4 years). These temporal consistency conditions represent the tone sounded before ( -200, -100ms), simultaneous with (0ms), or after (100, 200ms) the visual target event. The tone could also be absent (no sound). The experimental materials and procedures of Experiment 2 were identical to those in Experiment1.

Regarding the results in Experiment 1, compared with the baseline condition (no sound), the search response time was the shortest under the condition of ipsilateral, the mean fixation number was the least, and the mean saccade amplitude was the smallest, indicating that the search efficiency was the highest in the ipsilateral condition, the Pip-and-Pop effect was the largest in the ipsilateral condition. Bilateral condition was the second. The contralateral condition showed no significant difference in the above indicators compared with the no sound condition, indicating that no Pip-and-Pop effect was found in the contralateral condition. The results showed that the more consistent the spatial location of audiovisual stimuli were, the greater the Pip-and-Pop effect was. In Experiment 2, compared with the no sound condition, we found that the response time of search was significantly shorter, the mean fixation number in search decreased, the mean saccade amplitude increased, and the mean fixation duration became longer under the conditions of 0ms, -100ms and 100ms. Compared with the no sound condition, conditions -200ms and 200ms showed no significant difference in the above indicators. This indicated that the conditions of 0ms, -100ms and 100ms produced Pip-and-Pop effect, and the Pip-and-Pop effect was the largest under the condition of 0ms, while Pip-and-Pop effect did not find under the condition of -200ms and 200ms. The results showed that the more consistent the temporal of audiovisual stimuli were, the greater the Pip-and-Pop effect was.

In summary, the results showed that the more consistent the audiovisual stimuli were in space and time, the larger the Pip-and-Pop effect was. Therefore, the findings in the present study suggest that the temporal and spatial consistency of audiovisual stimuli modulates the Pip-and-Pop effect. The results provide evidence that the reason for Pip-and-Pop effect is multisensory integration.

Peer Review Status:Awaiting Review

 Hits 2509  Downloads 453  Comment 0