Previous reports have demonstrated that the comprehension of sentences describing motion in a particular direction (towards, away, up, or down) is affected by concurrently viewing a stimulus that depicts motion in the same or opposite direction. We report three experiments that extend our understanding of the relationship between perception and language processing in two ways. First, whereas most previous studies of the relationship between perception and language processing have focused on visual perception, our data show that sentence processing can be affected by the concurrent processing of auditory stimuli. Second, it is shown that the relationship between the processing of auditory stimuli and the processing of sentences depends on whether the sentences are presented in the auditory or visual modality.