Adam D. Sitz



Download Project (556 KB)


Understanding how mental resources are utilized for different tasks is crucial for optimizing performance and avoiding error. To this end, psychologists have identified several key dimensions that can be used to distinguish multiple mental resources. Such dimensions include, among others, the type of information presented in a task and the manner with which information is presented. The present experiment evaluated participant performance in the context of a single comprehension task comprised of two components each of which was matched with a distinct modality across conditions. One task component required participants to respond to critical signal phrases (example: Eagle 1 Hostile North Lead Group 43 Miles) as they were presented in a continuously updating list of neutral signal phrases (example: Viper 2 Contact North Trail Group 50 Miles). The other task component required participants to comprehend and retain news articles as they were presented over time. For every condition, each component was presented in one of two modalities, audio or text, such that all four possible combinations of task component (information type) and modality (manner of presentation) were examined. Evaluated together, the detection of critical signal phrases and retention of news article content (measured by means of a multiple-choice questionnaire) determined total task performance. My hypothesis predicted that the two combinations featuring incongruent (non-matching) presentation modalities would show better performance over the two congruent (matching) combinations of presentation modalities because of the shared use of multiple mental resources when the modalities were different. However, results indicate that in addition to cross modality interaction, the interaction between modality and task component was another necessary factor in predicting task performance. For example, because the task in this experiment requires comprehension of the written word, the longer information access time provided by the text modality has an advantage over the temporally limited audio modality.

Publication Date


Project Designation

Honors Thesis

Primary Advisor

Susan T. Davis

Primary Advisor's Department



Stander Symposium poster

Multimodal Evaluation of Resource Allocation in a Comprehension Task