To address the limitations of human-computer interfaces (HCIs) that rely purely on visual means of information communication, it is necessary to expand information communication to non-visual modalities. This project examines the viability of using sonification, or non-speech audio, and the parameters of pitch, intensity, and tempo, as part of non-visual HCIs by assessing the accuracy of listener comprehension of sonifications.

Specifically, this study explores both the accuracy with which a listener can visually replicate an auditory pattern and the cognitive context of this audio-to-visual translation. It was hypothesized that an untrained listener presented with a sonification would be able to create a corresponding visual representation with a high degree of accuracy (Î¼>50%), and that certain characteristics of the visual graph (i.e. the presentation of axes) would affect the nature and accuracy of this replication. A group of 50 subjects spanning demographic categories was given a set of 12 sonifications and asked to create corresponding visual representations. The control group was given only blank space to create the graph on, while the experimental group was given blank space, axes, and grid points, each for 4 trials.

The results indicate a 76% accuracy rate in users visually replicating an auditory pattern, showing evidence that users can extract and comprehend meaning from the patterns in a sonification. The analysis shows slight differences in accuracy levels between the experimental and control groups with significant differences by key demographic factors.

In conclusion, the hypothesis was supported by the data, showing the viability of sonification for information communication. By examining comprehension through the industry-relevant method of visual replication rather than previous experiments in pattern matching, this study demonstrates the potential of applying sonification to overcome visual fatigue in time-critical battlefield scenarios, provide HCIs for the visually impaired, and facilitate data analysis through multimodal information communication, among other fields.

Neel Patel

Affiliation

Refine Results

Listener Comprehension and Reaction-Time Study Of Sonification

Studying Listener Comprehension of Sonifications through Visual Replication