An inquiry into active spatial language using virtual reality
International Conference on Spatial Cognition, 2018. Rome, Italy.
Spatial perception is an essential aspect of human intelligence, which among other things, enables us to describe, learn and remember the world around ourselves. Thus, understanding the formation of spatial concepts and their use in active spatial experiences is an important challenge for the study of spatial perception. However, there has not been a comprehensive study for characterizing the use of active spatial language in diverse observational tasks in real world conditions.
In this research we aim to understand the interactions between visual space perception and language in various environmental conditions and tasks. In particular, we investigate spatial concepts in language that are used during active explorations and characterize them in relation to corresponding visuospatial context. We introduce an immersive virtual reality system that allows conducting spatial exploration tasks and recording perceptual and language data. Using this system, we conduct an experiment with 16 subjects, and generate a novel dataset, which includes audio recordings and time coded transcriptions, visual modalities such as RGB and depth, and camera extrinsic for a 10-minute exploration per subject.
We present a benchmark quantitative analysis of observational statements regarding their referential manner, explicit use of spatial descriptions, temporal continuity, as well as the degree of correspondence between the verbal statements and available perceptual data.
Our research contributes to development of immersive data collection and experimentation methods for the study of spatial perception. Our initial study enables characterization of active spatial language in relation to available spatio-temporal context and opens up possibilities for further inquiries.
