In collaboration with Dr. Jennifer Shields (Department of Architecture), this project investigates the combined application of spatial computing and Emotion AI (artificial intelligence focused on recognizing human emotional states in real time) at cultural heritage sites. Our aim is to explore how these technologies can enrich cultural interpretation, making experiences more immersive, adaptive, and accessible.
Spatial computing offers the potential to enhance multi-sensory engagement and deepen contextual understanding for all visitors, particularly those with cognitive or sensory impairments. By embedding video, audio, and interactive elements into the physical environment, spatial computing can augment the visitor’s experience in ways that are both intuitive and inclusive. Emotion AI complements this by enabling real-time assessment of visitor engagement through neurological and physiological signals. This allows us to identify emotional triggers and patterns of attention, making it possible to personalize content delivery and tailor experiences to individual needs and preferences.
As a case study, we are focusing on the 15th-century Wildmen statues at the entrance of Hearst Castle. With the support of Cara O’Brien, Director of the Castle, we are developing a permanent exhibit for the Visitor Center Museum. This exhibit will feature video and audio projections to highlight the statues’ historical and cultural significance, while also providing accessible interpretive content for visitors with sensory impairments.
We will conduct studies using eye-tracking technology and consumer-grade wireless EEG devices to explore more effective methods for integrating digital content into physical environments. These studies will evaluate how visitors respond both emotionally and cognitively to different modes of presentation.
Beyond research outcomes, this project also contributes to education by offering students from architecture and computer science the opportunity to participate in a meaningful learn-by-doing experience. Students engage in real-world exhibit design, the application of artificial intelligence for enhancing human experience, and the development of technologies that foster deeper, more inclusive connections between people and cultural heritage.
Funding
Funded by the Research, Scholarly, and Creative Activities Grant Program (RSCA) from the Division of Research and the Summer Undergraduate Research Program (SURP) of the College of Engineering (CENG) at California State University, San Luis Obispo.