All in Virtual Reality

The Latent Archive

National Endowment for the Humanities Digital Humanities Advancement Grant. 2022

Previously, extracting information from moving images has been challenging and time consuming, requiring historians and film scholars to access footage by manually reviewing sequences over and over to parse the setting, the rituals, camera angle, narratives, and the material cultures involved. Now, developments in computer vision and spatial analysis technologies have opened up exciting possibilities for these scholarly processes, with direct implications for improved public access and future translational tools for disabled communities. The “latent archive” that has always been embedded in moving images can now be captured via machine-enabled analysis: locating the urban or architectural setting, producing 3D spatial reconstructions, and allowing fine-grained examination of point-of-view and shot sequence.

September 1955 - A Virtual Documentary of the Istanbul Pogrom

Virtual Experience Design Lab, 2017

September 1955 is a 8-minute virtual-reality documentary of the Istanbul Pogrom, a government-initiated organized attack on the minorities of Istanbul on September6-7, 1955. This interactive installation places the viewer in a reconstructed photography studio in the midst of the pogrom, allowing one to witness the events from the perspective of a local shop-owner.

Virtuality & Presence

This course aims to help students invent and analyze new forms of extended reality (XR) experiences, computer-based art, gaming, social media, interactive narrative, and related technologies. This semester’s focus is on the theory, design, and implementation of experiences and technologies of virtuality. Toward this end, we shall look at topics including definitions of virtual, virtual reality, augmented reality, alternate reality, hybrid reality, virtual worlds, virtual selves, and more.

Immersive Media

In this course, students develop independent projects in immersive media as it relates to architectural design and spatial storytelling. We examine technologies of immersion including VR/AR, web-based experiences, sound installations, and virtual production workflows. We explore the field of immersive media through readings, precedent artwork, and hands-on experience with production tools. We specifically focus on the production pipelines that use real-time simulations and game engines. Students were encouraged to work on the aspects of their studio projects and thesis. The workshop provides a studio environment for the students to collaborate and provide feedback to each other.

Reasonable Perception: Connecting Vision and Language Systems

Human Robot Interaction 2018. ACM.

Understanding explanations of machine perception is an important step towards developing accountable, trustworthy machines. Furthermore, speech and vision are the primary modalities by which humans collect information about the world, but the linking of visual and natural language domains is a relatively new pursuit in computer vision, and it is difficult to test performance in a safe environment. To couple human visual understanding and machine perception, we present an explanatory system for creating a library of possible context-specific actions associated with 3D objects in immersive virtual worlds.

MIT.nano Opening Event

VR/AR Exhibition. 2018.

MIT Virtual Experience Design Lab was commissioned by MIT nano leadership for an immersive experience to be exhibited during the opening ceremony of MIT nano, the largest and most sophisticated nano science laboratory in US. At the core of the activities in the lab is the use of advanced instruments of measurement and vision in the nanoscale. At this scale, the air should be so clean (class 100 cleanroom means no more than 100 particles in a cubic meter of air is allowed) that without special protocols no one is allowed inside the spaces.

Design Tools of The Future Hackathon

Hackathon Organization as part of ACADIA 2017. MIT Media Lab.

How does a VR tool work in the future to help an architect presenting a design of a new public school to the local community? What is an AR tool that real estate brokers can use to help promote their properties to home hunters? What is a future MR tool on the construction site that helps building faster, safer and more economically?

Focusing on challenges such as these in the real design world context, including but not limited to architectural, landscape and product design as well as urbanism, real estate and construction, this hackathon fostered an interdisciplinary environment for proposing strong innovative solutions to the future

Mediate VR

MIT DesignX Startup Accelerator Challenge Winner. 2017.

Mediate VR is a platform for speech-driven user research in virtual reality. It captures the emotions, challenges, and pleasures of spatial experience through voice recordings.

Users explore a virtual environment, respond verbally to prompts, and engage in tasks. People don’t remember the details of experiences very well after-the-fact; eyewitness accounts are notoriously unreliable. But ask someone what they think or see during an event, and they can give specific, accurate feedback. Mediate contextualizes user voice recordings through data captured from the virtual environment, synthesizing insights and feedback in real time for our clients through an admin dashboard

nRoom: an immersive virtual environment for collaborative spatial design

CHIuXiD '15: Proceedings of the International HCI and UX Conference in Indonesia

In this paper, we present the results of an experimental study aiming to explore the collaborative user experience in an immersive virtual environment. We designed and implemented an application that enables users to collaboratively design a spatial layout using head-mounted VR displays and hand tracking devices. With a strong emphasis on the relationship between spatial interaction and communication, we assert that a shared-view virtual environment allows collaborative articulation of spatial design problems and improves communication between designers. Our study combines qualitative and quantitative methods to test the usability of the proposed system, and to determine the aspects of spatial communication in virtual environments.