Digital Dérive: Reconstructing Urban Environments based on Human Experience

AbstractThis paper describes a novel method for reconstructing urban environments based on individual occupant experience. The method relies on a low-cost off-the-shelf 360o camera to capture video and audio data from a natural walk through the city. It then uses a custom workflow based on an open-source Structure from Motion (SfM) library to reconstruct a dense point cloud from images extracted from the 360o video. The point cloud and audio data are then represented within a virtual reality (VR) model, creating a multisensory environment that immerses the viewer into the subjective experience of the occupant.This work questions the role of precision and fidelity in our experience and representation of a “real” physical environment. On the one hand, the resulting VR environment is less complete and has lower fidelity than digital environments created through traditional modeling and rendering workflows. On the other hand, because each point in the point cloud is literally sampled from the actual environment, the resulting model also captures more of the noise and imprecision that characterizes our world. The result is an uncanny immersive experience that is less precise than traditional digital environments, yet represents many more of the unique physical characteristics that define our urban experiences.