Publication | ACM International Conference on Interactive Tabletops and Surfaces 2022

VideoPoseVR

Authoring Virtual Reality Character Animations with Online Videos

VideoPoseVR reconstructs 3D pose and motion from 2D videos to enable rapid prototyping of character animation in VR. Users without animation experience can import and convert videos into a 3D motion dataset.

This paper introduces a video-based animation authoring workflow using online videos to author character animations in virtual reality (VR). It leverages the state-of-the-art deep learning approach to reconstruct 3D motions from online videos, caption the motions, and store them in a motion dataset. Creators can import the videos, search in the dataset, modify the motion timeline, and combine multiple motions from videos to author character animations in VR.

Download publication

Abstract

VideoPoseVR: Authoring Virtual Reality Character Animations with Online Videos

Cheng Yao Wang, Qian Zhou, George Fitzmaurice, Fraser Anderson

ACM International Conference on Interactive Tabletops and Surfaces 2022

We present VideoPoseVR, a video-based animation authoring workflow using online videos to author character animations in VR. It leverages the state-of-the-art deep learning approach to reconstruct 3D motions from online videos, caption the motions, and store them in a motion dataset. Creators can import the videos, search in the dataset, modify the motion timeline, and combine multiple motions from videos to author character animations in VR. We implemented a proof-of-concept prototype and conducted a user study to evaluate the feasibility of the video-based authoring approach as well as gather initial feedback of the prototype. The study results suggest that VideoPoseVR was easy to learn for novice users to author animations and enable rapid exploration of prototyping for applications such as entertainment, skills training, and crowd simulations.

Related Resources

Publication

2024

Experiential Views: Towards Human Experience Evaluation of Designed Spaces using Vision-Language Models

Exploratory research on helping designers and architects anticipate…

Publication

2018

Blocks-to-CAD: A Cross-Application Bridge from Minecraft to 3D Modeling

Learning a new software application can be a challenge, requiring the…

Publication

2020

AuthAR: Concurrent Authoring of Tutorials for AR Assembly Guidance

Augmented Reality (AR) can assist with physical tasks such as object…

Publication

2022

It’s Over There: Designing an Intelligent Virtual Agent That Can Point Accurately into the Real World

It is challenging to design an intelligent virtual agent (IVA) that…

Get in touch

Something pique your interest? Get in touch if you’d like to learn more about Autodesk Research, our projects, people, and potential collaboration opportunities.

Contact us