Publication | ACM Symposium on User Interface Software & Technology 2012

Waken

Reverse Engineering Usage Information and Interface Structure from Software Videos

Abstract

Waken: Reverse Engineering Usage Information and Interface Structure from Software Videos

Nikola Banovic, Tovi Grossman, Justin Matejka, George Fitzmaurice

ACM Symposium on User Interface Software & Technology 2012

We explore the possibilities and opportunities related to reverse engineering usage information from screen-captured application video tutorials. We develop an application-independent system that recognizes UI components and activities, such as cursor movements and icon clicks, from an input set of video tutorials, We then present Waken, an enhanced video player, which showcases some of the design opportunities the are introduced by having this additional meta-data. In particular, users can directly interact with UI components in the video, such as icons and menus, to display associated information, or navigate to relevant moments in other videos. Initial results suggest that the system can successfully reconstruct many aspects of a UI without any prior application-dependent knowledge.

Download publication

Related Resources

Publication

2002

Boom Chameleon: Simultaneous capture of 3D viewpoint, voice and gesture annotations on a spatially-aware display

We introduce the Boom Chameleon, a novel input/output device…

Publication

1999

Exploring interactive curve and surface manipulation using a bend and twist sensitive input strip

We explore a new input device and a set of interaction techniques to…

Publication

2004

A Remote Control Interface for Large Displays

We describe a new widget and interaction technique, known as a…

Publication

2009

Gluing constructions amongst constant mean curvature hypersurfaces in the (n+1)-sphere

Four constructions of constant mean curvature (CMC) hypersurfaces in…

Get in touch

Something pique your interest? Get in touch if you’d like to learn more about Autodesk Research, our projects, people, and potential collaboration opportunities.

Contact us