Recently Published by Autodesk Researchers

Autodesk Research teams regularly contribute to peer-reviewed scientific journals and present at conferences around the world. Check out some recent publications from Autodesk Researchers.

WhatELSE – Shaping Narrative Spaces at Configurable Level of Abstraction for AI-bridged Interactive Storytelling

Generative AI significantly enhances player agency in interactive narratives (IN) by enabling just-in-time content generation that adapts to player actions. In this paper, the team presents WhatELSE, an AI-bridged IN authoring system that creates narrative possibility spaces from example stories. WhatELSE provides three views – narrative pivot, outline, and variants – to help authors understand the narrative space and corresponding tools leveraging linguistic abstraction to control the boundaries of the narrative space. Taking innovative LLM-based narrative planning approaches, WhatELSE further unfolds the narrative space into executable game events.

In-Context Imitation Learning via Next-Token Prediction

This paper explores how to enable in-context learning capabilities of next-token prediction models for robotics, allowing the model to perform novel tasks by prompting it with human teleop demonstration examples without fine-tuning. In-Context Robot Transformer (ICRT) is a causal transformer that performs autoregressive prediction on sensorimotor trajectories, which include images, proprioceptive states, and actions. This approach allows flexible and training-free execution of new tasks at test time, achieved by prompting the model with demonstration trajectories of the new task. In a multi-task environment setup, ICRT significantly outperforms current state-of-the-art robot foundation models on generalization to unseen tasks.

A Contactless Multi-Modal Sensing Approach for Material Assessment and Recovery in Building Deconstruction

As material scarcity and environmental concerns grow, material reuse and waste reduction are gaining attention based on their potential to reduce carbon emissions and promote net-zero buildings. This study develops an innovative approach that combines multi-modal sensing technologies with machine learning to enable contactless assessment of in situ building materials for reuse potential. This research indicates that contactless assessment and automated deconstruction methods are technically viable, economically advantageous, and environmentally beneficial. Serving as an initial step toward novel methods to view and classify existing building materials, this study lays a foundation for future research, promoting sustainable construction practices that optimize material reuse and reduce negative environmental impact.