Project Quill: Sketch It. Say It. Shape It.

Historically, it’s been challenging to take an idea in your mind and turn it into a 3D design. With Project Quill, we’re developing new and more natural ways to express your design intent and quickly create, and then refine, your designs. In particular, we’re looking at pen and voice input, and finding that combined they can be much more powerful than either alone.

Figure 1. The sculpture on the left would be hard to describe in words alone, but one could easily sketch its general shape. On the other hand, the structure on the right would be hard to draw correctly, but is very easy to describe in words (“the Eiffel tower”)

Some things are easy to draw, but hard to describe, while others are easy to describe, but hard to draw (Figure 1). By allowing you to do both at once, you can work naturally, drawing as best you can, while describing what you’re trying to do along the way. In this way, pen and voice input together can compensate for the other’s shortcomings, creating a more natural way to work.

Of course, we need a way to take these pen and voice inputs and create realistic outputs, and we’re now able to do that with the latest AI models, coupled with our novel interaction techniques. We can feed in the sketch and voice input, along with other relevant information (for example: the type of building, the plot size, the location, the style of building, etc.), and create a design iteration (Figure 2).

Figure 2. Example of a building created with Project Quill, starting from a rough sketch, which goes through gradual improvements through voice and pen annotations – finally resulting in a 3D model.

Perhaps more importantly, the result that comes back from the model does not have to be the “final” design. You can continue to mark up the design with further refinements using the pen or describe the changes you’d like to see out loud. By allowing both options, you can use whichever form of input feels the easiest in the moment.

This style of interaction could be used in many domains, but we are particularly interested in how it could be integrated into Forma and Autodesk Assistant to let people rapidly create and edit architectural designs. At AU 2026 attendees can experience Project Quill to create their own buildings (Figure 3).

Figure 3. Sketching and talking about your building design (left), seeing a designed 3D model (middle), and placing the designed building into the cityscape (right).

As we continue with this research, we see what possibilities freeing professionals from working only through a keyboard and mouse can enable. After a lifetime spent learning machine languages and using only a keyboard and mouse to convey our ideas, the machines are learning how to interpret our languages, our sketches, and our handwritten notes. For the first time, machines are now meeting us on our territory. The impact this will have on how humans interact with machines will be profound. And Project Quill points to this future when we express ourselves more naturally and work improves and accelerates as a result.