ClayDream is an interactive system that turns clay sculpting and voice prompts into AI-generated visual variations in real time.
ClayDream is an embodied AI prototype that connects physical clay manipulation, voice input, hand tracking, and generative visuals. Users work with clay on a tabletop interface while the system captures the evolving form and translates it into a changing field of AI-generated visual variations.
Role
Concept, interaction design, and technical prototyping
Team
Solo project
Institution / Year
2026
Tools
Computer vision | Hand tracking | Voice input | Generative AI
Most generative AI interfaces begin with text and stay on screen. ClayDream starts from physical making instead, asking how a soft material, a hand gesture, and a spoken prompt can become inputs for computational imagination. The project positions clay as a bridge between tactile craft and AI-assisted visual exploration.
The concept combines four input and interpretation layers: voice becomes a prompt, the clay object is tracked as form, the user's hands are tracked as action, and an AI generative layer produces visual outputs that overlap with the physical process. This creates a feedback loop where making, describing, and seeing influence one another.
The prototype is built around a tabletop clay station paired with a display interface. As the user manipulates the clay, the system documents variations and presents AI-generated interpretations on screen. The experience makes generative AI feel less like a remote text box and more like a responsive creative partner embedded in the act of sculpting.
Development focused on designing the physical station, mapping the system flow, and testing how hand, clay, and voice inputs could be synchronized. The project assets document both the system diagram and the final interactive setup, including the display language and clay tracking interface.
ClayDream suggests a more embodied direction for AI tools, where computation supports material experimentation instead of replacing it. The project highlights how tactile input can make generative systems feel more playful, situated, and physically intuitive.
