Nonlinear Interactive Instruction: Embodied Learning Systems

Building on that, starting not with space, but with instruction itself.

Role UX Design Lead, Developer
Timeline 15 Weeks
Year 2024
Project GSAPP-CDP

Starting not with space, but with instruction itself.

How do you learn a craft technique? You don't read about it. You watch. You mimic. You adjust by feel.

Through speculative interfaces and real-time gesture tracking using Procession (developed by Spatial Pixel), I explored whether embodied, nonlinear learning could be digitized: Could a digital system teach not through rigid steps, but through movement, experimentation, and adaptation?

Diagram with pics

I found that craft learning is inherently nonlinear, cyclical, and deeply resilient—but also fragile when cut off from embodied practice.

Process Image 1 Process Image 2

đź§¶ Why Craft Gestures Matter

Question

Artisans tell stories through their hands. A slight pressure, a wrist turn, a coiling motion — these are more than just physical acts. They are repositories of tradition, intuition, and embodied skill.

But unlike spoken language, these gestures often go undocumented. When they disappear, we lose more than a technique—we lose culture, relationships to materials, and ways of inhabiting the world. Craft gestures are dynamic, not static. Yet most documentation—photographs, videos, written guides—flattens them into static representations. They fail to capture the richness of movement: its timing, its evolution, its silent dialogue with material.

Gesture

To understand this, I started by mapping the gestures of traditional pottery techniques. I found that each technique is a dance of hands, fingers, and wrists—a subtle conversation with clay that can't be captured in a single image or video.

How do you teach the slight press of fingers flattening clay?

How do you capture a thumb's subtle rotation into the curve of a bowl?

How do you guide a hand's gentle pull into the perfect curvse of a vase?

How do you teach the subtle pressure of a thumb against clay to create a smooth surface?

How do you guide a hand's gentle pull into the perfect curve of a vase?

Technical Details

The system utilizes:

3D Visualization Engine

Three.js-based 3D environment with interactive technique nodes and spatial navigation

Adaptive Learning Algorithm

Machine learning system that tracks user progress and suggests personalized learning paths

Interactive Content Management

Dynamic content delivery system that adapts to user skill level and learning preferences

Multi-Modal Learning Interface

Web-based platform supporting video, 3D models, text, and interactive elements