Crafticulate

A gesture-based archive that preserves traditional craft knowledge through the lens of technology. Focused on the embodied movements of pottery-making, the project captures, classifies, and reanimates hand gestures using real-time machine learning and computer vision.

Inspired by how techniques are passed through demonstration rather than text, Crafticulate transforms ephemeral acts into interactive digital memory. Each gesture—pinching, pulling, smoothing—becomes a traceable artifact, stored not as video, but as data: adaptable, searchable, and open to reinterpretation.

Users explore the archive through an interface that is itself a site of learning. They can view 3D gesture visualizations, listen to artisan reflections, and even try gestures live using their own hands via webcam. The system responds in real time, offering feedback and inviting users to contribute their own interpretations.

At its core, Crafticulate reimagines preservation not as storage, but as dialogue: between human and machine, memory and movement, tradition and transformation.

"how can we preserve gestures as living knowledge, not just recorded motion?"

The project was developed through iterative prototyping and real-time testing. I started by analyzing pottery-making gestures through observational video, then trained a gesture recognition model using MediaPipe and TensorFlow. From there, I built a 3D visualization interface to explore movement beyond flat video. A physical tray of clay forms was also created to ground the digital gestures in tactile reality.

Each user interaction feeds the system—expanding the dataset and refining the model to better reflect the diversity of embodied craft.

When you visit Crafticulate, you begin at the Gesture Gallery — a constellation of floating pottery gestures you can explore freely.

You can filter gestures by:

• Process (e.g., forming, shaping)
• Stage (e.g., centering, finishing)
• Application (e.g., bowl-making, vase-making)
• Hand Use (single-hand, double-hand)
• Technique (e.g., coiling, pinching)
• Cultural Context (comparisons across traditions)

🎟️ Following a Gesture Journey

Suppose you choose bowl-making. You're guided into a process timeline—almost like a recipe made of movements.

Each gesture flows into the next: kneading, pinching, pulling, lifting—each shaping the clay toward form.

🔍 Diving Deeper: Gesture Metadata Pages

When you click on a gesture—say, Ball Kneading—you open its deeper archive page, revealing:

• A 3D model of the gesture, which you can rotate and animate.
• A description of when and why this movement is used.
• The tool or material involved (hands, ribs, wheels).
• Voice reflections or short clips from artisans sharing personal insights.
• Cultural overlays comparing similar gestures across global craft traditions.

👐🏻 Practicing the Gestures Yourself

When ready, you can turn on your webcam and practice a gesture live. A real-time hand tracking model maps your movements through 21 keypoints, offering visual feedback—like a digital mirror guiding your hands.

Because gestures live on a spectrum, the system doesn't enforce rigid categories. Small variations are part of learning, just as they are in real craft.

The system utilizes: