Design Concept and Artistic Direction
Last year, during the Interactive Media (IM) show, I remember engaging with a fascinating sketch by another student. Their project used a webcam to capture a photo, which was then transformed into a shuffled grid puzzle for users to solve. I was inspired by the creative use of the webcam and wanted to recreate something similar, but with a different level of complexity and interaction. After brainstorming, I shifted the idea toward using a webcam not just to capture but to enable free drawing with hand gestures. This led to the concept of Neon Doodle World, where users could draw vibrant doodles with their fingers in real-time.
To make the experience even more engaging, I incorporated features inspired by concepts we learned in class. In addition to a standard paintbrush mode, I added a particle trail drawing mode for dynamic, animated effects. Users can also clear the sketch at any time, ensuring the canvas remains a space for unrestricted creativity.
Interaction Methodology
The interaction design leverages MediaPipe Hands, a library for real-time hand-tracking. The webcam detects hand landmarks, allowing users to control the drawing with simple gestures:
- Paintbrush Mode: Users draw strokes in a chosen color. A pinch gesture (bringing the thumb and index finger close) changes the color dynamically.
- Particle Trail Mode: A more playful drawing style where colorful, randomized particles trail the user’s hand movements.
- Clear Canvas: Users can clear the drawings with a single click, resetting the canvas for a fresh start.
The key interaction focus was to make the experience intuitive. By mapping hand coordinates to the canvas, users can freely draw without needing a mouse or keyboard. The pinch gesture ensures seamless control of color switching, and the modes allow creative exploration.
Canvas Design and Interaction Plan
The canvas is divided into two main components: the video feed and the drawing layer. The webcam feed is mirrored horizontally, ensuring the gestures feel natural and intuitive. The drawing layer exists as an overlay, maintaining a clear separation between the user’s input and the video feed.
Above the canvas, simple buttons control the modes and functionalities. Users can easily toggle between paintbrush and particle trail modes, clear the canvas, and change drawing styles. This clean and straightforward layout ensures accessibility while minimizing distractions from the creative process.
Base p5 Sketch and Initial Explorations
The Draft 1 sketch built the basic functionality of the project:
- It uses MediaPipe Hands to track hand movements and map them to the canvas.
- The paintbrush mode lets users draw smooth, colorful lines.
The particle trail mode creates fun, animated effects with random bursts of color. - A clear canvas button resets the drawing layer without touching the webcam feed.
Here’s an example of how the particle trail mode works:
} else if (mode === "particle-trail") {
// Create particle effects around the finger position
for (let i = 0; i < 5; i++) {
let particleX = x + random(-10, 10); // Random x-offset for particles
let particleY = y + random(-10, 10); // Random y-offset for particles
drawingLayer.fill(random(100, 255), random(100, 255), random(100, 255), 150); // Random color with transparency
drawingLayer.noStroke();
drawingLayer.ellipse(particleX, particleY, random(5, 10)); // Draw a small particle
}
}
Current Sketch
Next Steps and Expansions
Building on the Draft 1 sketch, the project will be expanded with several new features to improve interaction and enhance the user experience:
- Introduction Page: A new page will be added at the beginning to introduce the project, setting the tone and providing users with context about the experience they are about to have.
- Color Palette: A palette will be added to allow users to select specific colors for drawing, instead of cycling through preset options. This will make the experience more customizable.
- Undo Feature: Functionality will be introduced to undo the last drawing action, giving users more control and flexibility while drawing.
- Save Feature: Users will be able to save their creations as images, including both the drawings and the webcam overlay.
- Pause/Resume Drawing: A pause button will be added, allowing users to stop and resume drawing as needed.
These additions will make the project more interactive, user-friendly, and visually appealing, while staying true to the original concept of creating a dynamic, webcam-based drawing experience.