Neon Doodle World – Final Project

Concept

The inspiration for Neon Doodle World came from my fascination with interactive media and real-time creativity. Initially inspired by a project at the IM show that used a webcam for puzzle-based interaction, I wanted to expand on the idea by enabling freehand drawing using hand gestures. The goal was to create an intuitive and engaging platform where users could unleash their creativity through dynamic drawing features.

Neon Doodle World lets users draw directly on a digital canvas using their hands, with options for paintbrush strokes, playful sprinkle trails, color customization, and more. The neon-themed interface enhances the visual experience, creating an immersive environment for artistic expression.

Interaction Methodology

The project uses MediaPipe Hands for real-time hand tracking. The webcam detects the user’s hand movements, focusing on key points like the index finger, to map gestures onto the canvas. Here’s how users interact with the system:

Multi-Hand Support: Users can now draw freely using both hands simultaneously, doubling the creative possibilities.
Paintbrush Mode: Draw clean, solid lines in any color by moving your index finger.
Sprinkle Trail Mode: Generate dynamic sprinkle/particle effects for playful, animated drawings.
Eraser Mode: Remove specific parts of your drawing with precision.

Toolbar Options:

- - Clear: Wipe the entire canvas.
  - Undo: Remove the last action for greater control.
  - Pause/Resume: Temporarily disable or resume drawing.
  - Save: Capture and save your creation along with the webcam feed.

Hover Tooltips: Each toolbar button displays clear instructions when hovered over, making the interface user-friendly for all experience levels.
Brush Size Adjustment: A slider allows users to adjust brush size, enabling detailed or bold strokes depending on their creative needs.

The webcam feed is mirrored horizontally to make interactions feel natural, ensuring seamless mapping between gestures and canvas movements. A dynamic toolbar and hover instructions simplify the interface, ensuring accessibility and ease of use.

Images of my Sketch

How the Implementation Works

The application uses p5.js and MediaPipe for its core functionalities:

Hand Tracking: MediaPipe detects up to two hands simultaneously, updating their positions to control canvas actions.
Drawing Logic: Depending on the selected mode, the system renders strokes, particles, or erases sections in the drawing layer.
Undo and Redraw: All actions are stored in a stack, allowing users to undo changes by redrawing the canvas without the last action.
Color Palette: A scrollable palette lets users easily switch between vibrant neon colors.
Brush Size Adjustment: Users can control the size of their brush using a slider, displayed dynamically for feedback.
Save Feature: Combines the live webcam feed and the drawing layer into a single image file.

Code Highlight

One of the standout features is the Undo Functionality. Here’s how it’s implemented:

function undoLastAction() {
  if (actionsStack.length > 0) {
    actionsStack.pop(); // Remove the last action
    redrawCanvas(); // Redraw the canvas to reflect changes
  }
}

function redrawCanvas() {
  drawingLayer.clear(); // Clear the drawing layer
  actionsStack.forEach((action) => {
    if (action.type === "ellipse") {
      // Redraw ellipses
      drawingLayer.noStroke();
      drawingLayer.fill(action.color);
      drawingLayer.ellipse(action.x, action.y, action.size, action.size);
    } else if (action.type === "sprinkle") {
      // Redraw sprinkle trails
      action.particles.forEach((particle) => {
        drawingLayer.fill(random(100, 255), random(100, 255), random(100, 255), 150);
        drawingLayer.noStroke();
        drawingLayer.ellipse(particle.x, particle.y, random(5, 10));
      });
    }
  });
}

Embedded Sketch

User Testing

I asked my friend to test out the sketch, and she enjoyed the freedom of drawing with hand gestures. She decided to draw the UAE flag as a tribute to National Day, appreciating the vibrant color palette and smooth interaction that made it easy to bring her idea to life. Here’s the testing clip:

Reflection

Creating Neon Doodle World has been an incredible learning journey. I’m particularly proud of the following aspects:

Hover Instructions: These made the interface more accessible and beginner-friendly.
Adjustable Brush Size: This feature added a new dimension of control for users.
The Neon-Themed Interface: This enhances user immersion with vibrant visuals.
Added Features: Undo, save, and dynamic color selection significantly improved the final experience.

However, there are always areas to improve:

Adding gesture-based controls for even more intuitive interactions (e.g., using a hand gesture to clear the canvas).
Expanding the drawing modes with advanced brush effects or textured strokes.
Introducing background effects, such as gradients or animated visuals, to add depth to the canvas and enhance user engagement.

Challenges and Overcoming Them

Some key challenges I faced:

Multi-Hand Implementation: Tracking and rendering independent actions for two hands required a flexible system to handle simultaneous inputs.
Hover Tooltips: Designing tooltips that provide clear information without cluttering the interface was a balancing act.
Brush Size Slider: Ensuring the slider dynamically updated both brush size and its display required careful synchronization.

Resources

https://emiguerra.github.io/p5MediaPipe/

https://github.com/mekiii/P5-Particle-Handtracking

https://www.youtube.com/watch?v=BX8ibqq0MJU

https://editor.p5js.org/shinjia168/sketches/NXUrGxzye

IM Show Documentation
At the IM Show, I showcased Neon Doodle World to students and faculty. It was exciting to see people interact with the sketch, experimenting with features like the particle trails and color palette. I also shared the project with my former professors, Evi Mansor and Aya Riad, and the head of Interactive Media, Michael Shiloh, who all gave positive feedback. Here’s a video of people testing and enjoying the sketch:

Leave a Reply Cancel reply