Author: Dachi Tarughishvili

Sketch:

p5.js Web Editor | Zombie Automata

Inspiration

A part of the project that I am particularly proud of is the implementation of probabilistic infection dynamics

if (state === HUMAN) {
  let neighbors = countNeighbors(i, j, ZOMBIE);
  if (neighbors > 0) {
    if (random() < 1 - pow(1 - infectionRate, neighbors)) {
      nextGrid[i][j] = ZOMBIE;
    } else {
      nextGrid[i][j] = HUMAN;
    }
  } else {
    nextGrid[i][j] = HUMAN;
  }
}

This code not only introduces a realistic element of risk-based infection but also produces visually interesting outcomes as the patterns evolve. Watching the outbreak spread dynamically based on these probabilities was quite fun.

Reflection and Future Considerations

Sketch: p5.js Web Editor | Fabrik

Inspiration

// Array to store particles within grab radius
let grabbableParticles = [];

// Scan all cloth particles
for (let i = 0; i < cloth.cols; i++) {
    for (let j = 0; j < cloth.rows; j++) {
        let particle = cloth.particles[i][j];
        if (!particle.isStatic) {  // Skip fixed particles
            // Calculate distance from mouse to particle
            let d = dist(mouseX, mouseY, particle.position.x, particle.position.y);
            if (d < DRAG_CONFIG.GRAB_RADIUS) {
                // Store particle info if within grab radius
                grabbableParticles.push({
                    particle: particle,
                    distance: d,
                    offset: {
                        // Store initial offset from mouse to maintain relative positions
                        x: particle.position.x - mouseX,
                        y: particle.position.y - mouseY
                    }
                });
            }
        }
    }
}

// Sort particles by distance to mouse (closest first)
grabbableParticles.sort((a, b) => a.distance - b.distance);
// Limit number of grabbed particles
grabbableParticles = grabbableParticles.slice(0, DRAG_CONFIG.MAX_GRAB_POINTS);

// Only proceed if we have enough particles for natural grab
if (grabbableParticles.length >= DRAG_CONFIG.MIN_POINTS) {
    grabbableParticles.forEach(({particle, distance, offset}) => {
        // Calculate stiffness based on distance (closer = stiffer)
        let constraintStiffness = DRAG_CONFIG.STIFFNESS * (1 - distance / DRAG_CONFIG.GRAB_RADIUS);
        
        // Create constraint between mouse and particle
        let constraint = Constraint.create({
            pointA: { x: mouseX, y: mouseY },  // Anchor at mouse position
            bodyB: particle,                   // Connect to particle
            stiffness: constraintStiffness,    // Distance-based stiffness
            damping: DRAG_CONFIG.DAMPING,      // Reduce oscillation
            length: distance * 0.5             // Allow some slack based on distance
        });
        
        // Store constraint and particle info
        mouseConstraints.push(constraint);
        draggedParticles.add(particle);
        initialGrabOffsets.set(particle.id, offset);
        Composite.add(engine.world, constraint);
        
        // Stop particle's current motion
        Body.setVelocity(particle, { x: 0, y: 0 });
    });
}

Sketch: p5.js Web Editor | Brindle butterkase

Inspiration

switch(this.element) {
  case elementParams.fire:
    this.pos.y -= 1;
    this.vel.x += random(-0.1, 0.1);
    break;
  case elementParams.air:
    let time = (frameCount + this.offset) * 0.01;
    let noiseX = smoothNoise(this.pos.x * 0.006, this.pos.y * 0.006, time);
    let noiseY = smoothNoise(this.pos.x * 0.006, this.pos.y * 0.006, time + 100);
    this.vel.add(createVector(noiseX * 0.15, noiseY * 0.15));
    this.vel.limit(1.5);
    break;
}

This code efficiently handles the unique behavior of each element’s particles while remaining clean and maintainable. The fire particles rise and flicker naturally, while air particles follow smooth, noise-based patterns that create convincing wind-like movements.

Challenges

Sketch: p5.js Web Editor | black hole

Concept

for (let i = 20; i > 0; i--) {
    let radius = this.radius * (i * 0.7);
    let alpha = map(i, 0, 20, 100, 0);
    
    for (let angle = 0; angle < TWO_PI; angle += 0.05) {
        let time = frameCount * 0.02;
        let xOff = cos(angle + time) * radius;
        let yOff = sin(angle + time) * radius;
        
        let distortion1 = noise(xOff * 0.01, yOff * 0.01, time) * 20;
        let distortion2 = noise(xOff * 0.02, yOff * 0.02, time + 1000) * 15;
        let finalDistortion = distortion1 + distortion2;
        
        let spiralFactor = (sin(angle * 3 + time) * cos(angle * 2 + time * 0.5)) * radius * 0.1;

Reflection and Future Considerations

Listening to lecture about MUJO, I was quote moved by how this multimedia performance piece explores the concept of impermanence through multiple artistic dimensions. The work masterfully integrates dance, projection mapping, and sound in the desert landscape to create a profound meditation on the lasting nature of existence.
The decision to use desert dunes as both stage and canvas is particularly fascinating. The natural formation and erosion of sand dunes serves as a perfect metaphor for the piece’s central theme of impermanence, mirroring the way human experiences and emotions constantly shift and transform. The digital projections that create abstract dunes over real ones cleverly amplify this concept, creating a dialogue between the natural and the digital.
What makes MUJO especially compelling is its dual existence as both a live desert performance and a multi-channel installation. The installation version demonstrates how site-specific art can be thoughtfully adapted for different contexts while maintaining its core message. The multi-channel approach in the installation allows for a more fragmented and intimate exploration of the body’s relationship with elemental forces.
The collaboration between choreographer Kiori Kawai and multimedia artist Aaron Sherwood shows significant effort. The dancers’ movements, as they climb and descend the dunes, physically embody the struggle with constant change, while the immersive soundscape and visuals reinforce this theme. The technical aspects – from projection mapping to sound design – don’t merely serve as technicalities but actively participate in the narrative.
The work draws fascinating parallels between the impermanence of natural phenomena and human existence. Just as sand particles come together to form dunes only to be reshaped by wind, the piece suggests our bodies and thoughts are similarly temporary mediums. This Buddhist-influenced perspective on impermanence is expressed not just conceptually but through every artistic choice in the performance.
Additionally, having an opportunity to ask questions from their direct experience was very helpful as we were able to see not only the steps taken by them but what kind of hindrances they were challenged with throughout. Overcoming those obstacles, whether they are technological limitations or artistic was very interesting to learn and hear about.

(https://www.aaron-sherwood.com/works/mujo/)

Sketch: (won’t work without my server, explained later in code)

Timelapse:

SVG Print:

Digital Prints:

(This one is same as SVG version without edge detecting algorithm and simplification)

Concept Inspiration

As a technology enthusiast with a keen interest in machine learning, I’ve been fascinated by the recent advancements in generative AI, particularly in the realm of image generation. While I don’t have the expertise nor timeframe to create a generative AI model from scratch, I saw an exciting opportunity to explore the possibilities of generative art by incorporating existing AI image generation tools.

My goal was to create a smooth, integrated experience that combines the power of AI-generated images with classic artistic styles. The idea of applying different painter theme to AI-generated images came to mind as a way to blend cutting-edge technology with traditional art forms. For my initial experiment, I chose to focus on the distinctive style of Vincent van Gogh, known for his bold colors and expressive brushstrokes.

Development Process

The development process consisted of two main components:

1. Backend Development: A Node.js server using Express was created to handle communication with the AI API. This server receives requests from the frontend, interacts with the API to generate images, and serves these images back to the client.
2. Frontend Development: The user interface and image processing were implemented using p5.js. This includes the input form for text prompts, display of generated images, application of the Van Gogh effect, and SVG extraction based on edge detection algorithm.

Initially, I attempted to implement everything in p5.js, but API security constraints necessitated the creation of a separate backend.

Implementation Details

The application works as follows:

1. The user enters a text prompt in the web interface.
2. The frontend sends a request to the Node.js server.
3. The server communicates with the StarryAI API to generate an image.
4. The generated image is saved on the server and its path is sent back to the frontend.
5. The frontend displays the generated image.
6. The user can apply the Van Gogh effect, which uses a custom algorithm to create a painterly style.
7. User is able to export the image in PNG format with or without Van Gogh effect
8. User is also able to export two different kinds of SVG (simplified and even more simplified)
9. Version of SVG extraction for Pen Plotting is done through edge detection algorithm of which the user is able to calibrate sensitivity.

A key component of the project is the Van Gogh effect algorithm:

This function applies a custom effect that mimics Van Gogh’s style using Poisson disc sampling and a swirling line algorithm. Here is significant code:

// Class for Poisson disc sampling
class PoissonDiscSampler {
  constructor() {
    this.r = model.pointr;
    this.k = 50;  // Number of attempts to find a valid sample before rejecting
    this.grid = [];
    this.w = this.r / Math.sqrt(2);  // Cell size for spatial subdivision
    this.active = [];  // List of active samples
    this.ordered = [];  // List of all samples in order of creation
    
    // Use image dimensions instead of canvas dimensions
    this.cols = floor(generatedImage.width / this.w);
    this.rows = floor(generatedImage.height / this.w);
    
    // Initialize grid
    for (let i = 0; i < this.cols * this.rows; i++) {
      this.grid[i] = undefined;
    }
    
    // Add the first sample point (center of the image)
    let x = generatedImage.width / 2;
    let y = generatedImage.height / 2;
    let i = floor(x / this.w);
    let j = floor(y / this.w);
    let pos = createVector(x, y);
    this.grid[i + j * this.cols] = pos;
    this.active.push(pos);
    this.ordered.push(pos);
    
    // Generate samples
    while (this.ordered.length < model.pointcount && this.active.length > 0) {
      let randIndex = floor(random(this.active.length));
      pos = this.active[randIndex];
      let found = false;
      for (let n = 0; n < this.k; n++) {
        // Generate a random sample point
        let sample = p5.Vector.random2D();
        let m = random(this.r, 2 * this.r);
        sample.setMag(m);
        sample.add(pos);
        
        let col = floor(sample.x / this.w);
        let row = floor(sample.y / this.w);
        
        // Check if the sample is within the image boundaries
        if (col > -1 && row > -1 && col < this.cols && row < this.rows && 
            sample.x >= 0 && sample.x < generatedImage.width && 
            sample.y >= 0 && sample.y < generatedImage.height && 
            !this.grid[col + row * this.cols]) {
          let ok = true;
          // Check neighboring cells for proximity
          for (let i = -1; i <= 1; i++) {
            for (let j = -1; j <= 1; j++) {
              let index = (col + i) + (row + j) * this.cols;
              let neighbor = this.grid[index];
              if (neighbor) {
                let d = p5.Vector.dist(sample, neighbor);
                if (d < this.r) {
                  ok = false;
                  break;
                }
              }
            }
            if (!ok) break;
          }
          if (ok) {
            found = true;
            this.grid[col + row * this.cols] = sample;
            this.active.push(sample);
            this.ordered.push(sample);
            break;
          }
        }
      }
      if (!found) {
        this.active.splice(randIndex, 1);
      }
      
      // Stop if we've reached the desired point count
      if (this.ordered.length >= model.pointcount) {
        break;
      }
    }
  }
}

// LineMom class for managing line objects
class LineMom {
  constructor(pointcloud) {
    this.lineObjects = [];
    this.lineCount = pointcloud.length;
    this.randomZ = random(10000);
    for (let i = 0; i < pointcloud.length; i++) {
      if (pointcloud[i].x < -model.linelength || pointcloud[i].y < -model.linelength ||
          pointcloud[i].x > width + model.linelength || pointcloud[i].y > height + model.linelength) {
        continue;
      }
      this.lineObjects[i] = new LineObject(pointcloud[i], this.randomZ);
    }
  }
  
  render(canvas) {
    for (let i = 0; i < this.lineCount; i++) {
      if (this.lineObjects[i]) {
        this.lineObjects[i].render(canvas);
      }
    }
  }
}

Another key component of the project was SVG extraction based on edge detection.

1. The image is downscaled for faster processing.
2. Edge detection is performed on the image using a simple algorithm that compares the brightness of each pixel to the average brightness of its 3×3 neighborhood. If the difference is above a threshold, the pixel is considered an edge.
3. The algorithm traces paths along the edges by starting at an unvisited edge pixel and following the edges until no more unvisited edge pixels are found or the path becomes too long.
4. The traced paths are simplified using the Ramer-Douglas-Peucker algorithm, which removes points that don’t contribute significantly to the overall shape while preserving the most important points.
5. The simplified paths are converted into SVG path elements and combined into a complete SVG document.
6. The SVG is saved as a file that can be used for plotting or further editing.

This approach extracts the main outlines and features of the image as a simplified SVG representation.

// Function to export a simplified SVG based on edge detection
function exportSimpleSVG() {
  if (!generatedImage) {
    console.error('No image generated yet');
    return;
  }

  // Downscale the image for faster processing
  let scaleFactor = 0.5;
  let img = createImage(generatedImage.width * scaleFactor, generatedImage.height * scaleFactor);
  img.copy(generatedImage, 0, 0, generatedImage.width, generatedImage.height, 0, 0, img.width, img.height);

  // Detect edges in the image
  let edges = detectEdges(img);
  edges.loadPixels();

  let paths = [];
  let visited = new Array(img.width * img.height).fill(false);

  // Trace paths along the edges
  for (let x = 0; x < img.width; x++) {
    for (let y = 0; y < img.height; y++) {
      if (!visited[y * img.width + x] && brightness(edges.get(x, y)) > 0) {
        let path = tracePath(edges, x, y, visited);
        if (path.length > 5) { // Ignore very short paths
          paths.push(simplifyPath(path, 1)); // Simplify the path
        }
      }
    }
  }

// Function to detect edges in an image
function detectEdges(img) {
  img.loadPixels(); //load pixels of input image
  let edges = createImage(img.width, img.height); //new image for storing
  edges.loadPixels();

  // Simple edge detection algorithm
  for (let x = 1; x < img.width - 1; x++) { //for each pixel exlcuding broder
    for (let y = 1; y < img.height - 1; y++) {
      let sum = 0;
      for (let dx = -1; dx <= 1; dx++) {
        for (let dy = -1; dy <= 1; dy++) {
          let idx = 4 * ((y + dy) * img.width + (x + dx));
          sum += img.pixels[idx];
        }
      }
      let avg = sum / 9; //calculate avg brightness of 3x3 neighborhood
      let idx = 4 * (y * img.width + x);
      edges.pixels[idx] = edges.pixels[idx + 1] = edges.pixels[idx + 2] = 
        abs(img.pixels[idx] - avg) > 1 ? 255 : 0; //change this
      edges.pixels[idx + 3] = 255; //if difference between pixel brightness and average is above 3 its considered an edge. result is binary image where edges are white and none edges are black
    }
  }
  edges.updatePixels();
  return edges;
}

// Function to trace a path along edges
function tracePath(edges, startX, startY, visited) {
  let path = [];
  let x = startX;
  let y = startY;
  let direction = 0; // 0: right, 1: down, 2: left, 3: up

  while (true) {
    path.push({x, y});
    visited[y * edges.width + x] = true;

    let found = false;
    for (let i = 0; i < 4; i++) { //It continues tracing until it can't find an unvisited edge pixel 
      let newDirection = (direction + i) % 4;
      let [dx, dy] = [[1, 0], [0, 1], [-1, 0], [0, -1]][newDirection];
      let newX = x + dx;
      let newY = y + dy;

      if (newX >= 0 && newX < edges.width && newY >= 0 && newY < edges.height &&
          !visited[newY * edges.width + newX] && brightness(edges.get(newX, newY)) > 0) {
        x = newX;
        y = newY;
        direction = newDirection;
        found = true;
        break;
      }
    }

    if (!found || path.length > 500) break; // Stop if no unvisited neighbors or path is too long
  }

  return path;
}

//Function to simplify a path using the Ramer-Douglas-Peucker algorithm The key idea behind this algorithm is that it preserves the most important points of the path (those that deviate the most from a straight line) while removing points that don't contribute significantly to the overall shape.
function simplifyPath(path, tolerance) {
  if (path.length < 3) return path; //If the path has fewer than 3 points, it can't be simplified further, so we return it as is.

  function pointLineDistance(point, lineStart, lineEnd) { //This function calculates the perpendicular distance from a point to a line segment. It's used to determine how far a point is from the line formed by the start and end points of the current path segment.
    let dx = lineEnd.x - lineStart.x;
    let dy = lineEnd.y - lineStart.y;
    let u = ((point.x - lineStart.x) * dx + (point.y - lineStart.y) * dy) / (dx * dx + dy * dy);
    u = constrain(u, 0, 1);
    let x = lineStart.x + u * dx;
    let y = lineStart.y + u * dy;
    return dist(point.x, point.y, x, y);
  }

  //This loop iterates through all points (except the first and last) to find the point that's farthest from the line formed by the first and last points of the path.
  let maxDistance = 0;
  let index = 0; 
  for (let i = 1; i < path.length - 1; i++) {
    let distance = pointLineDistance(path[i], path[0], path[path.length - 1]);
    if (distance > maxDistance) {
      index = i;
      maxDistance = distance;
    }
  }

  if (maxDistance > tolerance) { //split and recursively simplify each
    let leftPath = simplifyPath(path.slice(0, index + 1), tolerance);
    let rightPath = simplifyPath(path.slice(index), tolerance);
    return leftPath.slice(0, -1).concat(rightPath);
  } else {
    return [path[0], path[path.length - 1]];
  }
}

Challenges

The main challenges encountered during this project were:

1. Implementing secure API communication: API security constraints led to the development of a separate backend, which added complexity to the project architecture.
2. Managing asynchronous operations in the image generation process: The AI image generation process is not instantaneous, which required implementing a waiting mechanism in the backend. (Promised base) Here’s how it works:
   • When the server receives a request to generate an image, it initiates the process with the StarryAI API.
   • The API responds with a creation ID, but the image isn’t ready immediately.
   • The server then enters a polling loop, repeatedly checking the status of the image generation process:

• • This loop continues until the image is ready or an error occurs.
  • Once the image is ready, it’s downloaded and saved on the server.
  • Finally, the image path is sent back to the frontend.
  • This process ensures that the frontend doesn’t hang while waiting for the image, but it also means managing potential timeout issues and providing appropriate feedback to the user.

3. Integrating the AI image generation with the Van Gogh effect seamlessly: Ensuring that the generated image could be smoothly processed by the Van Gogh effect algorithm required careful handling of image data.
4. Ensuring smooth user experience: Managing the state of the application across image generation and styling and providing appropriate feedback to the user during potentially long wait times, was crucial for a good user experience.
5. Developing an edge detection algorithm for pen plotting:
   • Adjusting the threshold value for edge detection was important, as it affects the level of detail captured in the resulting SVG file. Setting the threshold too low would result in an overly complex SVG, while setting it too high would oversimplify the image.
   • Ensuring that the custom edge detection algorithm produced satisfactory results across different input images was also a consideration, as images vary in contrast and detail. Initially, I had problem with edge pixels but later excluded them.
   • Integrating the edge detection algorithm seamlessly into the existing image processing pipeline and ensuring compatibility with the path simplification step (Ramer-Douglas-Peucker algorithm) was another challenge that required careful design and testing.
6. Image generation, I experimented with different image generation models provided by StarryAI. From default to fantasy to anime. Eventually I settled down for detailed Illustration model which is perfect for svg extraction as it provides more distinct lines based on cartoonish appearance and also works well for Van Gogh effect due to its bold colors and more simplified nature compared to more realistic images.

Reflection

This project provided valuable experience in several areas:

1. Working with external APIs and handling asynchronous operations
2. Working with full-stack approach with Node.js and p5.js
3. Integrating different technologies (AI image generation and artistic styling) into a cohesive application
4. Implementing algorithms for edge detection.

I am quite happy with the result and plotted image also works well stylistically although it is different from initial painter effect, it provides another physical dimension to the project which is just as important.

Future Improvements:

1. Implementing additional artistic styles
2. Refining the user interface for a better user experience
3. Combining art styles with edge detection for more customizable SVG extraction.
4. Hosting site online to keep project running without my interference. This would also require me to have some kind of subscription for Image generation API because current one is capped at around 100 requests for current model.

Sketch:  (can’t work without my server for now so just pasting for code and demonstration purposes)

Image based on prompt “Designing interactive media project about art generation” with Van Gogh effect.

Concept Inspiration

As a technology enthusiast with a keen interest in machine learning, I’ve been fascinated by the recent advancements in generative AI, particularly in the realm of image generation. While I don’t have the expertise to create a generative AI model from scratch, I saw an exciting opportunity to explore the possibilities of generative art by incorporating existing AI image generation tools.

My goal was to create a smooth, integrated experience that combines the power of AI-generated images with classic artistic styles. The idea of applying different painter themes to AI-generated images came to mind as a way to blend cutting-edge technology with traditional art forms. For my initial experiment, I chose to focus on the distinctive style of Vincent van Gogh, known for his bold colors and expressive brushstrokes.

Development Process

The development process consisted of two main components:

1. Backend Development: A Node.js server using Express was created to handle communication with the StarryAI API. This server receives requests from the frontend, interacts with the API to generate images, and serves these images back to the client.
2. Frontend Development: The user interface and image processing were implemented using p5.js. This includes the input form for text prompts, display of generated images, and application of the Van Gogh effect.

Initially, I attempted to implement everything in p5.js, but API security constraints necessitated the creation of a separate backend.

Implementation Details

The application works as follows:

1. The user enters a text prompt in the web interface.
2. The frontend sends a request to the Node.js server.
3. The server communicates with the StarryAI API to generate an image.
4. The generated image is saved on the server and its path is sent back to the frontend.
5. The frontend displays the generated image.
6. The user can apply the Van Gogh effect, which uses a custom algorithm to create a painterly style.

A key component of the project is the Van Gogh effect algorithm:

// Function to apply the Van Gogh effect to the generated image
function applyVanGoghEffect() {
  if (!generatedImage) {
    statusText.html('Please generate an image first');
    return;
  }

  vanGoghEffect = true;
  statusText.html('Applying Van Gogh effect...');

  // Prepare the image for processing
  generatedImage.loadPixels();

  // Create Poisson disc sampler and line objects
  poisson = new PoissonDiscSampler();
  lines = new LineMom(poisson.ordered);

  // Set up canvas for drawing the effect
  background(model.backgroundbrightness);
  strokeWeight(model.linewidth);
  noFill();

  redraw();  // Force a redraw to apply the effect

  statusText.html('Van Gogh effect applied');
}

This function applies a custom effect that mimics Van Gogh’s style using Poisson disc sampling and a swirling line algorithm.

Challenges

The main challenges encountered during this project were:

1. Implementing secure API communication: API security constraints led to the development of a separate backend, which added complexity to the project architecture.
2. Managing asynchronous operations in the image generation process: The AI image generation process is not instantaneous, which required implementing a waiting mechanism in the backend. Here’s how it works:
   • When the server receives a request to generate an image, it initiates the process with the StarryAI API.
   • The API responds with a creation ID, but the image isn’t ready immediately.
   • The server then enters a polling loop, repeatedly checking the status of the image generation process:

1. • This loop continues until the image is ready or an error occurs.
   • Once the image is ready, it’s downloaded and saved on the server.
   • Finally, the image path is sent back to the frontend.
   • This process ensures that the frontend doesn’t hang while waiting for the image, but it also means managing potential timeout issues and providing appropriate feedback to the user.

3. Integrating the AI image generation with the Van Gogh effect seamlessly: Ensuring that the generated image could be smoothly processed by the Van Gogh effect algorithm required careful handling of image data.

4. Ensuring smooth user experience: Managing the state of the application across image generation and styling and providing appropriate feedback to the user during potentially long wait times, was crucial for a good user experience.

Reflection

This project provided valuable experience in several areas:

1. Working with external APIs and handling asynchronous operations
2. Developing full-stack applications with Node.js and p5.js
3. Integrating different technologies (AI image generation and artistic styling) into a cohesive application

While the result is a functional prototype rather than a polished product, it successfully demonstrates the concept of combining AI-generated images with artistic post-processing.

Future Improvements

Potential areas for future development include:

1. Implementing additional artistic styles
2. Refining the user interface for a better user experience
3. Adding functionality to save generated artworks
4. Optimizing the integration between image generation and styling for better performance
5. Allowing user customization of effect parameters

These improvements could enhance the application’s functionality and user engagement.

References:

Coding Train

p5.js Web Editor | curl flowfield lineobjects image (p5js.org)

Sketch: 

Example drawing

Concept Inspiration

My project was created with a focus on intersection of art and mathematics. I was particularly intrigued by the concept of Fourier transforms and their ability to break down complex patterns into simpler components. After seeing various implementations of Fourier drawings online, I was inspired to create my own version with a unique twist. I wanted to not only recreate drawings using Fourier series but also add an interactive coloring feature that would make the final result more visually appealing and engaging for users.

Process of Development

I began by following the Coding Train tutorial on Fourier transforms to implement the basic drawing and reconstruction functionality. This gave me a solid foundation to build upon. Once I had the core Fourier drawing working, I shifted my focus to developing the coloring aspect, which became my main contribution to the project.

The development process was iterative. I started with a simple algorithm to detect different sections of the drawing and then refined it over time. I experimented with various thresholds for determining when one section ends and another begins and worked on methods to close gaps between sections that should be connected. Even now, it is far from perfect but it does what I initially intended to.

How It Works

The application works in several stages:

1. User Input: Users draw on a canvas using their mouse or touchscreen.
2. Fourier Transform: The drawing is converted into a series of complex numbers and then transformed into the frequency domain using the Discrete Fourier Transform (DFT) algorithm. This part is largely based on the Coding Train tutorial.
3. Drawing Reconstruction: The Fourier coefficients are used to recreate the drawing using a series of rotating circles (epicycles). The sum of all these rotations traces out a path that approximates the original drawing.
4. Section Detection: My algorithm analyzes the original drawing to identify distinct sections based on the user’s drawing motion.
5. Coloring: Each detected section is assigned a random color.
6. Visualization: The reconstructed drawing is displayed, with each section filled in with its assigned color.
7. Re: User is able to start the process again and creature unique coloring look.
8. Save: User is able to save the image to their local machine.

Code I’m Proud Of

While the Fourier transform implementation was based on the tutorial, I’m particularly proud of the section detection and coloring algorithm I developed:

function detectSections(points) {
  let sections = [];
  let currentSection = [];
  let lastPoint = null;
  const distanceThreshold = 20;

  // Iterate over each point in the drawing
  for (let point of points) {
    if (lastPoint && dist(point.x, point.y, lastPoint.x, lastPoint.y) > distanceThreshold) {
      // If the distance between the current point and the last point exceeds the threshold,
      // consider it a new section and push the current section to the sections array
      if (currentSection.length > 0) {
        sections.push(currentSection);
        currentSection = [];
      }
    }
    // Add the current point to the current section
    currentSection.push(point);
    lastPoint = point;
  }

  // Push the last section to the sections array
  if (currentSection.length > 0) {
    sections.push(currentSection);
  }

  // Close gaps between sections by merging nearby sections
  return closeGapsBetweenSections(sections, distanceThreshold * 2);
}

function closeGapsBetweenSections(sections, maxGapSize) {
  let mergedSections = [];
  let currentMergedSection = sections[0];

  // Iterate over each section starting from the second section
  for (let i = 1; i < sections.length; i++) {
    let lastPoint = currentMergedSection[currentMergedSection.length - 1];
    let firstPointNextSection = sections[i][0];

    if (dist(lastPoint.x, lastPoint.y, firstPointNextSection.x, firstPointNextSection.y) <= maxGapSize) {
      // If the distance between the last point of the current merged section and the first point of the next section
      // is within the maxGapSize, merge the next section into the current merged section
      currentMergedSection = currentMergedSection.concat(sections[i]);
    } else {
      // If the distance exceeds the maxGapSize, push the current merged section to the mergedSections array
      // and start a new merged section with the next section
      mergedSections.push(currentMergedSection);
      currentMergedSection = sections[i];
    }
  }

  // Push the last merged section to the mergedSections array
  mergedSections.push(currentMergedSection);
  return mergedSections;
}

This algorithm detects separate sections in the drawing based on the distance between points, allowing for intuitive color separation. It also includes a method to close gaps between sections that are likely part of the same continuous line, which helps create more coherent colored areas.

Challenges

The main challenge I faced was implementing the coloring feature effectively. Determining where one section of the drawing ends and another begins was not straightforward, especially for complex drawings with overlapping lines or varying drawing speeds. I had to experiment with different distance thresholds to strike a balance between oversegmentation (too many small colored sections) and undersegmentation (not enough color variation).

Another challenge was ensuring that the coloring didn’t interfere with the Fourier reconstruction process. I needed to make sure that the section detection and coloring were applied to the original drawing data in a way that could be mapped onto the reconstructed Fourier drawing.

Reflection

This project was a valuable learning experience. It helped me understand how to apply mathematical concepts like Fourier transforms to create something visually interesting and interactive. While the core Fourier transform implementation was based on the tutorial, developing the coloring feature pushed me to think creatively about how to analyze and segment a drawing. Nevertheless, following tutorial also helped me comprehend mathematical side of the concept.

I gained insights into image processing techniques, particularly in terms of detecting continuity and breaks in line drawings. The project also improved my skills in working with canvas graphics and animation in JavaScript.

Moreover, this project taught me the importance of user experience in mathematical visualizations. Adding the coloring feature made the Fourier drawing process more engaging and accessible to users who might not be as interested in the underlying mathematics.

Future Improvements

Looking ahead, there are several ways I could enhance this project:

1. User-defined Colors: Allow users to choose their own colors for sections instead of using random colors.
2. Improved Section Detection: Implement more sophisticated algorithms for detecting drawing sections, possibly using machine learning techniques to better understand the user’s intent.
3. Smooth Color Transitions: Add an option for smooth color gradients between sections instead of solid colors.
4. Interactivity: Allow users to manipulate the colored sections after the drawing is complete, perhaps by dragging section boundaries or merging/splitting sections.
5. Improved interface: make interface look more modern and polished.

References

1. The Coding Train’s Fourier Transform tutorial by Daniel Shiffman
2. P5.js documentation and examples
3. Various online sources

Sketch

Concept Inspiration

My project, titled “Be Not Afraid,” was inspired by the concept of biblically accurate angels, specifically the Thrones (also known as Ophanim). In biblical and extrabiblical texts, Thrones are described as extraordinary celestial beings. The prophet Ezekiel describes them in Ezekiel 1:15-21 as wheel-like creatures: “Their appearance and structure was as it were a wheel in the middle of a wheel.” They are often depicted as fiery wheels covered with many eyes.

I wanted to recreate this awe-inspiring and somewhat unsettling image using digital art. The multiple rotating rings adorned with eyes in my project directly represent the wheel-within-wheel nature of Thrones, while the overall structure aims to capture their celestial and otherworldly essence. By creating this digital interpretation, I hoped to evoke the same sense of wonder and unease that the biblical descriptions might have inspired in ancient times.

Process of Development

I started by conceptualizing the basic structure – a series of rotating rings with eyes to represent the Thrones’ form. Initially, I implemented sliders for parameter adjustment, thinking it would be interesting to allow for interactive manipulation. However, as I developed the project, I realized I preferred a specific aesthetic that more closely aligned with the biblical descriptions and decided to remove the sliders and keep fixed values.

A key requirement of the project was to use invisible attractors and visible movers to create a pattern or design. This led me to implement a system of attractors that influence the movement of the entire Throne structure. This is mainly expressed in rotation around the center and more turbulent up and down movement. Values for these were adjusted to make motion smooth and graceful, corresponding to that of divine being.

As I progressed, I kept adding new elements to enhance the overall impact and atmosphere. The central eye came later in the process, as did the cloud background and sound elements. The project was all about refinement after refinement. Even at this stage I am sure there are lots of things to improve since lot of is visual representation which at times can be quite subjective.

How It Works

My project uses p5.js to create a 3D canvas with several interacting elements:

1. Rings: I created four torus shapes with different orientations and sizes to form the base structure, representing the “wheel within a wheel” form of Thrones. Those wheels or rings were taken to be different values but eventually settled for four as it is not too overcrowded while delivering needed effect.
2. Eyes: I positioned multiple eyes of varying sizes on these rings, reflecting the “full of eyes” description associated with Thrones.
3. Central Eye: I added a larger eye in the center that responds to mouse movement when the cursor is over the canvas, symbolizing the all-seeing nature of these beings.
4. Attractors and Movement: I implemented a system of invisible attractors that influence the movement of the entire structure. This includes:
5. A central attractor that creates a circular motion.
6. Vertical attractors that add turbulence and complexity to the movement. These attractors work together to create the organic, flowing motion of the Throne structure, evoking a sense of constant, ethereal rotation as described in biblical texts.
7. Background: I used a cloud texture to provide a heavenly backdrop.
8. Audio: I incorporated background music and a rotation sound whose volume correlates with the ring speeds to enhance the atmosphere.

Code I’m Proud Of

There are several pieces of code in this project that I’m particularly proud of, as they work together to create the complex, ethereal movement of the Thrones:

1. The attractor system:

// Calculate attractor position
let attractorX = cos(attractorAngle) * attractorRadius;
let attractorY = sin(attractorAngle) * attractorRadius;

// Calculate vertical attractor position with increased turbulence
let verticalAttractorY = 
  sin(verticalAttractorAngle1) * verticalAttractorAmplitude1 +
  sin(verticalAttractorAngle2) * verticalAttractorAmplitude2 +
  sin(verticalAttractorAngle3) * verticalAttractorAmplitude3;

// Move the entire scene based on the attractor position
translate(attractorX, attractorY + verticalAttractorY, 0);

This code creates complex, organic motion by combining a circular attractor with vertical attractors. It achieves a nuanced, lifelike movement that adds significant depth to the visual experience, simulating the constant, ethereal rotation associated with the biblical descriptions of Thrones.

2. The ring and eye movement, including fading effects:

// Update outer ring spin speed
outerRingTimer++;
if (outerRingTimer >= pauseDuration && !isOuterRingAccelerating) {
  isOuterRingAccelerating = true;
  outerRingTimer = 0;
  fadeOutStartTime = 0;
} else if (outerRingTimer >= accelerationDuration && isOuterRingAccelerating) {
  isOuterRingAccelerating = false;
  outerRingTimer = 0;
  fadeOutStartTime = frameCount;
}

if (isOuterRingAccelerating) {
  outerRingSpeed += ringAcceleration;
  rotationSoundVolume = min(rotationSoundVolume + 0.01, 1);
} else {
  outerRingSpeed = max(outerRingSpeed - ringAcceleration / 3, 0.01);
  
  if (frameCount - fadeOutStartTime < decelerationDuration - fadeOutDuration) {
    rotationSoundVolume = 1;
  } else {
    let fadeOutProgress = (frameCount - (fadeOutStartTime + decelerationDuration - fadeOutDuration)) / fadeOutDuration;
    rotationSoundVolume = max(1 - fadeOutProgress, 0);
  }
}

rotationSound.setVolume(rotationSoundVolume);

// Update ring spins
rings[1].spin += outerRingSpeed;
rings[3].spin += innerRingSpeed;

// Draw and update eyes
for (let eye of eyes) {
  let ring = rings[eye.ring];
  let r = ring.radius + ring.tubeRadius * eye.offset;
  let x = r * cos(eye.angle);
  let y = r * sin(eye.angle);
  
  push();
  rotateX(ring.rotation.x + sin(angle + ring.phase) * 0.1);
  rotateY(ring.rotation.y + cos(angle * 1.3 + ring.phase) * 0.1);
  rotateZ(ring.rotation.z + sin(angle * 0.7 + ring.phase) * 0.1);
  if (eye.ring === 1 || eye.ring === 3) {
    rotateZ(ring.spin);
  }
  translate(x, y, 0);
  
  let eyePos = createVector(x, y, 0);
  let screenCenter = createVector(0, 0, -1);
  let directionVector = p5.Vector.sub(screenCenter, eyePos).normalize();
  
  let rotationAxis = createVector(-directionVector.y, directionVector.x, 0).normalize();
  let rotationAngle = acos(directionVector.z);
  
  rotate(rotationAngle, rotationAxis);
  
  if (eye.isInner) {
    rotateY(PI);
  }
  
  texture(eyeTexture);
  sphere(eye.size);
  pop();
}

This code manages the complex movement of the rings and eyes, including acceleration, deceleration, and fading effects. It creates a mesmerizing visual that captures the otherworldly nature of the Thrones. The fading of the rotation sound adds an extra layer of immersion.

I’m particularly proud of how these pieces of code work together to create a cohesive, organic motion that feels both alien and somehow alive, which is exactly what I was aiming for in this representation of biblically accurate angels.

Challenges

The biggest challenge I faced was definitely the movement and implementing the attractor system effectively. Creating smooth, organic motion in a 3D space while managing multiple rotating elements was incredibly complex. I struggled with:

1. Coordinating the rotation of rings with the positioning and rotation of eyes.
2. Implementing the acceleration and deceleration of ring rotations smoothly.
3. Balancing the various movement elements (ring rotation, attractor motion, eye tracking) to create a cohesive, not chaotic, visual effect.

Another significant challenge was accurately representing the complex, wheel-within-wheel structure of Thrones. Balancing the need for a faithful representation with artistic interpretation and technical limitations required careful consideration and multiple iterations.

Reflection

Looking back, I’m satisfied with how my “Be Not Afraid” project turned out. I feel I’ve successfully created an interesting  and slightly unsettling visual experience that captures the essence of Thrones as described in biblical texts. The layered motion effects created by the attractor system effectively evoke the constant rotation associated with these beings. I’m particularly pleased with how the central eye and the eyes on the rings work together to create a sense of an all-seeing, celestial entity.

Future Improvements

While I’m happy with the current state of my project, there are several improvements I’d like to make in the future:

1. Blinking: I want to implement a sophisticated blinking mechanism for the eyes, possibly with randomized patterns or reactive blinking based on scene events. This could add to the lifelike quality of the Throne.
2. Face Tracking: It would be exciting to replace mouse tracking with face tracking using a webcam and computer vision libraries. This would allow the central eye to follow the viewer’s face, making the experience even more immersive and unsettling.
3. Increased Realism: I’d like to further refine the eye textures and shading to create more photorealistic eyes, potentially using advanced shaders. This could enhance the “full of eyes” aspect described in biblical texts.
4. Interactive Audio: Developing a more complex audio system that reacts to the movement and states of various elements in the scene is definitely on my to-do list.
5. Performance Optimization: I want to explore ways to optimize rendering and calculation to allow for even more complex scenes or smoother performance on lower-end devices.
6. Enhanced Wheel Structure: While the current ring structure represents the wheel-like form of Thrones, I’d like to further refine this to more clearly show the “wheel within a wheel” aspect. This could involve creating interlocking ring structures or implementing a more complex geometry system.
7. Fiery Effects: Many descriptions of Thrones mention their fiery nature. Adding particle effects or shader-based fire simulations could enhance this aspect of their appearance.

References

1. Biblical descriptions of Thrones/Ophanim, particularly from the Book of Ezekiel
2. Provided Coding Train video about attractors
3. Various Art depicting thrones
4. General internet
5. Royalty free music
6. Eye texture PNG (Eye (Texture) (filterforge.com))
7. https://www.geeksforgeeks.org/materials-in-webgl-and-p5-js/

Update: Added eye movement, removed torus shape, increased eye frequency

Update2: removed outer frame, increased distance to Ophanim, fog effect, 2x zoom effect, modified picture (Photoshop Generative AI). Added more extensive comments. Eye twitch movement (random).

Sketch:

Concept: My project is an interactive digital artwork that simulates the movement and appearance of algae in a swamp environment. Inspired by what I have seen in my home country many times, it aims to capture the flow of algae in a river. I used different methodologies to create a dynamic, visually interesting scene that mimics the organic, flowing nature of algae. By incorporating various elements such as multiple algae clusters, water particles, and background rocks, I tried to recreate a cohesive river like ecosystem.

Inspiration: The inspiration for this project came from my trip to Dashbashi Mountain in Georgia. I saw algae flowing in a river near the waterfall, and it was very pretty, almost from a fantasy world. This happened very recently so it was the first thing that came to mind when I was thinking about this project. This brief encounter with nature became the foundation for my work, as I tried to translate the organic movement of algae and water into an interactive digital format.

IMG_7042 – Short clip of moving Algae

Process of Development: I developed this project iteratively, adding various features and complexities over time:

At first I visualized the algae movement. I realized it had to do something with waves and sinusoidal shapes were first thing that came to my mind. Unfortunately, few hours in implementation I realized assignment specifically asked for acceleration. Soon after implementing acceleration, I realized this movement alone limited me in creating algae flow so I had to go back and forth multiple times to get at result that somewhat resembled the movement while using acceleration. Unfortunately, I did not find any definitive tutorials of such movement. As such this is more of a strand like simulation which I slowly built up, line by line, also looking at other works like hair movement for inspiration, I will mention them in references.

These screenshots are from earlier development of the simulations:

As you can see by adding various parameters to each strand as well as overall cluster, we are able to achieve similar wavy pulsating pattern that algae have. I also added particle systems and noise-based algorithms for water movement (you can see inspiration of this from reference). To enhance the environment, I included rock formations and a sky.  I integrated sliders and toggles for user interaction. Finally, I kept refining values till I achieved desire perfomance and visuals. The simulation is pretty heavy to run and you can expect drastic FPS drops, based on number of strands we are running. Water simulation is a significant culprit here despite multiple scaling that was needed to achieve even that. 

How It Works:

Algae Simulation: I created multiple clusters of algae strands, each with unique properties. I animate each strand using sine waves and apply tapering effects and clustering forces for a more natural-looking movement. I also calculate acceleration and velocity for each strand to simulate fluid dynamics.

Water Effects: I used a particle system to create the illusion of flowing water, with Perlin noise for natural-looking movement. I applied color gradients to enhance the swamp-like appearance. There is also background audio of waterfall that helps the immersion.

Environmental Elements: I drew rocks using noise-based shapes with gradients and added a toggleable sky for depth.

Interactivity: I included multiple sliders that allow users to adjust various parameters in real-time.

If you want to know more about in depth working and how everything is related, it will be better to check out my code as it is commented thoroughly.

Code: 

One piece of code I’m particularly proud of is the function that generates and animates individual algae strands:

function algae(strandPhase, strandLength, strandAmplitude, clusterEndX, clusterPulsePhase) {
  beginShape();
  
  let taperingPoint = taperingPointSlider.value() * strandLength;
  let taperingStrength = taperingStrengthSlider.value();
  
  for (let i = 0; i <= strandLength; i += 10) {
    let x = i;
    let y = 0;
    
    let progress = i / strandLength;
    
    let taperingFactor = 1;
    if (i > taperingPoint) {
      taperingFactor = pow(map(i, taperingPoint, strandLength, 1, 0), taperingStrength);
    }
    
    let currentAmplitude = strandAmplitude * (1 - progress * 0.8) * taperingFactor;
    
    let movementFactor = sin(map(i, 0, strandLength, 0, PI));
    let movement = sin(strandPhase + i * 0.02) * currentAmplitude * movementFactor;
    
    let angle = map(i, 0, strandLength, 0, PI * 2);
    x += cos(angle) * movement;
    y += sin(angle) * movement;
    
    let curvature = sin(i * 0.05 + phase + clusterPulsePhase) * 5 * (1 - progress * 0.8) * taperingFactor;
    y += curvature;
    
    let clusteringForce = map(i, 0, strandLength, 0, 1);
    let increasedClusteringFactor = clusteringFactor + (progress * 0.5);
    x += (clusterEndX - x) * clusteringForce * increasedClusteringFactor;
    
    vertex(x, y);
  }
  endShape();
}

This code incorporates acceleration and velocity calculations to simulate realistic fluid dynamics, creating more natural and unpredictable movement. The function also creates a tapering effect along the strand’s length, generates wave-like movement using sine functions, and applies a clustering force to mimic how algae clumps in water. I’m especially pleased with how it combines mathematical concepts like acceleration, sine waves, and mapping with artistic principles to create a visually appealing and believable representation of algae in motion. The integration of user controls allows for real-time adjustment of parameters like acceleration strength, making the simulation highly interactive and customizable.

Challenges: 

Balancing visual quality with smooth performance was tricky, especially when animating multiple elements at once. Getting the algae to move naturally in response to water currents took a lot of tweaking. The water particle system was also tough to optimize – I had to find ways to make it look good without slowing everything down. Another challenge was making the user controls useful but not overwhelming.

Reflection:

This project was a good learning experience for me. I enjoyed turning what I saw at Dashbashi Mountain into a digital artwork. It was challenging at times, especially when I had to figure out how to make the algae move realistically. I’m happy with how I combined math and art to create something that looks pretty close to real algae movement. The project helped me improve my coding skills and while it’s not perfect, I’m pleased with how finished product looks.

Future Improvements:

Speed it up: The simulation can be slow, especially with lots of algae strands. I’d try to make it run smoother.
Better water: The water effect is okay, but it could look more realistic.
Add more stuff: Maybe include some fish or bugs to make it feel more like a real ecosystem.

References:

p5.js Web Editor | Blade seaweed copy copy (p5js.org)

p5.js Web Editor | sine wave movement (hair practice) (p5js.org)

p5.js Web Editor | Water Effect (p5js.org)

YouTube

Internet