Author: hk3905

Project Overview

This project explores body-based interaction through computer vision, where the user’s hands become controllers for two physically simulated characters. Using hand tracking, each finger is mapped to different limbs of the stickmen, allowing users to control movement in real time. The game is made out of 3 mini games that users can enjoy with their friends. The minigames include: basketball, football, and fencing. The goal of the game was to put the users in “shoes” or in this case maybe in hands of the puppet masters, controlling the stickmen and competing against each other in various sports.

Some inspiration was taken from other physics based video games such as Human Fall Flat. The key concept of that video game and the one I made is the physics engine. Human Fall Flat is heavily reliant on physics engine for the control of the characters which makes the game hard, but intentionally so. Similar to that my game is also meant to challenge players in a way. The movement is intentionally hard to master but I feel like that is what makes the game fun and what brings out the competitiveness in the players. But I also believe there is a thin line between fun challenging game and the game that is just broken and unplayable because of its difficulty. So through testing and continues optimization the goal was to create something that is challenging  enough to intrigue, but not challenging enough to scare players, but I will talk more about this later.

The core concept is to transform the human body into a direct input system, removing traditional controllers and instead using gesture-based interaction. The project focuses on making digital movement feel physical by combining ml5.js hand tracking with Matter.js physics simulation.

Process

The development of this project began as an exploration of how hand tracking could function as a primary input method for controlling a digital system. Rather than relying on traditional interfaces such as keyboards or controllers, the goal was to create an experience where the user’s own body, specifically their hands, became the mechanism through which interaction occurs. So at the very beginning of the project the goal was first to get hand tracking to work and test it out to see how difficult it is to implement and how it behaves.

(I believe WordPress doesn’t have camera access and thus the sketch appears blank, to view please open the sketch in another tab and allow camera usage)

I spent time observing how accurately the model could track fingers, how stable the tracking was under different lighting conditions, and how many hands could be detected simultaneously. These early tests revealed that while the tracking was generally responsive, it could fluctuate slightly depending on hand orientation and speed. This meant that any interaction system built on top of it would need to account for noise and instability.

Once I had gotten hand tracking set up and I was confident in it it was time for the stickman and the floor. To build the stickman, I used Matter.js to create a set of rigid bodies representing the head, torso, arms, and legs. These parts were connected using constraints in order to simulate joints. At this stage, the system was entirely physics-driven. The intention was to create a realistic representation of a body, where movement would emerge naturally from forces and connections rather than direct manipulation. Once the structure was in place, I attempted to control the limbs by moving the constraint anchor points based on the position of the user’s fingers.

This approach, while conceptually appealing, introduced significant problems in practice. The limbs became unstable, often oscillating or reacting unpredictably to small changes in input. Because the physics engine was constantly resolving forces, even minor inaccuracies in hand tracking resulted in exaggerated or chaotic movement. The limbs would move to their own will and there was almost nothing I could try to control them. My debugging attempt consisted of commenting out all fingers except for the thumb that controls the right hand and the middle finger that controlled the body and trying go get just one arm to behave normally. I quickly discovered that no matter what I did I couldn’t get the hand to work properly, the matter.js engine although powerful did have issue when multiple bodies were too close to one another which I also noticed when working on Assignment 10.

Thus a decision was made. It was time to go back to the drawing board and redo the whole design.

I decided to recreate the world but this time I only added the floor, gravity and one stick to the ground. The goal was to control its y axis with my thumb only and thus test out my new method which was mapping the movement instead of using constraints. The proved to be much more reliable than the old method which was more realistic. But at this point in development I decided that I will sacrifice some realism for better control and more enjoyable experience.

After successfully testing out the new movement on one stick I added another vertical one for torso and another one to represent the other arm. The new movement worked great with the rotation of the arms and the movement of the body and once I was happy with everything it  was time to transfer it to the original stickman. Each finger was mapped to a specific body part, and its vertical position was translated into a target angle using the map() function. These angles were then applied to the corresponding bodies using Matter.Body.setAngle(). To ensure smooth transitions, I used linear interpolation (lerp()), which allowed the limbs to gradually move toward their target positions rather than snapping instantly. Additionally, angular velocity was reset on each frame to prevent the physics engine from introducing unwanted rotational forces.

I also experimented with adding more complexity to the system, including additional joints such as elbows, in an effort to increase realism. However, this made the gameplay much much harder and more complicated. Controlling multiple joints with limited and noisy input proved to be extremely difficult, and the system became confusing rather than intuitive. So this was another idea that unfortunately had to be scrapped.

Once the interaction system was functioning reliably, I began developing different modes to explore how the characters could interact within a physics environment. The first mode implemented was basketball, where a ball interacts with the stickmen’s limbs and can be directed into a hoop. This required implementing collision detection and translating limb velocity into force applied to the ball. An immediate issue I  faced was the ball slowing down and getting stuck on the ball if it is not bounced and since there was no easy way to have the stickman lift the ball I made the ball bounce by itself if it was slowing down.

function updateBasketball() {
  // if ball is near ground and moving slow bounce it
  if (ball.position.y > height - 60 && Math.abs(ball.velocity.y) < 6) {
    Matter.Body.setVelocity(ball, {
      x: ball.velocity.x + random(-0.5, 0.5),
      y: -12,
    });
  }
}

 The football mode followed a similar structure but introduced goals on either side of the screen and required directional control of the ball. The fencing mode was more focused on direct interaction between the players, involving swords attached to the arms and collision detection to determine scoring. At this stage of development the fencing code is looking kind of empty because I hadn’t added multiplayer yet and there wasn’t anything to do with the sword with one player realistically.

During the implementation of these modes, collision handling became a recurring challenge. Matter.js generates multiple collision events for a single contact, which caused scoring to trigger multiple times unintentionally. The final solution involved introducing cooldown timers and state flags, ensuring that each scoring event could only occur once within a short time window. This approach stabilized the scoring system across all modes.

With everything in place, I expanded the project to support two players.  I sorted the detected hands based on their horizontal position on the screen. The leftmost hand was assigned to one stickman, and the rightmost hand to the other. This ensured that each player consistently controlled the correct character. However, an issue arose when using left hands instead of right hands. The mapping logic, which had been designed for right-hand input, caused controls to appear mirrored and unplayable when a left hand was used.

To resolve this, I incorporated the handedness property provided by ml5 and adjusted the mapping accordingly. By reversing the control logic for left hands, I was able to maintain consistent and intuitive behavior regardless of which hand was used.

I also focused on improving the visual design of the project. Initially, the system was functional but visually minimal. I introduced distinct environments for each mode, including a basketball court, a football field, and a fencing strip, to provide context and variation. Additionally, I implemented a lighting system consisting of animated spotlight cones originating from the top of the screen. These lights are slightly angled and move subtly over time, adding depth and atmosphere without interfering with gameplay. I also tested implementing crowd stands and a crowd in the background around where the hands should be positioned, but I found this to be too distracting for the players and decided to continue without it.

In talks with the professor we concluded that there should be a way to show the users that they are supposed to use their hands to control the stickmen and to begin they should put their palms in front of the screen. To address this I added visual guides in the form of hand outline images. These guides appear when no hands are detected. I also implemented a pulsing scale animation, which made the guides feel more alive and noticeable without being distracting. This, I believe, significantly improved the clarity of the interaction.

Another thing I wanted to implement was a visual element that clearly demonstrates the presence of physics in the system since the body movement is more reliant on mapping instead of constraints and physics. I experimented with adding clothing to the characters using chains of connected bodies, but this resulted in shapes that resembled capes and did not integrate well with the overall design. The movement was also visually confusing. I then shifted to implementing a hair system, where small segments are connected to the head using constraints. This approach was more successful, as the hair responded naturally to movement and provided a subtle but clear indication of physics at work. It also allowed for differentiation between characters through variations in color.

All this lead to the finish product. I liked writing this progress report as it gave me a chance to go back and explore all the different mechanics and how they were developed. It was honestly so interesting to see how the code changes depending on the “generation” of the project and how the playability and the overall feel of the game improves over time. I hope you enjoyed reading the process reflection as much as I did writing it :).

Video

Reflection and Future Improvements

Honestly I really enjoyed working on this project. Even though it was a lot of hard work and a lot more debugging and just constantly thinking about features, how to implement them and how to fix bugs I really enjoyed the whole process.

I learned a lot not only about programing in JavaScript and using ml5.js and matter.js, I learned so much about project development. Not being able to use constraints for movement and having to go with mapping taught me how not everything will always go with the plan but it is important to be able to adapt. If I could go back in time I would definitely write comments in the code more as I am writing it. I have this really bad habit of just being in the “zone” while coding and not writing any comments but I always end up going back and commenting everything after I am done with a certain feature or bug fix. Although this works with smaller projects, bigger projects like this taught me that it is important I change this habit.

In the future I would love to continue working on this project. Adding sound would be, in my opinion, the next logical step and feature to add. I would also love to add more different modes and maybe explore a different way to add clothing or some more design to the characters. But overall I am really happy with how the final project turned out.

Concept

For my final project, I want to create an interactive visual system in which particles move in a collective way and can be influenced through the user’s hand gestures captured by a webcam. The particles will behave like a living swarm using the flocking mechanism we have learned in class. Rather than interacting through a mouse or keyboard alone, the user will use their literal hands to push, scatter, gather, and possibly hold the particles in place. Through these gestures, the user will be able to create constantly changing visual compositions in real time.

The main interaction methodology will be webcam-based hand tracking. The user’s hands will become active forces inside the particle environment. Depending on the gesture or hand position, the particles may respond in different ways. A hand moving toward the swarm may push particles away as if creating a wave or gust. Slower or more stable hand positions may attract and gather particles into denser clusters. If both hands are used, they may create a temporary holding area between them, allowing the user to trap or guide a portion of the particles through the space. Faster gestures could create turbulence or scatter the particles more dramatically, while gentler movements could produce quieter, more controlled shifts. I may also include keyboard controls to switch between interaction modes if needed, but the webcam and hands will remain the main interface.

Hand Sketch

Interaction Design

• One hand: pushes, repels, or redirects nearby particles
• Two hands: gathers or traps particles between them
• Fast motion: creates turbulence and scattering
• Slow / steady motion: creates attraction or calm clustering

Initial p5 Sketch

Oasis After Dark

Concept

This project explores cellular automata through the creation of a small, interactive ecosystem that I call the Oasis After Dark. Instead of using traditional binary cellular automata, I wanted to experiment with a system where each cell represents a material with behavior. The sketch simulates four elements: sand, water, plant life, and fire, each governed by simple local rules.

The core idea behind the project is emergence. How complex, organic-looking patterns arise from simple interactions between neighboring cells. Sand falls and forms dunes, water flows and spreads, plants grow near water sources, and fire consumes organic matter. Individually, these behaviors are simple, but together they create a dynamic and evolving landscape.

Instead of generating art automatically I let the user be the artist and generate the oasis that they want.

Process

I began by building the grid structure and implementing the simplest material: sand. Each cell stores its type and updates based on local rules, which is the foundation of the cellular automaton.

The key behavior for sand is gravity: it falls downward if possible and slides diagonally if blocked. This was my first implementation of a local rule system.

if (canMoveInto(x, y + 1, [EMPTY])) {
  moveCell(x, y, x, y + 1, cell);
  return;
}

let dir = random() < 0.5 ? -1 : 1;

if (canMoveInto(x + dir, y + 1, [EMPTY])) {
  moveCell(x, y, x + dir, y + 1, cell);
  return;
}

Next, I added water, which introduced more complex movement. Unlike sand, water not only falls but also spreads sideways when blocked. This required adding multiple movement checks and randomness.

Water made the system feel more dynamic and introduced the first real interaction between materials. For example, sand can displace water, creating more complex patterns.

After movement-based elements, I introduced plants, which behave differently by growing instead of moving. Plants expand only when near water, creating a dependency between elements. But they also added another interaction. Fire can burn plants, and if the plants are connected and the fire touches one part, the fire will spread and burn down the whole “forest”.

And finally the sky looked a bit empty without anything on it so I decided to add some stars to make the sky fuller and the sketch look more alive.

function generateStars() {
  stars = [];
  
  for (let i = 0; i < 150; i++) {
    stars.push({
      x: random(width),
      y: random(height),
      size: random(1, 2.5),
      alpha: random(80, 180)
    });
  }
}

Code Highlight

One part of the code I am particularly proud of is the use of shuffled neighbor evaluation for plant growth and fire spreading:

function shuffledNeighbors(x, y) {
  let arr = [
    { x: x - 1, y: y - 1 },
    { x: x, y: y - 1 },
    { x: x + 1, y: y - 1 },
    { x: x - 1, y: y },
    { x: x + 1, y: y },
    { x: x - 1, y: y + 1 },
    { x: x, y: y + 1 },
    { x: x + 1, y: y + 1 },
  ];

  for (let i = arr.length - 1; i > 0; i--) {
    let j = floor(random(i + 1));
    [arr[i], arr[j]] = [arr[j], arr[i]];
  }
  return arr;
}

This function ensures that neighboring cells are processed in a random order rather than a fixed sequence. Without this, growth and spread would appear directional and artificial. By randomizing neighbor order I believe the project looks more natural. Plants grow in unpredictable directions, and fire spreads in a more natural, chaotic way.

Future Improvements

Overall I am really happy with how the project turned out. I think it is really similar to those sandbox games that let you create your own world. Choosing to let the users be the artist and create their own playground rather than creating generated art was I think the right direction for this project as it adds more interactivity and brings the user closer to the project than just tweaking some settings. As for the future improvements I would like to explore more materials and different interactions between them. Maybe adding smoke that emerges from “forest fires” and maybe letting some sand turn into glass when in too much contact with fire. Finally, this was a fun project to work, I learned a lot about cellular automata and its use and can’t wait to use it in my future projects!

Concept

For this assignment I was thinking of different ways I could implement the physics engine into some real world examples. One of the first things that came to mind was the pinball game. I remember when I was a kid and my dad bought me a small pinball machine I spent hours playing on it. Since the game is basically pushing a ball and letting it interact with paddles and bumpers it was perfect for this project and the use of Matter.js library.

The concept is similar to the normal pinball game, there is a ball on the far right and in the beginning you have to hit space to push it out of the starting position and then your goal is to control the paddles with the left and right arrows to score as many points as possible by hitting the bumpers until the ball inevitably hits the ground. Since I was working off of personal reference my game had 3 balls per round so I decided to add 3 lives to the game so the user can try to get the highest score.

Since the game elements take a lot of room of the canvas I had to make the canvas pretty big so I would recommend opening the sketch in the new window.

Process

I started the project by first creating an open canvas and just adding the bumpers and the spinner. The bumpers are the key component of the game so I believe the “world” should be created around them and not force them to fit into the “world”.

Once the spinner was working it was time to add the walls and the ball.

function updateSpinner() {
  Body.setAngle(spinner, spinner.angle + 0.04);
}

Making the spinner spin was quite simple and didn’t give me any issues. But then it was time to add the paddles and the collision with the ball.

For this I decided to use the Collision Events:

Events.on(engine, "collisionStart", function (event) {
  for (let pair of event.pairs) {
    let a = pair.bodyA;
    let b = pair.bodyB;

    // Ball + bumper
    if (
      (a.label === "ball" && b.label === "bumper") ||
      (a.label === "bumper" && b.label === "ball")
    ) {
      score += 10;
      collisionFlash = 10;
      shake = 10;

      let bumper = a.label === "bumper" ? a : b;
      bumper.hitTimer = 12;

      let currentBall = a.label === "ball" ? a : b;
      Body.applyForce(currentBall, currentBall.position, {
        x: random(-0.01, 0.01),
        y: -0.015,
      });
    }

After that was done it was time to move to the walls.

This was my original idea. In reference to what I remember my old pinball machine looking like I wanted to create a small opening where the ball would slide out of. But there was one huge issue with this that I overlooked at the time. If you notice 2 of the walls are overlapping which I didn’t think would be an issue but it proved to be a major one. The bug that kept happening is that because of the overlap the ball kept hitting an invisible wall and kept just bouncing back straight to the beginning. This is obviously a game breaking bug so it needed to be fixed. I tried lowering and raising the angled wall but nothing seemed to fix it because overlapping was the only way of keeping the ball from getting stuck between the 2 walls so I came up with an alternate position for the angled wall so it wouldn’t have to overlap.

By placing it in the top it didn’t overlap with anything and it also added some diversity to the game as the ball could bounce off of it and create different plays each game.

Another challenge that occurred was that the ball would move too fast and would thus phase through the walls and the paddles which would make the user lose points for doing nothing wrong. So I decided to limit the ball speed.

function limitBallSpeed() {
  let maxSpeed = 16;

  let vx = ball.velocity.x;
  let vy = ball.velocity.y;
  let speed = sqrt(vx * vx + vy * vy);

  if (speed > maxSpeed) {
    let scale = maxSpeed / speed;
    Body.setVelocity(ball, {
      x: vx * scale,
      y: vy * scale,
    });
  }
}

This fixed the issue and made the game much more enjotable.

After that it was time for some style changes and just adding the score and lives and game over screen which were quite simple and I also decided to add a screen shake feature when the bumpers are hit for some extra depth. And thus we have the final product.

Code Highlight

One of the most interesting parts of the project is how the paddles are implemented using constraints and controlled rotation, rather than simple position changes.

paddleLeftPivot = Constraint.create({
  bodyA: paddleLeft,
  pointB: { x: 270, y: 610 },
  pointA: { x: -50, y: 0 },
  stiffness: 1,
  length: 0,
});

paddleRightPivot = Constraint.create({
  bodyA: paddleRight,
  pointB: { x: 630, y: 610 },
  pointA: { x: 50, y: 0 },
  stiffness: 1,
  length: 0,
});

This code creates a pivot system, anchoring each paddle to a fixed point in the world. Instead of freely moving, the paddles rotate around these anchor points, similar to real pinball machines.

To control the paddles, I use angular velocity:

// Left paddle
if (keyCode === LEFT_ARROW) {
  Body.setAngularVelocity(paddleLeft, -0.35);
}

// Right paddle
if (keyCode === RIGHT_ARROW) {
  Body.setAngularVelocity(paddleRight, 0.35);
}

And then limit their rotation:

let leftAngle = constrain(paddleLeft.angle, -0.9, 0.25);
let rightAngle = constrain(paddleRight.angle, -0.25, 0.9);

Future Improvements

Overall I am really happy with how the project turned out. I have learned so much about the Matter.js physics system and I am eager to learn more and explore using it in my future projects. As for the improvements and future features I would really like to maybe add music and sound effects to the game as I believe that is a one of the core parts of the original arcade pinball machines. I would also like to maybe add some background images instead of just a gradient background. My pinball machine was green with some cartoon characters on it and many online I have seen have some cartoonish design so maybe exploring that would be the next step, but I don’t want to make it hard to focus on the game so it will need some testing.

Concept

After working on the flocking system for the assignment 8 I wanted to do something similar but also structurally different for this assignment. I wanted to make a collective behavior of the flock, or in this case, the school of fish which changes under extreme circumstances. The system begins in a calm state where fish move in cohesion in a fluid formation until the system is disturbed by a shark that comes to a random vertical point of the screen which makes the fish swim in different directions thus breaking the formation and the flocking system for a shirt period of time.

The project focuses on tension and release, where the calm flock represents order, the shark introduces disruption, and the recovery phase reflects reformation. Rather than presenting a literal narrative, the work aims to explore how collective systems respond to external pressure and how coherence can break and re-emerge over time.

Process

In the beginning my goal was to just add the fish and implement simple movement and flock behavior so I didn’t really focus too much on the look of the fish.

After the flocking behavior was working it was time to add the shark spawning and the fish avoiding it.

Again I was just testing things out so the shark looks more like an ellipse than a fish. But since everything was working correctly it was time to add the colors to the fish and some more details to the shark to get the final design.

Code Highlights

// different phase transition
if (phase === "calm" && phaseTimer > 300) {
  phase = "warning";
  phaseTimer = 0;
  shark.activate();
} else if (phase === "warning" && phaseTimer > 90) {
  phase = "panic";
  phaseTimer = 0;
} else if (phase === "panic" && shark.offscreen()) {
  phase = "recovery";
  phaseTimer = 0;
} else if (phase === "recovery" && phaseTimer > 300) {
  phase = "calm";
  phaseTimer = 0;
  shark.reset(); // prepare for next cycle
}

The structure converts the flocking algorithm into a time-dependent dance routine, in which different algorithms and parameters are used depending on the stage of the process. The system does not remain fixed with one behavioral pattern but moves through various stages without necessarily programming an event sequence.

I also played a bit with linear interpolation (lerp) to make the shift between the calm mode and the panic mode feel more natural rather than abrupt.

this.maxSpeed = lerp(this.maxSpeed, this.baseMaxSpeed, 0.03);
this.maxForce = lerp(this.maxForce, this.baseMaxForce, 0.03);

Future improvements

I am very happy with how the project turned out. I think it demonstrates well the tension and release part of the assignment and I think that choosing to give fish different colors was the right move artistically as it makes the assignment more visually appealing. If I was to add anything to the assignment that would definitely be sound. I believe that maybe different sound depending on the tension or if the shark is in sight would work really well for the overall project. But overall I am happy with the final result and am glad I got to work on flocking mechanism even more.

Concept

When thinking about “autonomous agents” in the nature one thing that came to mind were birds. Most of the time they move around by themselves, deciding on directions and the destination for us seemingly randomly, but when in a flock if they are following a leader suddenly they all come together to move in large groups following one. This is something I wanted to recreate in my project.

The user can click on the screen to spawn the leader bird, which will attract the rest of the flock. After the birds have gathered around it they will start orbiting around instead of all just piling up in one place. After which the user can click on another point on the screen and watch as the birds move towards it, each going their own direction and following the flow and the flock.

Process

I started by just having lines instead of birds as I just wanted to get the autonomous movement going:

After this was done it was time to add the birds. I decided to create them as pretty simple models, but I also wanted to add some flapping of the wings to make the visuals more appealing.

This was done with the following:

let flap = sin(frameCount * 0.2 + this.pos.x * 0.05) * 0.4;

I used the sin function to make the flapping smooth and natural, but I also made sure that the wings don’t move the the same position at the same times, instead when one moves up the other moves down and vice-versa.

push();
rotate(-flap);
ellipse(
  -this.size * 0.1,
  -this.size * 0.22,
  this.size * 0.9,
  this.size * 0.28
);
pop();

push();
rotate(flap);
ellipse(
  -this.size * 0.1,
  this.size * 0.22,
  this.size * 0.9,
  this.size * 0.28
);
pop();

Once I was happy with the look I decided to make the leader also a bird and allow the user to click to move it. I also decided to lower the alpha of the background to make the birds leave streaks behind so it gives it more of an art aesthetic.

Code Highlight

separate(vehicles) {
  let perceptionRadius = 25;
  let steering = createVector();
  let total = 0;

  for (let other of vehicles) {
    let d = dist(this.pos.x, this.pos.y, other.pos.x, other.pos.y);

    if (other !== this && d < perceptionRadius && d > 0) {
      let diff = p5.Vector.sub(this.pos, other.pos);
      diff.div(d * d);
      steering.add(diff);
      total++;
    }
  }

  if (total > 0) {
    steering.div(total);
    steering.setMag(this.maxSpeed);
    steering.sub(this.vel);
    steering.limit(this.maxForce * 1.2);
    this.applyForce(steering);
  }
}

One important action in this system is separation, where no two birds overlap and flock structure is maintained. Each bird looks at its local neighbors and applies a repulsive force that increases with proximity. The repulsive force increases significantly as distance decreases because it is divided by the square of the distance. This produces a natural spacing effect so that the flock is cohesive but not overly dense.

I would also like to highlight this part of the code:

fill(112, 231, 255, 40);
noStroke();
rect(0, 0, width, height);

Instead of clearing the screen and redrawing the background each frame I decided to dram a rectangle over the screen with a certain color and decrease its alpha. This gradually fades the previous frames instead of just clearing them instantly which makes the birds’ previous position to be visible for a short duration of time. This motion blur is something I wanted to create so we can better visualize  the movement of the birds and it also allows the user to create a canvas where they paint on by moving the leader and letting the birds paint as they move towards it.

Future improvements

I am really proud of how the project turned out in the end. If I was to add any other features that would probably be adding different colors for the birds and their trails as well as maybe playing with sound somehow as that is something I haven’t really worked on much before. Overall I am happy with the end result and have learned more about p5 while working on it.

The Digital Milky Way

Concept

When thinking about the midterm project I was trying to come up with a design that would be both enjoyable to build and beautiful for the viewers to experience. Something that has always fascinated me with both intrigue and beauty is the space, endless galaxies, start, planets… So my goal was to recrate some of that in p5 and try to make an artwork out of it.

I had also worked on space design for my previous assignments and found that those were the assignments I enjoyed working on the most and the ones that in my opinion turned out the most beautiful so the decision was obvious and quickly made.

Like many other projects of mine I didn’t want the user to be just a viewer. I wanted everyone to be able to experience the artwork by interacting with it. So besides adding the 3 modes that can be accessed with pressing numbers 1,2 and 3 on the keyboard I also implemented a click function which pushed away the particles creating interactive art.

The three modes

The project is divided into 3 different states all showing another beautiful part of space.

1. Planet with a ring

The first mode is inspired by Saturn which is known for its beautiful ring that circles it. It is also inspired by  my previous work where all my planets had rings. I think this brings depth to the planets and makes them more than just colored circles on the screen. For the ring I decided to integrate a particle system that we have been using in class which did make the project look way better, but also did give my laptop some troubles which I will talk more about later.

2. Black hole

The second mode of the project takes us into the fascinating world of black holes. To be more precise, the second mode gives the user the ability of the black hole at their fingertips, or should I say, mouse pointer. The particles are now start being dragged into the black hole as the planet disappears outside of the users view.

2.1 Black hole “sub-mode” (It’s not a bug its a feature!) 

When testing and implementing all the different modes I discovered something very interesting. If, during the 2nd mode, the user puts their mouse in the middle and lets the particles come together and then quickly removes the mouse it makes the particles explode in different directions creating a beautiful scene. I decided to leave the bug and use it as a feature in the design.

3. Galaxy

The third and final mode is the galaxy mode. Inspired by our galaxy, The Milky Way, it is supposed to represent the beautiful formation of galaxies in and outside of our observable universe. Particles are the main theme of this mode as they create a loop like shape that simulates galaxies. But since I felt that this mode could have some extra interactivity I implemented the click mechanic that pushes particles away.

I believe all the modes turned out how I wanted them to be and one of my favorite things to do is switch between modes and watch how the particles react to the different states they are given and I hope the users will enjoy them too.

Milestones

The road from the blank canvas to the finished product was a fun one but it did bring some challenges along the way. In the following passage I will try to explain some of the systems implemented in the project and how they work as well as walk you through some challenges I faced and what I did to overcome them.

From the start of the project I knew I was going to work with the particle system and guided by previous experience from class where my laptop kind of struggled to run the basic particle system I knew I was in for a ride. But to make my life easier, before implementing the particle system using textures and WEBGL I decided to create them with simple dots that my laptop would be able to run and which I could later replace for particles.

This turned out to be a great idea as it helped me develop the logic without making my laptop work too hard so I could focus on the logic more than worrying about performance.

Orbit mode

The orbit mode was used as a base for the whole project. Essentially, I wanted to simulate a static planetary system where all objects moved in a predetermined circular pattern. Instead of coding circular motion, I wanted to implement a combination of forces to simulate such a system.

Each particle is assigned a target radius, which determines the ring it belongs to. These radii are grouped into bands to create multiple layers of rings:

let targetR =
  band < 0.60 ? random(115, 170) :
  band < 0.90 ? random(185, 245) :
                random(265, 335);

There are also multiple forces that act on the particles to create the orbit effect.

Gravity like attraction:

let grav = toCenter.copy().mult(0.14 * planet.massScale / (1 + d * 0.006));

Tangential force:

let tangential = createVector(-toCenter.y, toCenter.x);
tangential.mult(0.22 * planet.spin);

Spring force:

let err = d - p.targetR;
let spring = toCenter.copy().mult(err * 0.008);

Together all of these help create the ring like structure we see on the screen.

Galaxy mode

One of the most important design decisions was to re-seed particle positions when switching to galaxy mode:

initGalaxy();

Instead of placing particles in discrete rings, they are distributed in a dense radial disk, with higher concentration near the center:

let diskR = 18 + (-log(1 - u)) * 60;

Also a characteristic of galaxies is the fact that particles closer to the center rotate faster than those further away. I implemented this using:

let orbit = 1.0 / sqrt(r * 0.85)

Thankfully I didn’t have any major challenges other than my laptops performance and the rest of the project went smoothly. I had to increase the pixel density when I was saving the photos but since my laptop struggles so much to run it I returned it to 1 so I could actually watch my project.

Video documentation

Video showcasing the interaction withing the project:

Final sketch

Reflection and Future Improvements

Working on this project was very fun and scary at the same time. It was fun to get to implement things that we have worked on in previous classes and to bring everything together into one project, but it was scary to think about everything that could go wrong on such big project and to have to worry about laptop performance holding me back.  But overall I am very happy with how the project turned out and am excited for others to experience it also.

As for the future improvements I would definitely love to play with colors and add more planets and different galaxies to the project. I would also maybe explore implementation of sound in some way but am not sure what kind at this moment.

Inspiration

The inspiration for this project comes from Team Lab’s Graffiti Nature and Beating Earth from Team Lab Phenomena Abu Dhabi. I was inspired by the fact that the installation uses digital ecosystems to create something that feels alive, immersive, and responsive to human presence. The creatures within the installation are soft, glowing, and constantly moving, while the environment does not necessarily feel like an animation, but rather something that feels alive and responds to the things that are happening within it.

This does not simply show moving creatures and plants within an environment; rather, it feels like an entire ecosystem that incorporates life, growth, and death. This became the basis of my project and the reason I choose this visual.

My twist

My initial twist was to introduce the idea of healing versus pollution. While the original project centers on a living, breathing ecosystem, I wanted to take it a step further to highlight the effect of human intervention on such an ecosystem.

In my sketch, I wanted to provide the user with some power. Glowing flower-like seeds, spawn around the canvas and serve as an energy source for the fish. The user also has the power to spawn flowers on click which gives the user a constant choice between helping and damaging the environment. This would enable the fish to thrive and reproduce. On the other hand, I wanted to provide the user with another power: to introduce pollution blobs to the environment. The fish would start avoiding the place of pollution simulating real life where our pollution and wrongdoing force away the creatures, or in this example fish, from their natural habitat. This twist provided more depth to the project. Instead of simply recreating an animated ecosystem, I was able to transform it into a system where the user was responsible for creating a world.

Process

At first I started with the simplest step, the background. I noticed that in the Team Lab space the background had subtle lines that would give the space more depth and maybe try to simulate being under water.

This was done just by using simple lines and playing withe the color and placement. Once I was happy with the background it was time to go onto more exciting stuff and that was adding the fish. At first I was a bit scared of tackling this task as I was afraid simple design would make the project look too bland so I decided to add some glow effect and transparency as well as adding a subtle trail behind the fish to make the project look like a real time piece of art.

After the fish were done it was time for the final steps of adding the seeds spawning and the ability for the user to spawn pollution.

Code I am proud of

One part of the code I am particularly proud of is the logic that allows the fish to detect and move toward the nearest seed. I like this section because it makes the creatures feel more alive and intentional. Instead of moving randomly across the screen, the fish respond to the environment by seeking out glowing seeds, which helps create the feeling of a living digital ecosystem.

// attracted to healthy seeds
let closestSeed = null;
let closestSeedDist = Infinity;

for (let s of seeds) {
  if (!s.healing) continue;
  let d = dist(this.pos.x, this.pos.y, s.pos.x, s.pos.y);
  if (d < closestSeedDist) {
    closestSeedDist = d;
    closestSeed = s;
  }
}

if (closestSeed && closestSeedDist < 180) {
  let desired = p5.Vector.sub(closestSeed.pos, this.pos);
  desired.setMag(0.12);
  this.applyForce(desired);
}

This simple implementation makes the world feel so much more alive and brings my recreation much closer to the original Team Lab project.

Future improvements

I am already really happy with the final result, but if I was to add anything new it would probably be the explosion mechanism that the Team Lab uses. This would be a fun implementation and I could probably use the particle system that we learned in class how to use, but because of the timeframe I decided not to include that in the current state of the project. Overall I am happy with the final result and have learned that recreating someone else’s work with a twist is actually an amazing way to learn and practice p5.

Concept

For the midterm I decided to go back to space theme like I did in assignment 3. I really liked the design of the planets I came up with and wanted to possibly further explore that design with adding particles as an addition. I want again to use the planets as attractors but I would like to explore the idea of using the particle systems as little particles in space that go around these planets and make different constellations like Saturn’s ring.

Implementation

• Background atmosphere

  • Stars placed in the background randomly.

  • A starfield with subtle twinkling for texture and scale.

• Planet body

  • A solid planet drawn with  shadows and rings to give depth.

• Ring particle system

  • Each particle is assigned a target ring radius (three radius “bands”).

  • Motion is driven by a combination of forces:

    • Inward pull (gravity-like) to keep particles bound to the planet.

    • Tangential swirl to create orbit motion.

    • A spring force that pulls particles back toward their target radius so the ring stays structured.

Light damping/drag to keep it smooth and stable.

States/variation

At this point in time I still don’t have multiple states of the sketch, I am still thinking about what I could make different exactly for now some ideas were either placing different kinds of planets that have varying gravitational pull or removing the planets to create galaxy like objects. Or potentially giving an option to add multiple planets.

Scary parts

Right now the scariest part was actually making the particles go in a circle around the planet. I achieved this by doing the following:

First I calculate the vector pointing from a particle to the planet

let toCenter = p5.Vector.sub(planet.pos, p.pos);
let d = max(toCenter.mag(), 1);
toCenter.normalize();

After which I create a tangential force which actually gives the particles the circular motion

let tangential = createVector(-toCenter.y, toCenter.x);
tangential.mult(0.22 * planet.spin);
p.applyForce(tangential);

I also added some pull to keep the particles from going away

let grav = toCenter.copy().mult(0.14 * planet.massScale / (1 + d * 0.006));

And that’s how I created the ring around the planet. I am just a little worried if making the galaxy style sketch would bring more complexity to the design and if I will need to use something different for the particle movement.

 

Concept

When looking at Memo Akten’s work there was one piece that really took my attention. It was the “Simple Harmonic Motion #12 (2015)”. This although seemingly simple work with people hitting drums with their flashlight really caught my attention by the way it was done with playing with the lights and having all the actors “controlled by the computer”. So I decided that this is the perfect piece to somehow try to recreate in a similar way in p5.

Process

To begin with i first made the “map”. I decided to go with a “stage” looking design to fit the theme so to create something that fits what I imagined I made a completely black background with gray boxes to indicate stages where the people could move.

After it was time to create the performers. I created a performer class where I could create each of the human like performers that would move around the screen. I also decided to give them hands to give them some personality.

At first I experimented with random timing for movements, but this did not feel intentional or rhythmic. The animation looked chaotic rather than composed. To fix this, I replaced random triggers with sine wave oscillators, meaning that every action was driven by harmonic motion instead of randomness.

let g = (sin(frameCount * 0.02 + this.pid * 0.7) + 1) * 0.5;
this.onStage = g > 0.65;

let a = (sin(frameCount * 0.05 + this.pid) + 1) * 0.5;
this.armTarget = a > 0.75 ? 1 : 0;

After motion was working, I implemented the spotlight effect. Rather than drawing a single circle of light, I layered multiple translucent ellipses. This creates a gradient glow that visually resembles stage lighting and feels more organic than a flat shape.

The final step was adding sound. Each time a performer raises their arms while in the spotlight, a drum sound is triggered. I detect this moment by checking for a transition from arms-down to arms-up. This event-based logic ensures the sound only plays at meaningful moments instead of continuously.

Code Highlight

The proudest part of my project was definitely the movement.

let g = (sin(frameCount * 0.02 + this.pid * 0.7) + 1) * 0.5;
this.onStage = g > 0.65;

let a = (sin(frameCount * 0.05 + this.pid) + 1) * 0.5;
this.armTarget = a > 0.75 ? 1 : 0;

I am particularly proud of this part because it captures the entire conceptual idea in just a few lines of code. Instead of manually controlling each performer or assigning scripted movements, each performer is moved by a sine wave with a slightly different phase offset. This means every performer follows the same rule but behaves differently over time.

Future Improvements

If I were to continue working on this project I would maybe like to add some interactivity into it. At this point in time I am not sure how interactivity would work on this project and in what sense it could be implemented so I would love some suggestions. I would also maybe work a bit on visuals, but overall I am really happy with the final project.