Project Name: Starry Night: A Flowing Painting
A. Concept
B. Implementation
Here are the questions to solve to achieve my goal of creating a flowing painting.
-
- Swirling movement of the particles
- Location of the center
- Aesthetics of the strokes
1. Swirling movement of the particles
Firstly, I need to know how to make the particles move in a circular pattern. The particles need to be placed in a flow field that they will be given the velocity according to their position in the field. However, there are different kinds of flow fields: Perlin noise flow fields, vortex flow fields, magnetic flow fields, and so on. I searched the Internet and found a simple way of creating this swirling effect.
Basically, you create a background image showing the distribution of the centers of vortices based on a 2D Perlin noise space. And then you calculate the “pressure differential” around every cell in the grid. That differential vector will point to the center of the vortex, creating an attraction effect to the particles. However, if you rotate every vector by 90 degrees, they will suddenly be turned to some force similar to the “tangential force”. This will magically turn the movement of the particles into a swirling pattern.
Generating a flow field with Perlin noise is a good starting point.
p.s. Here’s a good way of modifying the noise field to make it cleaner and less “noisy”. You can change the parameters in this function called noiseDetail(octave count, falloff).
I also make a 3D noise field that changes through time.
Here are some generative patterns I created from the swirling field:
2. Location of the center
Then, we need to know the rotation center of the stars.
In the previous sketches, I used the Perlin noise field to determine the rotation centers. Every time, the resulting image is different because of the difference in random Perlin noise.
However, the Perlin noise is too random and you can’t really decide where your centers are. Therefore, I came up with the idea of allowing the users to paint on a buffer canvas and have control over their center locations.
The method is to use pixel manipulation.
I wrote a function called renderImg() that modifies an initially pure black buffer canvas. Whenever the users press their mouse, the pixels around the mouse position will gradually turn brighter according to their distance from the mouse position.
(A clearer demonstration)
Then, the users will be able to draw their own “base image” in the field and affect the movement of the particles.
Do you want to have a try? ( press i to show your base image)
i
Some result:
(Stroke-like effect)
Lastly, apart from user modification, I also want to “animate” the original painting by Van Gogh. This requires us to take the original painting as the source image.
But how can the computer know the locations of rotation centers of the stars from the painting?
At first, I tried to detect the pixels that have a higher grey scale (brighter). But as you can see, there is only one star that is statistically much brighter than others. This method didn’t work well.
Then I noticed that all the stars are kind of yellow. Therefore, I thought of calculating the Euclidean distance of the RGB value of the pixel with the RGB value of a “standard yellow color”. After some tests, I found that this rule works the best (but still not perfect) anyways I tried…
abs(r-g)<=20;
r: 180 += 20;
g: 165 += 20;
b: 60 += 30;
And then I used the previous renderImg() function to render the base image based on the original painting. You can see that nearly half of the stars have been detected.
Finally, I combined the Perlin noise flow field with this swirling field. Adding some dynamics to the sketch. I also grabbed the original color from the painting and modified the color of the pixels.
Here is the final sketch:
C. Reflection
Program thinking that I gained from this project:
- Combination of forces
- I combined three kinds of force to affect the velocity (change of position) of my particles: Centripetal force, tangential force, and force due to Perlin noise). The combination adds a lot to the dynamic and aesthetic of the project.
- View buffer canvas as a data source
- A generative art must have a generative rule. And I think my “generative rule” is the base image placed on a buffer canvas that contains information in every single pixel and tells the particles which direction to go. At first, I placed 2D or 3D Perlin noise images on the buffer canvas, then I allowed the users to add their own stars to this canvas through pixel manipulation, and at last, I extracted information from the original Starry Night painting and used it as a data source.
- Computer vision
- Of course, I only scratched the surface of “computer vision”. But I believe that my attempt to identify the positions of the stars in the painting through different methods is kind of asking the computer to see and interpret the visual information that humans can easily capture. Through setting rules such as “find pixels with higher greyscale” or “find pixels with smaller RGB distance to the standard yellow”, the computer tries to better detect the positions of the stars with optimized logic.
Unsolved problem:
The biggest challenge I encountered in this project is the pixelDensity() and scaling down and up the image. Since the source image was too large, it needs to be scaled down. Plus I had some buffer canvases, and I often needed to check the index and location of certain pixels and modify them. I used a lot of pixelDensity() function, but it led to some weird results that I couldn’t understand. Therefore I had to change all the pixel densities to 1 to avoid those problems, but it made the resolution of my project less ideal.
Future Improvements:
I’m excited to make the project run in a 3D space where the particles rotate around a sphere. Also, due to the limited capacity of p5js running in a browser, I couldn’t add more particles to make a more appealing effect. So if possible, I will run it in Processing or other faster tools.