Concept:
My concept is about creating a flower that is both digital and physical, making it come to life with user interaction. On the screen, the flower sways in the wind and reacts to how you move your mouse and blow into a microphone. At the same time, a real flower-like structure made with Arduino mirrors this behavior, with its petals moving using a motor and LED lights changing color.
The screen version uses code to draw a flower that looks alive. The petals change size and color when you move your mouse, and the flower sways gently like it is in the wind. If you blow into the microphone, the sway becomes more dramatic, like a strong gust of wind.
The physical version is controlled by an Arduino board. The motor moves the petals, and the LED lights change brightness and color. It reacts to the same inputs as the digital version. Data flows between the digital and physical versions in real time, making them feel connected.
This idea mixes art and technology. It is interactive and fun while also showing how digital and physical systems can work together.
Images:
mouse is moved downwards
mouse is moved upwards
mouse is moved horizontally
User Testing:
Description of Interaction Design:
- Mouse Movement:
- Moving the mouse up or down changes the size of the flower petals.
- Moving the mouse left or right changes the LED color of the physical flower and the digital flower’s appearance.
- Sound Input:
- Blowing into a microphone connected to the Arduino makes the flower sway more dramatically. Both the digital and physical flowers respond to the intensity of the blowing.
- Real-Time Synchronization:
- The digital flower sends data (servo angle, LED brightness, and color) to the Arduino over a serial connection.
- The Arduino processes this data to control the physical flower’s movement and lighting.
How Does the Implementation Work?
The implementation brings the flower to life through interaction and synchronization between a digital animation (p5.js) and a physical flower model (Arduino). The interaction happens in real-time as data flows between the two systems via serial communication.
Since the p5 should control the arduino it means that the p5 should send data to arduino over serial and the arduino should keep listening to the serial port for any data to control the servo motor and the LED.
Arduino Code:
The Arduino code controls the physical flower’s servo motor (petals) and LED lights. It also listens to the microphone input to adjust the flower’s sway.
Key Logic:
Serial Communication: The Arduino reads input data from p5.js to control the servo angle and LED properties.
if (Serial.available() > 0) {
String data = Serial.readStringUntil('\n'); // Read data from p5.js
parseData(data); // Parse angle, brightness, and color
}
- Servo Movement: Smoothly moves the servo motor to the target angle to simulate natural petal motion.
void smoothServoMove() {
if (currentServoAngle != targetServoAngle) {
currentServoAngle += (currentServoAngle < targetServoAngle) ? 1 : -1;
petalServo.write(currentServoAngle); // Adjust servo position
}
}
- LED Control: Updates LED brightness and color based on values received from p5.js.
void updateLEDs() {
uint32_t color = calculateColor(ledBrightness, ledColorValue);
for (int i = 0; i < NUM_LEDS; i++) {
strip.setPixelColor(i, color); // Set LED color
}
strip.show();
}
- Microphone Input: Reads sound intensity from the microphone to determine the sway effect.
int mn = 1024, mx = 0;
for (int i = 0; i < 100; ++i) {
int val = analogRead(MIC_PIN);
mn = min(mn, val);
mx = max(mx, val);
}
Serial.println(mx - mn); // Send sound intensity to p5.js
P5.js Code:
The p5.js code creates a dynamic flower animation and sends control data to the Arduino. It receives microphone input values to update the digital flower’s behavior.
Key Logic:
- Sending Data to Arduino: Sends data in the format
servoAngle,brightness,colorValueto control the physical flower.
let servoAngle = constrain(int(map(mouseY, 0, height, 75, 0)), 0, 75);
let ledBrightness = constrain(int(map(mouseY, 0, height, 0, 200)), 0, 200);
let ledColor = constrain(int(map(mouseX, 0, width, 0, 255)), 0, 255);
let dataString = `${servoAngle},${ledBrightness},${ledColor}\n`;
serial.write(dataString); // Send data to Arduino
- Flower Animation: Draws a flower that sways using Perlin noise for natural motion. The petals’ size and color respond to mouse movement.
function flower() {
let windFactor = noise(frameCount * 0.01) * 30;
let targetSway = inData > 20 ? map(inData, 20, 1024, 0, 50) : 0;
windSway += (targetSway - windSway) * 0.1; // Smooth sway transition
push();
translate(width / 2 + windSway, height / 2);
for (let i = 0; i < 10; i++) {
let petalSize = map(mouseY, 0, height, 25, 50);
let petalColor = lerpColor(color(230, 190, mouseY), color(255, mouseY, mouseY), i / 10);
fill(petalColor);
ellipse(0, 40, petalSize, petalSize * 2);
rotate(PI / 5);
}
pop();
}
- Leaf Falling Effect: Creates falling leaves when the microphone detects strong wind.
if (inData > 40 && !isBlowing) {
isBlowing = true;
fallingLeaves.push(new fallingLeaf(width / 2.1, height * 0.6));
}
Together, these systems create a synchronized and interactive flower experience. The digital and physical elements complement each other, making it both engaging and technically impressive.
Embedded Sketch:
Aspects of the Project I’m Particularly Proud Of:
- Seamless Synchronization: The real-time interaction between the digital and physical flower is smooth and responsive. This makes the project feel alive and dynamic, as the changes on the screen and in the physical flower happen instantly.
- Interactive Design: The project responds intuitively to user inputs like mouse movement and sound. For instance, the way the flower sways naturally with Perlin noise and the dramatic movement triggered by blowing into the mic make the interaction feel organic.
- Creative Use of Technology: Combining p5.js for digital visuals with Arduino for physical movements and lights was challenging but rewarding. The servo motor, LED lights, and microphone sensor work together to mirror the digital animation.
- Visual Appeal: The gradient background, dynamic petals, and swaying effect are visually engaging. Adding elements like falling leaves and soil patterns enriches the experience and makes the animation more immersive.
- Problem-Solving and Optimization: Addressing challenges like abrupt movements, serial communication errors, and performance issues (e.g., limiting leaves per blow) helped make the project smoother and more efficient.
Links to Resources Used:
- p5.js Official Documentation:
- https://p5js.org/reference/
Used to understand functions for drawing, animation, and serial communication.
- https://p5js.org/reference/
- Arduino Official Documentation:
- https://www.arduino.cc/reference/en/
For working with servo motors, LEDs, and reading microphone input.
- https://www.arduino.cc/reference/en/
- Adafruit TiCoServo and NeoPixel Libraries:
- p5.serialport.js:
- https://github.com/vanevery/p5.serialport
To manage serial communication between p5.js and Arduino.
- https://github.com/vanevery/p5.serialport
- Node.js Serial Server:
- https://nodejs.org/
Used to set up the server for serial communication.
- https://nodejs.org/
- Articles and Tutorials:
- “How to Use Perlin Noise in Creative Coding”
https://thecodingtrain.com/
Helped with implementing smooth sway effects for the flower and leaves.
- “How to Use Perlin Noise in Creative Coding”
- Arduino and p5.js Community Forums:
- Arduino Forum: https://forum.arduino.cc/
- p5.js Forum: https://discourse.processing.org/
For troubleshooting serial communication and input handling.
Challenges Faced and How I Overcame Them:
- Synchronization Issues:
- Problem: The digital flower and the physical flower did not always behave the same due to delays in serial communication.
- Solution: Optimized the data sent over the serial connection by minimizing unnecessary updates and using a consistent format. Added smoothing functions to reduce abrupt changes.
- Abrupt Movement:
- Problem: The flower’s sway and servo movements were jerky, making the animation feel unnatural.
- Solution: Used Perlin noise for smooth sway effects and gradually interpolated servo angles with
windSway += (targetSway - windSway) * 0.1.
- Serial Communication Errors:
- Problem: Occasionally, the Arduino would fail to read the data correctly, leading to erratic behavior or crashes.
- Solution: Implemented error handling by trimming data and ensuring the format was consistent. Added checks for valid data ranges before processing.
- Overloading the System:
- Problem: Generating too many falling leaves or frequent updates caused performance issues.
- Solution: Limited the number of leaves generated during each blow cycle and optimized the logic for removing off-screen leaves.
- Mic Input Sensitivity:
- Problem: The microphone often picked up background noise, causing unintentional flower movements.
- Solution: Calibrated the microphone by adding a threshold to filter out low-intensity sounds and used the
mapfunction to scale input values proportionally.
- User Understanding of Controls:
- Problem: Users found it challenging to understand how to interact with the system.
- Solution: Added an instructional overlay with text and icons to guide users on how to use mouse movement and microphone input effectively.
Future Improvement:
- Improved Interaction Design:
- Add more user interaction options, such as touch inputs for mobile devices or keyboard controls.
- Allow users to customize the flower’s appearance (e.g., petal shape, color palette) through an interface.
- Enhanced Physical Integration:
- Use more advanced sensors, like an IMU (Inertial Measurement Unit), to detect hand gestures for controlling the physical flower.
- Add more LEDs or motors to create a multi-layered flower for a more complex and realistic effect.
- Better Performance Optimization:
- Explore using faster communication protocols like Bluetooth or Wi-Fi instead of serial communication to reduce latency.
- Optimize the falling leaf particle system for smoother animation with larger leaf counts.
- Visual Enhancements:
- Make the background gradient dynamic, transitioning based on user interactions or environmental factors like sound intensity.
- Add weather effects, like raindrops or sunlight, to complement the flower’s animation.
- Educational Features:
- Include real-time data visualization (e.g., a graph showing microphone input or servo angle changes).
- Add explanations of how the flower’s behavior is controlled, making it a teaching tool for IoT and creative coding.
- Scalability:
- Expand the project to include a garden of flowers, each responding differently to inputs.
- Enable networked interaction, where multiple users can control different flowers from different locations.
These improvements can make the project more engaging, robust, and versatile for various applications, from education to art installations.
IM SHOW
For the IM Show, I wanted my interactive flower project to be as intuitive and engaging as possible. One key element of the setup was the microphone, which is essential for controlling the flower’s dramatic sway. To make it clear where visitors should blow, I printed out a small microphone image and taped it to the actual microphone sensor. This simple addition helped guide people, ensuring they could interact with the project effortlessly without needing additional instructions.
Challenges:
While setting up the project at the IM Show, I encountered an unexpected issue: when I switched the p5.js animation to full-screen mode, the application started lagging. This was frustrating because the smooth motion of the flower and leaves is a big part of the experience.
After some quick troubleshooting, I realized the issue was related to rendering performance on larger canvases. To fix it, I adjusted the canvas dimensions to be slightly smaller than the full screen. Here’s how I did it:
function setup() {
let width = window.innerWidth * 0.9; // 90% of screen width
let height = window.innerHeight * 0.9; // 90% of screen height
createCanvas(width, height);
noStroke();
}
This solution maintained the immersive feel of a full-screen display while reducing the computational load. The animation ran smoothly for the rest of the show, and visitors enjoyed interacting with the flower.
With the microphone image and the optimized display, people were able to blow into the mic, move the mouse, and watch both the digital and physical flowers come alive in real-time. Seeing the excitement and curiosity on people’s faces as they controlled the flower made all the effort worth it. It was a fantastic opportunity to share my work and learn from the feedback of others!
Second Version:
