WORK TYPE
Group work
@UC Berkeley
TIME
Oct 2023 - Dec 2023
3 months
SKILLS
Physical Computing
Circuit Prototyping
TOOLS
Arduino
MaxMsp
The wearable device enabled users to create musics with various plants through touch, tap, rub ,and all different kinds of motions, and the generated music would reflect both the attributes of the plant and the interaction between them.
The music sounded clear and crispy according to the feature of the petals that contains more moisture and has a smooth surface.
The wood branch had a relatively rough and dry surface, which resulted in a deep drum rhythms accompanied by low-pitched piano.
People tend to feel overwhelmed by the fast-moving and unstoppable pace of the life in modern society. Their minds are saturated and dulled by the massive loads and gradually lose the ability to perceive the existence of the environment we're living. The indifference we hold toward the environment further deprive our ability to save memories. We are like the left passenger standing still on the platform, watching trains roaring overhead and leaving nothing but blank.
Thus, we aimed to recall people's awareness to the world we live in with the most primal healing power, the Nature, through our most intuitive sense, touch.
Since the change of lifestyle would be a prolong process and need the persistent commitment to the activity, the main challenges we addressed was to attract people to interact with plants and keep it as a routine. Therefore, We investigated into the formation of people's memory and habit, and we built the feedback loop by referring to the process. Every action is accompanied by the corresponding response at the moment, and our brain decides whether it's impressive enough to be processed and saved to our library of memory. By the next time we encounter the related things that recall the event, the bound of memory will be reinforced and evoke the emotion at the scenario, and that is the principle of how a core memory or habit is formed.
The parts we needed to focus on were the instantaneous feedback of the interaction, the later on reminder, and how to incorporate these interactions and people's perception into meaningful output transformation. We came up with sound as the solution to the medium of the feedback, for its capabilities of giving immediate stimulation as well as being saved and replayed.
In this stage, we tried to transmit the serial data Arduino got from the sensors and mapped them with different attributes in the music in an external music process software Maxmsp. However, the differences of music were hard to observe when the input values were converted to similar elements, and the connection of sound between each trait of plants was weak. Thus, we came up with the solution of decreasing the amount of input sensors and creating reasonable mappings of the plant traits and music attributes, which led to our final design decision.
After the prototype, we turned to focus on tactile experience within the interaction with plants and selected two iconic senses - the texture and moisture content of plants - that are the most obvious senses except vision which has already been bombarded in users’ lives. As for the input artifact for recording the data of interaction, we hope to encourage people’s engagement with minimum interference to their original behavior. Thus, we minimized the form of the input devices to wearable accessories.
From our observation in the Post-human Design workshop led by proffesor Kimiko, rubbing on objects and feeling the textures is the most intuitive interaction people will perform in the natural environment. Thus, we installed one SW-18010P vibration sensor on the fingertip of the user and invited them to touch the surface of plants with fingers moving around as a scanner to record the bumpiness and roughness. Additionally, the sensed value can also indicate the strength and frequency of the motions the user interacts with plants.
We used the value of capacitive sensing as the indication of the object’s moisture content, which is directly proportional to each other. One advantage of capacitive sensing is that users can interact with plants with any part of their body and do not have to directly contact the plant with the sensor due to the conductivity of the human body, which enables the flexibility of the users’ motions. We designed a bracelet with foil paper inside, which is directly attached to our skin and can conduct the capacity of touched object.
To create a sensible and reasonable conversion from raw data of plants to real-time music sound, we analyzed the perceptions and feelings that each trait of plants might bring. The high capacity value indicates high moisture content, which is associated with a vibrant and water-drop like sound. Thus, we mapped it to the higher piano pitch proportionally. As for the bumpiness of the surface, which reminds people of the percussion, it’s transferred a lower and stronger beat of drum sound if the surface is rougher.
The algorithm of the real-time communication of serial data and value conversion to instrument sounds in Maxmsp
Since our goal of this project is to evoke people’s consciousness of the environment and call for being back to nature, we created a physical artifact that enables the users to play the music they gathered and remix them with tangible interaction that will enforce people’s memories with the nature and make them stay away from other digital devices. Thus, we borrowed the interaction of a record player, with a turntable tonearm for turning on/off the player, the needles for switching and remixing different tracks of music, and a spinning record.
First, users can spin across the tonearm above the record, and then the music will begin playing as the record starts spinning. Then users can adjust the needles to play, switch the track, and remix different tracks. We implemented photocell as the switch of needles. When the needle pin points to the record, it indicates that the player will play the corresponding track.
This design provides a cross-sensory experience of human-plant interaction and ensures the encouragement for people to interact with plants with a closer scale. The users found it intriguing after understanding the pattern of the sounds and the corresponding attributes of plants. Many aspects can be explored to refine the design:
The wearable devices were wired, resulting in users' insecurity and limiting their actions. A wireless deign will enable users to interact and observe plants in a more natural way.
The current design of the physical player borrowed the image of a record player too much, which is confusing and fragile for users to operate it. Instead of using metaphor of other artifacts, the connection between the generated music and the original plant should be stronger, which can be realized by real-time projection or display of the images of the plant for instance.
Including the bio-metric data such as the user's pulse rate into music creation not only can personalize the creation and increase the sense of engagement but also can indicate and track the user's mental status.
This design can support a whole ecosystem including the human community and the natural environment. By improving the inclusivity, this mutual creation can be enriched and reinforce the connection between a single user and the ecosystem.