Augmented Reality Music Cubes
Introduction
Since I love creating physical objects, toys, and electronic interactions, I wanted to create something in augmented reality to mixes object and electronics. So, I thought of the idea of linking sounds to physical music notes in origami boxes as a way for the audience to create their own music (inspired by AR paper cubes).
This could be useful app for students or kids to learn music and compose their own harmonies by building notes up with easy-to-maneuver origami music boxes. People could experiment with building structures with the origami music boxes, which will then create interesting sounds with the layer of notes. I do eventually want to design my own app with this idea.
Role: (Solo Project) Art, Programming, Design
Tools: Unity, Vuforia, Adobe Illustrator, Photoshop
Duration: 2 weeks
HOW IT WORKS
First, take PDF of music notes (D, A, G, and etc.) and cut them, fold over the tabs, and then glue them to create origami boxes.
Maneuver the boxes around to create various harmonies. You can scan over the notes/boxes of each and hear the sounds play. Line the notes one by one (or put them on top of each other) and scan the phone over them and hear the musical harmonies.
A simple harmony, Canon, can be played if you line up the music boxes in sequential order with D, A, Bm, F#m, G, D, G, A
For fun, you can build various structures such as walls, buildings, etc. and hear the notes layer!
Images of Music Origami Box Templates
Process
Before coming with this idea, I knew I wanted to do something with augmented reality and sound. So I brainstormed ideas such as having roses (black and red) and having a demonic love song for the dark rose and the same song but sweet for the red rose. Also, I thought of creating a music band with sounds like drum, vocal, guitar, etc. linked to physical objects shaped like those musical instruments so that a full song can be played.
Lo-Fidelity Concept Screens
For an app concept, I want the phone to have an ability to play the note once detected by your choice, as well as visualizing the detection. To do that, an animation of a note might pop out from the image target. I also think people may want to experiment spatially with music, and save the music that was recorded on the phone, as well as the notes that were played on the phone. Thus, I have a record, save, and history feature in this app concept.
Thoughts
For next time, I want to create more specific and contrasted design for the origami boxes pdf. Also, I would want to iterate and perhaps create different shapes of origami prisms to create even crazier structures with sounds. I also want to create a design app for it in the future.
More and better documentation to come!
Next Iteration: Augmented reality object and sound
This was built from the previous project I did where I had origami music boxes and it would play sounds. It was also built from my Augmented Photo Album project, where I had video and sounds overlay on specific moments in family. In order to iterate/make it inter-actionable on those and make it more interesting, I chose household products that would juxtapose each other, and create sounds that would layer.
Each object is different from one another, and I thought the collection of objects can also speak to the person who owns it. In this collection, I was thinking about a newly wed, or even a mom who has kids. I think it is interesting concept that you can look at an object objectively, and it will give clues about the personality and role of the person who owns it. Sometimes, they may even reveal embarrassing secrets, such as the stool softener.
These objects in the household can be performed in theater, where holding up and and combining various objects can create experimental music
In terms of difficulties, I had trouble coming up with a collection of items and sounds that made sense and were interesting at the same time. I think it could be pushed further if I embedded secrets and more surprises in these sounds. Technically, the build on the phone was a bit slow as I was detecting many objects at a time, and the sounds didn’t fade out as I wanted it to. If I were to iterate, I want to pay more attention to the narrative of these objects and experiment with more interesting sounds. I was stuck at the purpose of my AR music of objects, that I think I could have pushed the idea a bit further.
I had more objects with me, but decided to cut them out because their image targets were hard to detect.
Objects included a wedding card, toy dog, crayons, marbles, stool softener, and acne cream
