AUGMENTED REALITY PHOTO ALBUM

I created a augmented reality physical photo album which has videos inside the pictures as a way to relive your memories. Here, I grabbed a family video and made them into still images and videos in the album. Since the videos did not come with audio, I created my own audio for the photos and videos in the album. In this project, I wanted to explore ways of capturing and remembering your memories in the most real form which includes movement and sounds. The photo album is a way of looking back at the happy moments that you would want to remember and cling on to, in case you forget.

Role: (Solo Project) Art, Programming

Tool: Unity, Vuforia, C#, Reaper, Photoshop

Duration: 1 week

Thanks to Alex Petrusca for helping me with programming the audio detection part

 

HOW IT WORKS

Hold the physical photo album up to the unity app (on phone or on laptop) and videos and audio of the family will play on top of the photos, which are the image targets.

 

TECHNICAL PROCESS

Firstly, I wanted test if Vuforia would recognize photos easily, so I had a test photo and put various 3D objects and lights on it, to see if it will recognize the photo. It turns out, that with some contrast correction, most photos that are asymmetric are good for image detection. Detecting photos are crucial for the app because my plan was for a video to play on top of the photo, as if the photos were moving. 

 

Then, I grabbed still screenshots from Jim Cave's family videos as image targets. So, when the target is recognized, the same videos will play on top of the images, as if you are actually reliving those physical photos from the album.

 

Next, I figured out how to put a video on top of the photo with help from Sydney Zhang. To do that, I made a very flat prism the size of the photo and overlaid it on top of the image target on Unity. Then, I created a video player and a video texture. I put the video texture in the video player, and then I created an empty material. Next, I put the video texture onto the empty material, and put the whole material on the prism I created initially. The image target will now have a video playing on top of it! I tried first with a sample heart target to see if the video would actually work.

Lastly, I constructed and mixed my own audio I found online and put it in the script of unity, so when certain targets are detected, audio will play.

 

photo album AR.gif

Last Notes

I think this is an idea that could be developed further if I were to make it again, and I also think it would be interesting to further develop the aesthetics of the physical album. If I were to iterate, I  want to create a narrative with the movies, and place more specific and interesting audio recording which would reveal more of the family in a more real, and not just idealized, sense.

Furthermore, I think this technology could be turned into an app where people could load their own personal videos, and a physical photo album could be made, as a way for people to further relive their happy moments.

Note: The videos and photos are grabbed from Jim R Cave’s youtube videos, a family I don’t know. However, I think it is interesting to explore the family photo album as an art object of voyeurism and construction of family moments that want to be remembered.

Thanks to Claire, Sydney, Peter, and Alex for helping me out learn Vuforia for the first time. Special thanks to Jim R Cave for the youtube home videos.

This technology was also used in my HCI project for Paper Hearts, a greeting card digital service, with gifs down below