Digital Experience Designer
Challenge

Goldfish

A joyful interactive experience between VR and the reality through the perspective of a goldfish.

[VR, Interaction Design & Production]

Challenge

Challenge

To explore the possibility of bringing the sense of touch into VR, Roland and I attended the Reality Virtually Hackathon 2019, a three day hackathon that brings together interdisciplinary minds to create new experiences using VR, AR and MR at MIT Media Lab. How can we come up with an idea and form a team to build a proof-of-concept within three days?

Ideation

Ideation

In most VR games, we are sitting in a small space alone looking into a boundless world. What if we revert the dynamic, experience VR in a swimming pool, free from the constraint of gravity while we interact with the environment above us? How does it feel like being a small fish looking at the world outside of the fish bowl? With these questions in mind, Roland and I decided to create a performative VR sculpture that resembles life through the lens of a goldfish.

Team

Team

We pitched our idea to 300+ hackathon attendees and found three like-minded creatives to join our team: Huiyi (a Masters Candidate at ITP with expertise in interactive performance), Connor (a PHD Candidate at MIT focused on Lidar research) and Maxime (a game music developer from Berklee College of Music).

 As a team, we finalized our storyboard and divided production based on our expertise and interest: Roland on creating and attaching fins to user’s hands with Leap Motion in Unity.

As a team, we finalized our storyboard and divided production based on our expertise and interest: Roland on creating and attaching fins to user’s hands with Leap Motion in Unity.

 Maxime on sound design using Wwise. Connor on interaction design for the cat paw using Unity and HTC Vive.

Maxime on sound design using Wwise. Connor on interaction design for the cat paw using Unity and HTC Vive.

screenshot3.png
screenshot2.png
 Huiyi and I on developing the environment and visual effects in Unity. I also served as the point person for our team which I attended meetings with the hackathon organizing team, documented our process and oversaw our project archive.

Huiyi and I on developing the environment and visual effects in Unity. I also served as the point person for our team which I attended meetings with the hackathon organizing team, documented our process and oversaw our project archive.

User Scenario

User Scenario

Two players are involved. Player 1 is put into the perspective of a goldfish via a VR headset and a Leap motion that turns Player 1’s hands into fins. Player 1 explores the world he/she lives in contained by a fish tank. Player 2 acts as a cat holding a cat paw with tracker attached. Player 2 uses his/her paw to attack Player 1 where his/her claw will appear in Player 1’s point of view.

user_testing_1.gif
User Testing I

User Testing I

The virtual cat paw appeared to be effective due to users’ responses. Many found the approaching claw scary.

IMG_7156.JPG
IMG_7148.JPG
IMG_7149.JPG
IMG_7151.JPG
Fabrication

Fabrication

Besides using software, we also built a physical cat paw out of recycling materials we found at MIT’s junkyard and attached it to a tracker.

User Testing II

User Testing II

We showcased our project at Reality Virtually’s Public Expo on January 21, 2019.

 Little kids loved our project!

Little kids loved our project!

 Especially when given the cat paw to attack Player Goldfish.

Especially when given the cat paw to attack Player Goldfish.

 Adults enjoyed our Goldfish experience too!

Adults enjoyed our Goldfish experience too!

 It was a competition for some users.

It was a competition for some users.

IMG_0778.JPG
 Some Goldfish Players liked to fight back.

Some Goldfish Players liked to fight back.

 While others enjoyed the haptic feedback offered by the cat paw.

While others enjoyed the haptic feedback offered by the cat paw.

Reflection

Reflection

We successfully completed our cat interaction with a swimming goldfish as a proof of concept. More importantly, we are very happy that we not only created a project that brought so much joy into people’s lives but also bonded with interdisciplinary minds who shared the same passion.

Next Step

Next Step

We would like to experiment how the sensations of our project can be enhanced by submerging in a swimming pool. We are in conversation with a fellow hacker we met during the Hackathon, Pierre Friquet, who have created an underwater VR headset.

We are also interested in how haptics can improve motor functions and hand accuracy of stroke and parkinson's patients and would like to work with institutions to extend our research.

Meanwhile, we envision Goldfish to be an interactive installation at an aquarium or a science museum.