babl is a mixed reality language learning platform 

designed for Head-mounted displays

Why is the second language so hard to learn? Why school only teaches expressions that native speakers don't use?
BABL started with a bold idea - Could the learners experience a real live language through local people's life?

BABL is a design challenge project for Microsoft Design Expo 2017 with the prompt Intentional Design for Positive Cultural Impact in Mixed Reality. It asked us to consider the context and implications of the technology we’re designing for, while simultaneously demonstrating its clear value and differentiation.

For 4 months, we conducted three main design researches - exploratory, generative, evaluative research - and presented the research takeaways to Microsoft UX design lab.

TRUST-BASED lANGUAGE LEARNING NETWORK

BABL is a social language learning platform grown by participants; the more people join the network, the more language contents is created. The system help learners find the right host based on their interests, motivations and profiles.
 

TOUR MODE - mixed reality as Shared reality

Tour mode is an immersive live tour hosted by one's local partner. Learners will see, hear and feel how local people live.
With mixed reality technology, we envision that BABL can bridge between learners and local people and start the meaningful relationship. Through the shared reality, learners can get immersed into the new environment and eventually integrate themselves with the target culture.

PRACTICE mode - capture & embody LIVE LANGUAGE

Practice mode provides a safe space for learners to practice spoken language with captured moments from the tour mode. Learners can control the speed, toggle the caption and get live feedback from the system.
They can also annotate and put it into the captured scene with the connected screen device.

GESTURE INTERACTIONS

Once getting into the mixed reality mode, learners and hosts interact with the system with gaze, gesture and voice.  
BABL supports immersive and easy learning experience with the combinations of gestures. 

Swipe to open and close the toolbox panel
Reveal left to right for right-handed users and right to left for left-handed users

Gaze to target an icon & freeze the view
Pinch and drag to doodle/pin on the field of view and release pinch to finish

Gaze to target an icon & Pinch to toggle caption/grammar
Speak to proceed conversation practice

Write a memo on the babl app & push the memo into the MR view.
Drag & drop to place the memo where it’s most relevant.