Portal + Portable

Touchable Augmented Reality On Your Phone

Brown HCI Group

mixed virtual and physical environment Portal-ble is a portable augmented reality format where the smartphone is a portal into the virtual world, so that the virtual and real are merged into a single view seen through the phone display. And now we can interact with both virtual and real objects with our hand.

Seen through the phone (images on right), which coffee is real? Actually, both cups are virtual. And the chair with the red seat is virtual, but the other furniture is real (physical).

usage of portal-ble An infrared sensor mounted on the phone tracks the hand so you can grab and manipulate virtual objects. The system automatically helps users better perceive depth, judge how to grasp objects, and manpulate them, by overcoming our innate human imprecision and tracking inaccuracies.

Go anywhere with Portal-ble.

No headset. No gloves. No joystick. No markers.   Just your hand and your phone.

Video Demo

We are researchers at the Brown University HCI Group, and we want to share this augmented reality format. Instructions for assembling Portal-ble are available on the GitHub page, where the open source software can be downloaded. We use a Leap Motion attached to the back of a phone using a 3D-printed mount. The Leap Motion is connected to an Intel compute stick that wirelessly transmits hand positions to the phone.

portal-ble hardware components person with portal-ble device


A longer description of Portal-ble and its evaluation is published at UIST 2019, the ACM Symposium on User Interface Software and Technology. You can download the paper. Please read the paper and visit the source code repository for implementation details.

The authors (we) have a background in computer science and in design, and are Jing Qian, Jiaju Ma, Xiangyu Li, Benjamin Attal, Haoming Lai, James Tompkin, John Hughes, and Jeff Huang. This work is funded by the National Science Foundation and by Pixar.