Perspective Block

Sharing Perspective in Remote Collaborative AR

Role:

  • HCI Researcher
  • Interaction Designer
  • Augmented Reality (AR) Developer

Tools:

  • Unity Engine
  • Vuforia Augmented Reality Library
  • Google Cardboard
  • User Studies
  • Qualitative Interviews

Abstract

Augmented reality provides an intuitive way to work with 3D models because of the natural mapping to our physical movement and the benefits of AR technology. However, it does not easily support remote collaboration due to difficulties in communicating spatial and contextual information. We investigate this problem by introducing the Perspective Block, a novel handheld interaction tool. This block has an embedded camera and sensors that can be used to capture important perspective information such as contextual location and video feeds. By using this block, collaborators will be able to share their perspective with each other using a variety of interaction techniques, allowing for more natural communication. These interaction techniques were evaluated through an informal study in order to gather feedback on their effectiveness and potential of use in a real world context.

Poster

Problem

3D Models are found everywhere in our world but navigating and working with them still feels unintuitive using traditional 2D interfaces. Augmented Reality (AR) provides a more natural way of interacting with them, but it does not easily support collaborative work, especially remotely. In particular, communicating perspective information with your partner is very difficult as it is hard to convey important spatial and contextual information.

Approach

Our approach to this problem is to explore the concept of perspective sharing through a novel handheld tool. This tool, the Perspective Block, allows the the user to convey perspective information through an embedded camera and sensors. Using AR, two collaborators will be placed in a space with a shared 3D object that they can independently interact with.

The prototype was built using Google Cardboard for the head-mounted AR display, a smartphone for the Perspective Block and the Vuforia library for rendering augmented models.

Various interaction techniques to share perspective information were implemented and compared to one another.

pblock-approach1
pblock-approach2

The left figure shows the Perspective Block setup and the right figure shows what is seen on the AR display.

View Pointers

These were visual representation of the remote partner’s view perspective in the 3D space. The pointers are mapped directly to the partner’s movements so you can can get a sense of what they are looking at without being too distracting. We tried a yellow block that matches the partner’s block in space, a subtle green spotlight, as well as a geometric viewcone.

pblock-viewpointer1
pblock-viewpointer2
pblock-viewpointer3

Video Feed

These video feed techniques captures a live feed from the partner’s block and embedded it into your AR view. That way, you can directly monitor what your partner is currently seeing right now, allowing for a more traditional form of communication similar to video conferencing. The difference is that you can control whether you want a direct feed or something more subtle with the other techniques.

pblock-videofeed1
pblock-videofeed2

Evaluation

We ran an informal study session in order to gather feedback on the viability of the prototype and concept. Two architecture students were recruited to assist as the prototype had potential in that field as a potential alternative to traditional 3D software tools. The study consisted of a critique task where they were asked to perform an evaluation on 3D house models. They were taught how to use all the interaction techniques and were given the freedom to use them however they wish to assist in their task. A quantitative survey was held afterwards for feedback.

The participants found the prototype to be novel and presents interesting ideas, but it was also quite difficult to use. They were not sure if the tool had provided much feedback over traditional software as the AR does not act that much to the task other than increased complexity. The video feed techniques were preferred to the pointers, likely because it was easier to understand when directly looking at a video feed compared to trying to interpret more information.

Discussion

There is still a lack of viable remote collaboration tools for working with 3D models. Alternatives include screen sharing software or just verbal conversations, so there is still a large gap in the market here. The interaction techniques explored here explore ways in how AR might be used to help address this problem, and further work can take these and apply it to more context specific situations. Although the Perspective Block in its current iteration is not ready to solve the problem, it demonstrated the value of perspective sharing in remote collaborative work and paved the path for future work to follow.