Improving MR Visualization of Digital Twins using HoloLens 2

Abstract

One goal of the project described in this paper is to create learning algorithms for machines and robots that lack a precise virtual controller for correct simulations. Using a digital twin approach, the developed mixed reality application aims for an overlay of a virtual robot model with the real world counterpart using Microsoft HoloLens 2 smart glasses. The application should help users to have an inside look into the results of the learning algorithm and therefore supervise and improve those results. The main focus of this paper is the visual representation of the digital twin on the smart glasses. One of the challenges is the level of abstraction and specific use of shaders (program code defining material attributes) to help the user differentiating between virtual and real objects. Therefore different presentation methods are described and evaluated. Study results with 48 persons show that the most abstract representation (wireframe) scores lowest, whereas a half-transparent model works best.

Keywords: Mixed Reality, Visualization, Digital Twin, Human-in-the-loop