Generation of Synthetic 3D Data Using Simulated Mr Examinations in Augmented Reality

Future medical imaging devices such as computed tomography (CT) and magnetic resonance (MR) are expected to become increasingly autonomous. Therefore, design criteria and workflow of such devices will change substantially. Moreover, sensor data of the system are required to develop scene understanding algorithms to support and guide the user. In this work, we present an approach to generate synthetic 3D point cloud data from a simulated MR examination experienced on the Microsoft Hololens2. The complete workflow of an MR examination using a virtual, autonomous MR scanner is reproduced in an AR scene. The user is recorded by a system of active stereo vision RGBD-cameras while interacting with AR elements. A registration routine of the AR scene and the RGBD-cameras is described, and accuracy measurements are provided. The real point clouds are fused with virtually generated point clouds from the AR scene.