Mixed reality simulation combines real and virtual objects in the same simulation environment. Applications that also include some form of haptic interaction are categorized as visio-haptic mixed reality (VHMR). With the advancement of augmented reality technologies, there has been a growing interest on VHMR. Positive contribution of haptic interaction especially in training simulators is brought to attention in recent literature.
A haptic interface is designed to generate force feedback in a mixed reality environment. This haptic interface allows users to use their real-life tools. Real-life tools are attached to end-effector of the haptic system when haptic interaction is needed according to virtual object shape. Therefore 3D position of the tool in space should be determined precisely.
This study focusses on 3D position tracking of slender real-life tools (i.e. scalpel, screwdriver) in a novel VHMR environment, in which the free end of the tool is automatically engaged/disengaged by a haptic interface according to the haptic rendering requirements of the simulation. A depth-camera based method is developed, markered and markerless tracking approaches are compared in terms of speed and precision.