The training and simulation industry has made tremendous progress with software virtualization of physical environments, but the integration of tactile input/output remains nascient. While there are many ways to connect wired peripherals to desktop computers, there are few solutions for fully mobile applications. Connecting physical devices through USB or BLE to battery-powered AR/VR headsets is technically challenging as these headsets run mobile operating systems like Android or Windows Holographic that do not possess drivers or even ports for connecting external hardware. Further, the precise communications protocols that these devices use are rarely, if ever, documented.
This paper presents a recently patented engineering technique and solution for rapidly integrating BLE/USB devices into mobile platforms. When confronted with a known or unknown device, the process begins by leveraging open source tools to discover and analyze the device's emitted messages. From this log, a human-readable configuration file is constructed that contains semantic decriptions and measured values in a specific shareable format. Finally, configurations are fed into a lightweight and cross-plaform device driver that proxies raw wired and wireless packets including the oft-overlooked HID protocol. The end result is mobile support for a wide array of wearable and mountable input devices, and a manner for adding new devices without coding.
Immersive training applications will be empowered to connect wearables like smartwatches and fitness bands, mountables like commercial flight simulator controllers and smart home panels, and countless other new and old IoT devices. As the communication is inherently bidirectional, integrations provide for both user sensor input to the app and tactile feedback from the app. General compatibility with input devices will allow the next generation of immersive training apps to be more realistic, versatile, and transportable for a variety of mission needs.
Keywords
AUGMENTED AND VIRTUAL REALITY (AR/VR),WEARABLE DEVICES
Additional Keywords
tactile input, sensors