Prototyping of Immersive HRI Scenarios

Dennis Krupke, Sebastian Starke, Lasse Einig, Frank Steinicke, Jianwei Zhang
Proceedings of CLAWAR 2017, page (accpeted) - sep 2017
This paper introduces an open-source software toolkit for combining the Robot Operating System (ROS) and Unity3D to versatile robotic applications involving virtual environments. Despite the availability of high-quality robot control and simulation systems like ROS, there is still no framework available for designing complex human-robot interaction tasks and making use of Virtual Reality without expert knowledge in robotics and ROS. Virtual Reality, especially involving head-mounted displays as well as various input and feedback devices can increase the experienced sensation, improve the understanding of certain 3D scenes or provide a test environment for the training of non-expert operators. As a solution, we propose Unity3D for designing the interaction interface to virtual and real robots. Our implementation of the bilateral communication layer between Unity3D and the ROSbridge and the importer for XML-based les in Uni ed Robot Description Format allows synchronizing the state of the real robot and the one of its simulated counterpart in the Unity3D environment in a very comfortable way. Using the proposed system we present di fferent use cases and demonstrate how to prototype di erent interaction concepts eciently.

Images and movies


BibTex references

  author       = "Krupke, Dennis and Starke, Sebastian and Einig, Lasse and Steinicke, Frank and Zhang, Jianwei",
  title        = "Prototyping of Immersive HRI Scenarios",
  booktitle    = "Proceedings of CLAWAR 2017",
  pages        = "(accpeted)",
  month        = "sep",
  year         = "2017",
  url          = ""

Other publications in the database

» Dennis Krupke
» Lasse Einig
» Frank Steinicke
» Jianwei Zhang