Speakers: Thomas Endres & Jonas Mayer
In this talk, Thomas Endres and Jonas Mayer demonstrate how game engines can be used to efficiently train and evaluate robot foundation models for new control tasks. Using Counter-Strike as a reproducible test environment, they show how core robotics challenges - including continuous action control, real-time inference, and imitation learning - can be investigated under realistic conditions.
The speakers explain how they adapted NVIDIA's GR00T-1.5 model for this task, addressing challenges such as inference latency, discrete input signals, and discontinuous movement. They discuss the role of high-quality demonstration data, analog control inputs, and techniques such as real-time action chunking in achieving stable control. The talk illustrates how these engineering principles transfer to industrial robotics applications that rely on simulation-based training and learning-based real-time control.