Dieter Fox got his PhD in Computer Science at the University of Bonn, the former sleepy German capital, in the late 1980s. He then made his way way West as Director for Intel’s Seattle Lab and roughly for the last 20 years he has been a Professor of Robotics at the University of Washington (UW).
NVIDIA, the AI chip maker, hired him as the (Senior) Director of Robotics Research a little more than a year ago, and now they made him the Leader of their new AI Lab in Seattle.
The charter of the lab is to drive breakthrough robotics research to enable the next generation of robots that perform complex manipulation tasks to safely work alongside humans and transform industries such as manufacturing, logistics, healthcare, and more, said Nvidia in a release.
“In the past, robotics research has focused on small, independent projects rather than fully integrated systems. We’re bringing together a collaborative, interdisciplinary team of experts in robot control and perception, computer vision, human-robot interaction, and deep learning,” said Fox.
Close to 50 research scientists, faculty visitors, and student interns will perform foundational research in these areas. To ensure the research stays relevant to real-world robotics problems, the lab will investigate its work in the context of large scale, realistic scenarios for interactive manipulation.
Fox talks about the AI Lab in the video below.
NVIDIA’s deep learning-based perception system was trained using the cuDNN-accelerated PyTorch deep learning framework, and built on these techniques:
- Dense Articulated Real-Time Tracking (DART): DART, which was first developed in Fox’s UW robotics lab, uses depth cameras to keep track of a robot’s environment. It is a general framework for tracking rigid objects, such as coffee mugs and cereal boxes, and articulated objects often encountered in indoor environments.
- Pose-CNN: 6D Object Pose Estimation: Detecting the 6D pose and orientation of known objects is a crucial capability for robots that pick up and move objects in an environment. This problem is challenging due to changing lighting conditions and complex scenes caused by clutter and occlusions between objects. Pose-CNN is a deep neural network trained to detect objects using regular cameras.
- Riemannian Motion Policies (RMPs) for Reactive Manipulator Control: RMPs are a new mathematical framework that consistently combines a library of simple actions into complex behavior.
- Physics-based Photorealistic Simulation: NVIDIA’s Isaac Sim tool enables the generation of realistic simulation environments that model the visual properties of objects as well as the forces and contacts between objects and manipulators.
