NVIDIA enhances robotics platform with latest AI & simulation tech

NVIDIA is expanding its robotic platform, NVIDIA Isaac, with the fusion of the latest generative AI and advanced simulation technologies. The enhancement is set to speed up the development of AI-assisted robotics. NVIDIA's efforts revolve around adding new foundation models, a robot learning framework, and AI workflow orchestration and robot perception tools, among other elements.

The additions were disclosed by NVIDIA head and founder, Jensen Huang, at GTC. Among the announcements, the company revealed Isaac Manipulator and Isaac Perceptor, collections of foundation models, robotics tools, and GPU-assisted libraries. Project GR00T, a foundation model for the learning of humanoid robots, was demonstrated by Huang during the event.

According to Huang, the creation of foundational models for general humanoid robots is one of "the most exciting problems to solve in AI today." He highlighted the significant potential of these advancements for helping roboticists worldwide to progress towards artificial general robotics. Huang also revealed a new computer for humanoid robots, highlighting the NVIDIA Thor system-on-a-chip, and new tools for the NVIDIA Isaac robotics platform, like Isaac Lab for robot learning and NVIDIA OSMO for hybrid-cloud workflow orchestration.

The NVIDIA Isaac Manipulator, a collection of advanced motion generation and modular AI capabilities for robotic arms, was another revelation. Robotic developers can leverage software components tailored for specific tasks, allowing repetitive workflows for dynamic manipulation tasks. Deepu Talla, vice president of robotics and edge computing at NVIDIA, commented on the importance of these new tools in the acceleration of smarter and more flexible robots.

Several leading robotics companies - including Yaskawa, Solomon, PickNik Robotics, READY Robotics, Franka Robotics, and Universal Robots, a Teradyne company - are collaborating with NVIDIA to bring Isaac Manipulator to their customers. In a bid to push the boundaries of where robots can operate across industries, Yaskawa's president, Masahiro Ogawa, asserts that offering NVIDIA AI tools and capabilities will have a significant impact on various industries.

Further, NVIDIA is introducing foundation models to boost existing robot manipulation systems, assisting in the development of robots for tasks such as smart manufacturing, handling pick-and-place tasks, machine guidance, and assembly.

Isaac Perceptor, providing multi-camera, 360-degree vision capabilities, is another offering advancing the use of autonomous mobile robots in industries such as manufacturing to promote efficiency, safety, reduced error rates, and cost savings. NVIDIA, in partnership with Segway Robotics and Leopard Imaging, developed the NVIDIA Nova Orin DevKit to fast-track the development, evaluation, and deployment of Isaac Perceptor.

Project GR00T is a standalone initiative with a focus on humanoid robots that incorporate multimodal instructions, including text, video, and demonstrations, as well as previous interactions, to generate desired robot actions. This novel approach is made possible through the use of various Isaac tools available to robotics developers.

Finally, in continuing to facilitate robotics development, NVIDIA has introduced Isaac Lab, aimed at promoting deep reinforcement learning in a simulated environment for advanced robotic locomotion, and NVIDIA OSMO, a platform designed for cloud-native robotics workflow scheduling. Isaac Lab and OSMO, along with NVIDIA's Isaac Manipulator and Isaac Perceptor, are set to become available to customers and partners in the second quarter of this year.