Nvidia announced significant updates to its Isaac SIM robotics simulation tool at the Consumer Electronics Show (CES) on Tuesday.

The Isaac SDK is the first open-source robotic AI development platform with simulation, navigation and manipulation. Developing partners use software tools to build and test virtual robots in realistic environments under various operating conditions. Now accessible from the cloud, Isaac SIM is built on Nvidia Omniverse, a platform for building and managing Metaverse applications.

The demand for intelligent robots is increasing as more industries adopt automation to address supply chain challenges and labor force shortages. According to ABI Research, the installed base of industrial and commercial robots will grow more than 6.4 times from 3.1 million in 2020 to 20 million in 2030.

According to Gerard Andrews, product marketing manager for Nvidia’s robotics developer community, developing, validating and deploying these new AI-based robots requires simulation technology that places them in realistic scenarios.

Isaac Sim enables roboticists to import the robot model of their choice and fully utilize its software stack to create a realistic environment to validate the physical design of the robot and ensure performance. Users can generate synthetic datasets during simulations to train the robot’s AI models that are used in the robot’s perception system. Researchers can take advantage of the Reinforcement Learning API to train models in the robot’s control stack.

The latest version focuses on improving performance and functionality for manufacturing and logistics robotics use cases. The software now supports adding people and complex conveyor systems to the simulation environment, and more assets and popular robots are pre-integrated to reduce simulation lead times.

release highlights

Robotic Operating System (ROS) developers find support for ROS 2 Humble and Windows. Robotics researchers gain many new capabilities aimed at advancing reinforcement learning, collaborative robot programming, and robot learning.

Systems improvements focus on the needs of humans working with collaborative robots (cobots) or autonomous mobile robots (AMRs). Isaac Sim’s new people simulation capabilities add normal human-like behaviors to the simulation.

For example, developers can now add human characters to simulations of a warehouse or manufacturing facility, tasked with performing common behaviors such as stacking packages or pushing carts. Many of the most common behaviors are already supported using commands.

To reduce the difference between the results observed in the real world versus the simulated world, physically accurate sensor models are essential. Nvidia’s RTX technology enables the Isaac SIM to render physically accurate data from sensors in real time. Ray tracing with greater speed and accurate sensor data under different lighting conditions or in response to reflective materials, in the case of RTX-simulated LIDAR (Light Detection and Ranging).

More tools for robotic researchers

Isaac Sim also provides a number of new simulation-ready 3D assets critical to creating physically accurate simulated environments. According to Nvidia, everything from warehouse parts to popular robots is ready, so developers and users can start building quickly.

Three new capabilities strengthen the toolset for robotics researchers:

• Advancement in Isaac’s gym reinforces learning.
• Isaac Cortex improves collaborative robot programming.
• A new instrument, Isaac Orbit, provides a simulation operating environment and benchmarks for robot learning and motion planning.

Isaac Sim supports the simulation of warehouse conveyors and people. (Image credit: Nvidia)

Expanded use of robotics underway

According to Nvidia, the robotics ecosystem is already spread across a range of industries from logistics and manufacturing to retail, energy, sustainable farming and more. Its Isaac robotics platform provides advanced AI and simulation software as well as accelerated computing capabilities to the robotics ecosystem. Over a million developers and over a thousand companies rely on one or more parts of it.

Samples of robotic operations include:

• TeleExistence has deployed beverage restocking robots in 300 convenience stores in Japan.
• To improve safety, Germany’s national railway company Deutsche Bahn trains AI models to handle important but unpredictable corner cases that rarely happen in the real world – such as luggage falling onto a train track.
• Sarcos Robotics is developing robots to pick up and place solar panels in renewable energy installations.
• Festo uses Isaac Cortex to simplify programming for cobots and transfer simulation skills to physical robots.
• Fraunhofer Isaac is developing advanced AMR using the anatomically accurate and full-fidelity visualization features of SIM.
• Isaac is using Replicator for flexible synthetic data generation to train AI models.