Artificial Intelligence

How NVIDIA Is Making It Easier to Bring Robots to Campus

NVIDIA is providing tech stacks, software frameworks and teaching kits for universities and colleges to research innovation in physical AI.

Brian T. Horowitz

Twitter

Brian T. Horowitz is a writer and editor covering enterprise IT, innovation and the intersection of education and technology.

Physical artificial intelligence has emerged as the next wave of AI, and colleges and universities are exploring how to incorporate autonomous machines into their work and studies. Universities are experimenting with physical AI in various industries, from healthcare to retail, and training students to join those fields.

Today’s physical AI allows organizations to automate physical work and perform complex tasks. These robots can manipulate objects, like on a conveyer belt, and perform surgical tasks such as threading needles and performing stitches. Humanoid robots are general-purpose robots that require fine motor skills and the ability to interact with the physical world.

“Physical AI is really the convergence of AI algorithms (the brain) and physical hardware (the body) and enabling that robot or a machine to reason, perceive and act in the real world,” says Ronak Shah, senior manager of higher education and research at NVIDIA.

The previous generation of robots performed repetitive motions in warehouses. Now robots learn new skills through trial and error, which is too risky for physical hardware. We use reinforcement learning in high-fidelity simulations, allowing an AI to practice a task millions of times and prepare for real-world unpredictability, Shah says.

DIG DEEPER: Find out how NVIDIA can help colleges take AI integration to the next level.

“They require you to understand computer science, neural networks, large language models, computer vision, sensor and signal processing, simultaneous localization and mapping, and various other engineering technologies,” Shah says.

Colleges and universities are testing early-stage robots for food delivery as well as providing self-driving shuttle service for some campus routes, according to Shah.

Teaching Kits and Programming Robots at Universities

NVIDIA offers teaching kits for robotics that become part of a university’s curriculum. The kits include labs, lectures, homework assignments, slides and demo problems, Shah says. NVIDIA’s Deep Learning Institute enables students to access graphics processing units in the cloud through self-paced or instructor-led courses. NVIDIA developed its robotics teaching kit along with leading research labs such as the the University of Pennsylvania’s GRASP Laboratory at Penn Engineering.

Meanwhile, NVIDIA’s Deep Learning Institute lets educators run workshops on the fundamentals of building AI or programming robots, Shah adds. Online learning courses from NVIDIA provide training in Isaac Sim, a reference application that allows developers to develop, simulate and test AI-driven robots, and Isaac Lab, an open-source framework for robotic learning. NVIDIA also provides schools such as Carnegie Mellon University and University of Pittsburgh with access to the NVIDIA NeMo software framework for custom generative AI and NVIDIA NIM microservices.

The Georgia Tech Research Institute has been working with NVIDIA on applied robotics research and applying autonomy to defense-related problems, according to Shah. The school offers programming and training of intelligent machines for unmanned air, ground, surface and subsurface systems that are controlled autonomously. They perform research for DoD missions.

“From a university point of view, it's a massive engineering challenge because to enable physical AI, you have to solve these really large AI problems,” Shah says. “These are also really large sensor and signal processing problems, and that's where NVIDIA’s platform excels.”

Click the banner below to learn how university leaders are adapting to rapid AI advancement.

How Full-Stack Systems Power Physical AI Research at Universities

The two leading R1 research institutions in Pittsburgh are using NVIDIA’s full-stack systems as part of physical AI research. In October 2024, the University of Pittsburgh launched an initiative with NVIDIA to advance robotics, autonomy and AI. At Carnegie Mellon University, researchers are exploring how to validate physical AI performance because robots are unpredictable.

Researchers and educators at universities are exploring how to use large language models to help robotics navigate environments without prompting or programming, Shah explains. Robots incorporate data from cameras, radar, LiDAR, and other sensors.

“At R1 universities, those CIOs are absolutely seeing a lot of demand from their faculty for research, for the need for building large-scale GPU clusters, for training these LLMs from scratch, or specialized derivatives,” Shah says.

“Our goal is to enable general-purpose robots, and that requires a powerful foundation model,” he continues. “Researchers use our platform to train models like Project GR00T, which acts as a generalized brain. It can understand natural language, perceive the world through its sensors, and generate the actions needed to complete a task, effectively learning new skills that can be transferred across different robots and environments.”

The AI model is only as good as the data it receives. That's why so much research focuses on the entire pipeline, from raw sensor and signal processing to perception. The data from cameras, LiDAR, and other sensors are the “tokens” that feed the physical AI foundation model, just as words are the tokens for an LLM, he adds.

Physical AI started with moving materials in manufacturing and self-driving cars in transportation, but research at universities can allow additional applications for AI to emerge. In the future, Shah expects universities to become more multimodal and to be trained in more diverse environments.

On campuses, Shah expects to see additional developments on the horizon in many areas.

“Beyond the research labs, robots will play a large role in operations, commercial kitchens, food delivery, campus security, autonomous science and many other applications,” he says.

Brought to you by