Building an AI Center of Excellence for Higher Education

What Is an AI Center of Excellence in Higher Education?

An AI CoE in higher education creates an operating model that allows AI to function across the institution, from classrooms and research labs to student services.

“In higher education, an AI center of excellence is the team and operating model that makes AI usable across the whole campus,” says Louis Stewart, head of strategic initiatives for the developer ecosystem at NVIDIA.

Without that structure, AI adoption doesn’t stop. Different departments just adopt different tools, apply different standards and set different expectations for students. “If you don’t centralize, AI still shows up everywhere, just inconsistently,” Stewart explains. “That creates avoidable risk around privacy and academic integrity, and it slows the institution’s ability to turn experimentation into real results.”

At New York University, the AI CoE reflects the need for coordination without overcentralization. According to Kelsey Cook, senior director of AI strategy and enablement, success involves three priorities: keeping the institution safe, enabling innovation where it naturally occurs and making that innovation visible across a distributed environment. “A successful CoE has to grow people, not just serve them,” Cook adds.

Shadow AI on Campus: From Uncoordinated Use to Structured Adoption

Much of the urgency around AI CoEs stems from the rise of “shadow AI,” or unsanctioned tool use across campus. But both Stewart and Cook suggest that the term can be misleading.

Rather than viewing shadow AI as a compliance problem, they see it as evidence of demand. “Most people are trying to be productive, not break rules,” Stewart says. The solution is not to shut down experimentation but to understand it and guide it.

That starts with visibility. Institutions need to understand what tools are being used, where and for what purpose. From there, they can provide clear guidance, vetted tools and safe environments for experimentation.

Cook reframes the issue like this: “The most important question is, how do we create a unified presence around the technology we use and make sure that the innovation that’s already happening is visible and safe?”

DISCOVER: Five considerations for building an AI-ready infrastructure.

Balancing AI Governance With Academic Freedom

One of the defining challenges in higher education is governance. Unlike enterprises, universities don’t operate under a single chain of command. Faculty autonomy, research independence and shared governance are foundational. That makes a purely centralized model unworkable. Instead, institutions are adopting a hybrid approach.

“Centralized governance needs to own the nonnegotiables, but within that structure, there has to be room for departments and faculty to experiment,” Cook says. “The right tool for a computational researcher is not the right tool for a finance administrator or an artist,” she adds.

At NYU, this has meant implementing strict guardrails to protect institutional data, including ensuring that university data and prompts are not used to train public large language models. However, Cook stresses that governance is not about acting as a gatekeeper. By framing governance as a shared responsibility — one that asks whom a decision affects, what risks it introduces and how it can be reversed — the CoE shifts from restricting innovation to enabling it. “The goal isn’t to eliminate friction,” she says. “It’s to make sure friction is doing useful work.”

Stewart echoes that perspective, noting that effective governance frameworks are not just comprehensive but usable. When policies are clear and tools are easy to adopt, safe use becomes easier than unsafe use.

Building an AI CoE: Roles and Stakeholders

Because AI touches every part of the institution, a successful CoE must bring together stakeholders who don’t typically collaborate. Academic leadership, IT and security, legal and privacy teams, research leaders, and teaching and learning specialists all play essential roles. Faculty champions are equally important, translating strategy into practice at the departmental level. Stewart also points to career services as a critical, often-overlooked partner. Connecting AI initiatives to workforce outcomes helps build trust and demonstrates tangible value.

At NYU, many of these groups already exist within governance structures, including faculty committees, IT councils and leadership groups. The challenge is not creating new bodies but aligning existing ones.

“The infrastructure for coordination already exists,” Cook says. “Our job now is to make those structures work in concert rather than in silos.”

Role-Based AI Literacy: Faculty, Staff and Administrators Need Different Training

AI literacy is central to any higher ed CoE, but it must be tailored to individual roles.

As Stewart notes, “meaningful AI literacy is role-based”; for example, faculty need to design assignments for an AI-enabled world, while staff must be able to use AI safely in advising and administrative workflows. Scaling that knowledge requires approaches such as train-the-trainer models and structured learning pathways.

At NYU, the emphasis is on expanding access to these capabilities so the benefits don’t only flow to well-resourced departments.

How AI CoEs Serve Both Research and Operations

Higher education AI CoEs must operate across two distinct but interconnected domains: research and institutional operations. Focusing too heavily on one or the other risks limiting impact.

Stewart recommends treating the CoE as a portfolio rather than a single initiative, with one track supporting advanced research workflows and another focused on improving the systems that affect every student. Keeping those tracks aligned through shared tools, policies and metrics ensures that AI remains a campuswide capability rather than a siloed effort.

This dual focus reflects the broader mission of higher education, where innovation and public service are inseparable.

LEARN MORE: How these higher ed institutions are immersing faculty in AI adoption.

How to Launch an AI CoE and Why Adaptability Matters

For institutions building an AI CoE from the ground up, the first steps are relatively straightforward: establish leadership, assess current usage and launch with a focused set of use cases that demonstrate immediate value.

But long-term success depends less on structure than on adaptability.

“You can design a governance structure, hire a team and write a charter, and still build something too rigid to actually work,” Cook says. Universities are not single organizations but collections of communities, each with its own needs, language and relationship to technology.

“The most important quality of a CoE isn’t its structure,” she adds. “It’s its adaptability.”

That adaptability, combined with clear governance, cross-functional collaboration and a focus on people, ultimately is what distinguishes an effective AI CoE from just another institutional initiative.