Why Workload Placement Determines AI ROI in Higher Ed

What Is a Hybrid Cloud Strategy in the Context of AI?

Dave McCarthy, vice president of cloud and edge infrastructure services at IDC, says in higher education AI, a hybrid cloud strategy is far more than a tactical combination of on-premises hardware and public cloud instances.

“It serves as a deliberate, integrated architectural framework designed to harmonize data flows, model training, fine-tuning and inference across private facilities, public clouds and edge endpoints,” he explains. 

Rather than treating these environments as isolated silos, a true hybrid strategy establishes common data planes, unified security protocols and containerized workload mobility.

This allows an institution to dynamically match each phase of the AI lifecycle to the infrastructure environment that best optimizes performance, security, compliance, latency and cost.

Running AI in Hybrid Environments Without a Strategy

Robert Daigle, director of global AI business at Lenovo, says over the past two years, organizations have prioritized pilots and proofs-of-concept, often deploying AI wherever infrastructure and data were readily available rather than following a cohesive, long-term plan. 

This “build fast” approach has helped validate use cases, but it has also introduced unintended consequences ranging from fragmented architectures to inconsistent governance.

“Without a clear strategy, these environments quickly accumulate technical debt, leading to suboptimal designs, duplicated infrastructure and escalating costs,” Daigle says.

In effect, many institutions have arrived at hybrid AI by necessity, not design.

The next phase of AI maturity will require a shift from experimentation to intentional architecture, where strategy, governance and workload placement are aligned from the outset to avoid cost overruns, and to advance teaching, learning and research at scale.

DIVE DEEPER: Learn what you’ll need to build a foundation for scalable AI.

On-Premises vs. Cloud vs. Edge Decision Framework

Murali Gandluru, vice president of product management for data center networking at Cisco, says workload placement decisions must be driven by workload intent and its specific requirements.

“The edge is essential for real-time inference where latency is a dealbreaker, such as autonomous systems on a factory floor,” he explains.

On-premises environments are often the best choice for training models on massive research and institutional datasets, where data gravity makes moving information to the cloud prohibitively expensive or risky.

He adds that the public cloud remains the ideal venue for burst capacity, rapid prototyping and accessing specialized AI services that are not feasible to maintain in-house.

“The leadership challenge is ensuring these choices don’t lead to forklift upgrades every time a workload needs to scale,” Gandluru says.

A Flexible Governance Framework for Distributed AI Pipelines

Pravjit Tiwana, senior vice president and general manager of cloud storage and services at NetApp, says the most successful institutions separate governance from infrastructure.

“In the past, governance was often tied to a specific environment,” he notes. “In AI, that approach breaks down because workloads, models and data are inherently distributed.”

From his perspective, governance needs to become policy-driven, data-centric and portable. Colleges and universities need consistent controls for lineage, access, retention, sovereignty and model usage, regardless of whether a workload runs on-premises, in a public cloud or at the edge.

McCarthy says that to build a flexible framework, institutions must transition from rigid centralized gatekeeping to automated, policy-as-code guardrails embedded directly into continuous integration/continuous delivery pipelines. 

“This approach balances innovation and control by establishing global compliance baselines, such as data lineage and bias checks, while granting localized teams the autonomy to execute within those boundaries,” he explains. 

DISCOVER: How to optimize your institution’s infrastructure for AI.

Cost Visibility and Resource Allocation Across AI Environments

Tiwana says the institutions that succeed won’t necessarily be the ones that spend the most, but rather the ones that can move workloads to the most efficient environment at any time.

“That requires visibility not only into model costs but also into storage, networking, data movement and infrastructure utilization,” he says.

He points out that colleges and universities are asking for the ability to run AI workloads wherever economics, performance and governance are most favorable, without having to rebuild their pipelines. 

“That’s one reason we believe portability across AI infrastructure will become increasingly important,” he says. 

Gandluru says institutions must move beyond basic billing dashboards to full-stack observability, where IT leaders can correlate graphics processing unit utilization, network interface card metrics, optics health and job-level telemetry — monitoring everything from GPU usage to data egress fees in one place.

“That correlation lets you make data-driven decisions about where workloads should run and catch performance bottlenecks before they hit application throughput,” he says. 

LEARN MORE: Observability tools help institutions monitor their IT environments.

Security and Compliance in Hybrid AI

Daigle says AI is forcing a fundamental shift away from perimeter-based security. In a world where data, models and AI agents operate across edge, data center and cloud, the traditional notion of a fixed boundary no longer applies.

“Security and compliance must now be policy-driven, data-centric and continuously enforced across a highly distributed environment,” he explains.

Equally important is security at the model and training level. Institutions must ensure that training data is governed, models are validated and outputs are aligned with institutional policies.

This includes rigorous review processes, compliance checks and centralized oversight frameworks to manage risk across the AI lifecycle.

A Roadmap for Lift-and-Shift to Intentional Architecture

McCarthy says IT leaders can move past basic lift-and-shift tactics by designing architectures around specific workload intents, mapping data dependencies, latency needs and compliance limits before placing compute.

“This transition is enabled by standardizing on cloud-native containerized platforms and a unified data fabric to ensure seamless, cost-effective portability across environments,” he says. 

Daigle advises colleges and universities to focus on placing workloads where they best support teaching, learning and research. This requires leaders to evaluate workloads based on performance, compliance and cost requirements rather than defaulting to a single environment.

“The most successful organizations are taking a workload-first approach by building flexible hybrid architectures that allow them to run the right workload at the right time,” he says.