The New Frontier: Mike Kelly on the Value of High-Quality Data and High-Level Support

EDTECH: What are the opportunities for analytics in higher education?

MIKE KELLY: Most of the analytics efforts I’ve heard of are focused on the student, in terms of what’s happening in the classroom and their recruitment, retention and graduation. We don’t tend to do as much on transactional systems, business processes or building operations, which are more the norm in industry; it’s looking at logistics and efficiency.

A lot of institutions are like small cities: We serve meals. We receive deliveries. We have electricity and health centers. Some have pharmacies and hospitals. If a city’s public works or transportation department can get analytics benefits, it stands to reason that we can realize those same benefits.

EDTECH: What is the analytics strategy at your institution?

KELLY: The holy grail would be significant improvements in student outcomes. I wouldn’t say we have a comprehensive strategy; we have a few project under way, including the EAB analytics system and a homegrown initiative named Beyond the Classroom Matters, where we are trying to assess the impact of students’ involvements outside of the classroom.

Our Student Affairs and Academic Support division has argued for many years that extracurricular activities, like working outside of school, have an impact on students’ educational engagement and outcome. We are trying to measure and assess some of that.

EDTECH: How would you describe your role on campus?

KELLY: We have a distributed approach, where the Student Affairs and Academic Support division is the champion of the EAB and Beyond the Classroom Matters initiatives. As CDO, I’ve been involved on a limited basis. I’m more data governance–focused, developing good practices of data stewardship and standards and of data quality and integrity. It’s a comprehensive framework aimed at producing the most reliable data we can for both mandatory reporting and nonmandatory analytics.

We envisioned my role as a 50-50 balance between governance and analytics, but what we found is that investing in the quality and standardization of data in our systems is critical. I’m hoping to free up some time, so I can lend my expertise and tackle broader, more extensive analytics efforts in the future.

EDTECH: What are best practices for moving toward a data-driven decision-making model? 

KELLY: Executive support is very important. If someone on high says, “We’ve got to do better and I expect outcomes on my desk,” it will provide resources, bandwidth and interest in what you are doing. You will have a hard time getting traction with a grassroots analytics initiative.

A team approach is important. Without it, you are not getting a complete picture. You can’t have IT doing analytics and leave admissions, the registrar’s office and the academic departments out in the cold. You will reach the wrong conclusions if you are not talking to the people working on the front line with students. By the same token, a student-oriented analytics initiative that does not look at students’ financial well-being might arrive at the wrong conclusions.

Data governance – and ensuring data is as reliable and high-quality as possible – is another piece.

Another best practice is a deep consideration of privacy. In higher education, we have the ability to get an intrusively deep picture of human beings. We can see what time they swiped their IDs to their dorms, what time they went to the dining hall, and when they went to the gym or entered certain buildings for a lecture. We can put together a disturbingly detailed picture of a human being within our premises. We really need to put thought and consideration to privacy in analytics.

EDTECH: What other issues do higher education institutions face as they implement data analytics initiatives?

KELLY: In higher education, the educational enterprise is really good at postulating hypotheses and doing research against that to find out what we think is true. Faculty research is all about that, and we are on the same path with analytics.

We think there is insight we can glean, but we also have to hesitate to leap to conclusions. There is a very human element to all of this. We have to couch findings in terms of the truism, “correlation is not causation.” The data may initially point in one direction, but we want to move cautiously. We don’t want to assume the conclusions the data points us to are always going to be right.

EDTECH: What are the challenges of implementing an analytics program? 

KELLY: The biggest challenge is the same for analytics as it is for everything in higher education: There is not enough money to support it. The second challenge is agreeing to goals and priorities. There are a lot of topics and subjects we can apply analytics to. We currently have distributed initiatives going on.

EDTECH: From a technical perspective, what are the biggest hurdles to gathering and leveraging data from multiple systems?

KELLY: A comprehensive data warehouse strategy is needed, and the ability to trace the lineage of data elements and identity management is a huge technical and functional effort. And then there’s the mashup process, where sometimes multiple systems contain the same data because of system integrations.

Sometimes it becomes problematic in trying to figure out which system has the most authoritative data — the cleanest data — at any point in time. Things like students’ GPAs or housing information and knowing which system has the best, most accurate information.

EDTECH: How should colleges evaluate whether they need a CDO or equivalent?

KELLY: Data governance and analytics initiatives won’t just happen; they need constant leadership. If a vision is taking shape but progress is slow, a CDO should help.

Part of The New Frontier series of interviews with top data experts in higher education.