What AI Feature Creep Means for Student Data Privacy
Typically, K–12 procurement happens on a regular schedule. Every few years, contracts are up for curriculum or hardware, and buyers put out a request for proposal. That’s when they evaluate potential vendors and their products to find the right fit for the school’s needs — and that’s usually where the assessment ends. It’s rare for schools to go back and re-review tools in the middle of a contract.
That lack of review could affect security, though. Maybe a routine learning management system upgrade introduces AI grading for student work, or the HR system embeds AI candidate screening capabilities. It could even be an update to your security camera system that starts assessing footage and compiling AI analytics.
That’s called AI feature creep. Suddenly, this system that you approved two years ago has new power and new uses, sometimes even giving students access to AI tools that they didn’t previously have. On top of that, third-party AI services might be accessing data you didn’t sign off on at the time. When AI can access student data and private information, that’s not just a security issue, it’s a community standards issue, and transparency is a must.
Mapping AI Tools and Tracking Updates
I always recommend that districts review existing tools regularly, not just when it’s time for contract renewals. It’s a good way to make sure changes that could affect data privacy and security aren’t slipping through unnoticed. But that can be easier said than done, especially for small teams that are covering multiple roles. I talk to K–12 technology leaders every day, and the truth is, they’re often overwhelmed. To go to them and say, “Now you have to go back and vet all of your existing solutions” will only compound that. The best thing districts can do is to build a culture of security that includes everyone, not just IT.
A good starting point is building an inventory of tools that currently use AI. It should include what AI providers and models are being used (such as OpenAI or Google Gemini), what data it can access and what security controls are in place. Then, assign a risk tier. AI grammar checking is going to be a low risk, because it’s likely not touching much data. But AI threat assessment or mental health analysis is a high risk and requires human oversight. Finally, you need to put someone in charge of decision-making. Name someone, such as a superintendent, who is accountable for oversight and has the authority to hit pause while a new tool is evaluated.
Now, when vendors add or update AI capabilities, you already have a way of tracking it. Ask vendors to notify you before AI is added so you can assess and approve the capability before it’s rolled out. When systems are updated or patched, review the update notes. This is a good practice to understand what has changed and how those changes affect your security.
How To Get Ahead of AI Feature Creep in Procurement
Lately, it seems like everything has some sort of artificial intelligence baked in, down to AI-generated email response suggestions and AI-summarized search results. Those tools are already making their way into the devices and programs that K–12 schools use, with more to come. Preparing during procurement can set you up for fewer surprises down the line.
During your next request for proposals, ask questions of your vendors, even if they don’t currently have AI enabled in their tools or services. You can ask things like:
- What AI capabilities does your product currently have?
- What AI upgrades are planned in the next year?
- What data can the tools access?
- Which AI provider are you using?
- What security controls are built around the AI tools?
- How is AI activity logged?
By understanding what AI tool features your district is currently using and what could be coming, you can move from reacting to AI feature creep to proactively planning and preparing for a secure AI environment.