Deepfakes in Education: Cyberbullying in the Age of AI

How Deepfakes Are Being Used Against Students and Staff

Deepfakes are driving a rise in “sextortion.” 

“Kids are sending out a picture to somebody, and that picture gets converted into something it wasn’t. Now they’re saying, ‘Hey, pay this or I’m going to tell all your friends and family,’” says Jennifer Duer, executive vice president of product at Lightspeed Systems. 

Faculty and staff are also being victimized. 

“Students are taking pictures of staff members and using AI to make it look like staff members did something inappropriate,” she says. “That causes investigations and staff members getting suspended. It causes a lot of administrative burden for the school and also distress for that staff member and that community.” 

Beyond just time and trouble, deepfake cyberbullying can become a legal issue.

The Legal Landscape: State Laws, School Liability and Student Privacy

The federal government recently enacted the Take It Down Act, which criminalizes the nonconsensual online publication of intimate visual depictions, including AI-generated deepfakes. Most states now have similar legislation in place.

Many schools are still catching up with the specific language of these laws. 

“At the school level, there is usually no specific deepfake policy,” says Teddy Hartman, senior director of privacy and data policy at GoGuardian. “But most school systems do have responsible use and cyberbullying policies in place. Those still work, even with this very extreme and graphic form of cyberbullying.”

Given the risk of civil liability, schools need to get out ahead of this. 

“Are parents going to sue the school if this happens on school property, and their student is being cyberbullied? There are a lot of things for them to think through,” Duer says.

Deepfake Detection Capabilities and Limits

There are off-the-shelf deepfake detection tools available — tools that leverage the power of AI to analyze images and flag suspect content — but they have their limitations.

“Some hit a pretty good benchmark in the 90%, but it’s not something you can fully rely on,” Duer says. “AI is getting better at tricking those. It used to have telltale signs, where somebody had an extra finger or weird-looking teeth. But now they look really realistic.”

More general detection and prevention tools can help here. At GoGuardian, for example, “a visual nudity detection system can detect nude images dynamically,” says Brian Larkin, director of product management. The algorithms automatically block that content, whether it’s real or a deepfake, “and it’s not going to be accessible on school-issued devices.”

Lightspeed’s solutions can help as well, by detecting explicit images and alerting the school. An anonymous reporting tool “gives students a way to raise their hand and say, ‘This is going on, and I don’t think it’s right,’” Duer says.

SUBSCRIBE: Sign up to get the latest EdTech content delivered to your inbox weekly.

Updating Acceptable Use Policies and Student Codes of Conduct for Generative AI

At Westfield High School in New Jersey, at least 30 female students were victimized by a student who created deepfakes morphing their faces onto pornographic images.

The school responded with an acceptable use policy for generative AI. It specifically calls out deepfakes, prohibiting “AI-generated content that falsely represents individuals, events or information, especially when intended to deceive, harm or manipulate others.”

Existing policies that don’t cover synthetic media will need updating. 

“They need to call out specifically synthetic media of real people. Whether that’s used for harm or not, kids really shouldn’t be making fake pictures of others,” Duer says.

Student Digital Literacy: Teaching Evaluation of AI-Generated Content

In the era of deepfakes, “schools need really strong digital citizenship classes,” Duer says. “They can be teaching them things like how to do a reverse image search, using the tools at your disposal to say, ‘Is this something I can find online? Is it something that existed somewhere else?’”

It’s important, too, that kids understand how digital media can be abused once it’s out in the world. “That means knowing the consequences of what can and unfortunately does happen with deepfakes,” she says.

Overall, kids need to be taught to ask questions. 

“It’s about being skeptical: Is that really that person posing in that way?” Hartman says. “We need to be teaching the benefits of AI and also the harms, and how to evaluate content.”

Incident Response for Deepfake Events

In addition to preserving the evidence and involving law enforcement, if necessary, schools can respond by steering parents to available resources. 

“There’s the Take it Down website, a free service that can help you remove and stop the sharing of nude images of yourself or others,” Duer says.

Tech tools can also help schools respond to deepfake incidents. GoGuardian has an automated process for schools to report images of children through the National Center for Missing and Exploited Children, “and then, we have a whole process where we can take it down off their system,” Hartman says.