How Does AI-Generated Content Work?
Tianfu “Matt” Wu, associate professor in North Carolina State University’s Department of Electrical and Computer Engineering and head of the university’s Laboratory for Interpretable Visual Modeling, Computing and Learning, says there are two major models being tested: a text encoder trained to understand text inputs and an image decoder trained to synthesize a photorealistic image from the output of the text encoder.
“To a certain extent, the models are trained to encode and align different content, such as imagery and text, in some mathematical latent vector space, on top of which the models can be deployed to generate content,” Wu says. “The mapping between content and vector is made possible due to the remarkable capability of deep neural networks and the availability of web-scale paired data, such as image-text pairs.”
Text prompts, however, are only one possible input or “condition” for generative AI content. Other inputs include sketches, other images or randomly sampled vectors, which follow a process called “unconditional generation.”
What Potential Does AI-Generated Content Hold?
While generative AI might simply seem like a fun experience for casual users, Wu says this technology could significantly affect society. In fact, AI-generated content could serve as “powerful and useful assistants and assets to human experts by significantly improving their productivity and by potentially augmenting their creativity,” he says.
