FAQs are below...
What is Artificial Intelligence (AI)?
AI is an umbrella term for a range of technologies and approaches that often attempt to mimic human thought and action to solve complex tasks. Things that humans have traditionally done by thinking and reasoning are increasingly being done by, or with the help of, AI. (Source: ICO)
AI is very different to Artificial General Intelligence (AGI), which one might say is the ‘Terminator/Skynet’ scenario!
What is Machine Learning (ML)?
ML refers to a sub-field of artificial intelligence that gives computers the ability to learn without explicitly being programmed.
Machine learning and artificial intelligence are terms used interchangeably, but most current ‘AI’ tools are machine learning. (Source: MIT)
What is Generative-AI (Gen-AI)?
Generative-AI (or 'Gen-AI) refers to a class of artificial intelligence (or more accurately, ‘machine learning’) algorithms that generate new data samples (‘outputs’), such as image, text, or audio, that are similar to the training data it was trained on. These algorithms can create content autonomously without direct human input, aside from the 'prompt' needed to start the process. In these FAQs, we will naturally be concentrating on image generation as the output type.
How does a generative-AI algorithm generate outputs?
Generative-AI generates outputs by learning patterns and features from a large dataset of existing creative works (the 'training data' or 'training dataset') and extrapolating learned patterns from these works, often using techniques such as Diffusion, Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs). In response to a text input from a user (a 'prompt'), the algorithm is able to generate the output.
What is a Diffusion Model?
Diffusion Models are used to generate outputs similar to the data on which they are trained. Fundamentally, Diffusion Models work by destroying the training data through the successive addition of Gaussian noise, and then learning to recover the data by reversing this noising process. After training, the Diffusion Model is used to generate data by simply passing randomly sampled noise through the learned denoising process.
The most common text-to-image generators use this process, such as DALL-E 3, Stable Diffusion and Midjourney.
What is a Generative Adversarial Network (GAN)?
A Generative Adversarial Network (GAN) is a type of generative model consisting of two neural networks, a generator and a discriminator, which are trained simultaneously in a competitive setting. The generator generates fake images, while the discriminator tries to distinguish between real (training dataset) and fake images. Through this adversarial training process, GANs learn to generate increasingly realistic images.
How does 'style transfer' work in generative-AI?
Style transfer is a technique in generative-AI where the style of one image (such as the brushstrokes of a painting) is applied to another image while preserving its content. This is typically achieved using convolutional neural networks (CNNs) to separate and recombine the content and style features of the input images, (aka the training dataset).
What does 'inputs' and 'outputs' mean?
Inputs usually means the 'prompts' that a user will provide in order for the algorithm to generate something. Inputs can also mean the training data that the algorithm was trained on, which is usually a very large collection of images and related text identifying each of those images, in order to function.
Outputs are the result of an input (a 'prompt'). In the case of text-to-image generators, the outputs are images. Note they are NOT photographs, despite being capable of being photo-realistic.
More resources for members-only below. If you're a member and logged in, you will have access to all the resources on this page. If you're not a member, you can join here.
Are there tools I can use to protect my work?
Yes, there are some mechanisms that you can use to protect your work, although if it has already been caught up in a scrape of the Internet, these tools will not retroactively protect those images.
Glaze and Nightshade are software solutions that are FREE and have been designed and built by a team at the University of Chicago, led by Professor Ben Zhao, and his team.
These two new and ‘free to use’ tools, are effective in protecting digital images online from being identified by machines, scraped and used in generative-AI programs. The motives of Ben and the team are purely altruistic, with the key aim of levelling the generative-AI playing field, so that visual artists regain their bargaining power.
Glaze – A cloaking tool
One solution is called Glaze, and is software that, “’cloaks’ images so that models incorrectly learn the unique features that define an artist’s style, thwarting subsequent efforts to generate artificial plagiarisms."
“Artists really need this tool; the emotional impact and financial impact of this technology on them is really quite real,” Zhao said. “We talked to teachers who were seeing students drop out of their class because they thought there was no hope for the industry, and professional artists who are seeing their style ripped off left and right.”
As a solution, the idea is to use AI against itself. ‘Style transfer’ algorithms, which are a close relative of generative art models, take an existing image – a portrait, still life, or landscape – and recreates it in a particular art style, such as cubism, watercolor, or in the style of well-known artists such as Rembrandt or Van Gogh, without changing the content.
Glaze works by running this ‘style transfer’ process on an original artwork, identifying the specific features that change when the image is transformed into another style. It then returns to the source and confuses those features, just enough to fool AI models trying to match the features, while leaving the original artwork almost unchanged to the naked eye.
Because these AI-models must constantly scrape websites for new data, an increase in cloaked images will eventually poison their ability to recreate an artist’s style, Zhao said. The researchers are working on a downloadable version of the software that will allow artists to cloak images in minutes on a computer before posting their work online.
Nightshade – A data poisoning tool
The other solution is called Nightshade and is software that is labelled as a data-poisoning tool, which “messes up training data in ways that could cause serious damage to image-generating AI models.”
The software enables visual artists to add invisible changes to the pixels in their artworks before they upload it online so that if the works are scraped into an AI training set, it can cause the resulting model to break in chaotic and unpredictable ways.
The Nightshade tool is intended as a way to fight back against AI companies that use visual artists’ work to train their models without creators’ permission. Using it to “poison” this training data could damage future iterations of image-generating AI models, which include generative-AI models like DALL-E, Midjourney, and Stable Diffusion, by rendering some of their outputs useless, so for example images of dogs become cats, cars become cows, and so on.
Ben Zhao, who has led the team developing Nightshade, said, “the hope is that it will help tip the power balance back from AI companies towards artists, by creating a powerful deterrent against disrespecting artists’ copyright and intellectual property.”
