Not sure on DALL-E, but I think many image generators use an image classifier as part of their process. The classifier uses labels for its training, but the image AI doesn’t have direct intervention.
I think you take the classifier like CLIP and run it on an image to tell you it is likely “car” and “ red”. Then add noise to the image. Then provide the noisy image and classifications to the image AI. So it will try to find “red” and “car” and add more of it to the details. Then the resulting image is run through CLIP and the classifications compared to the original classifications to define the loss function.
Not sure on DALL-E, but I think many image generators use an image classifier as part of their process. The classifier uses labels for its training, but the image AI doesn’t have direct intervention.
I think you take the classifier like CLIP and run it on an image to tell you it is likely “car” and “ red”. Then add noise to the image. Then provide the noisy image and classifications to the image AI. So it will try to find “red” and “car” and add more of it to the details. Then the resulting image is run through CLIP and the classifications compared to the original classifications to define the loss function.