Wow, this is going to explode picture books and book covers.
Hiring an illustrator for a picture book costs a lot, as it should given it’s bespoke art.
Now publishers will have an editor type in page descriptions, curate the best and off they go. I can easily imagine a model improvement to remember the boy drawn or steampunk bear etc.
Book cover designers are in trouble too. A wizard with lighting in hands while mountain explodes behind him—this can generate multiple options.
It’s going to get really wild when A/B split testing is involved. As you mention regarding ads you’d give the system the power to make whatever images it wanted and then split test. Letting it write headlines would work too.
Perhaps a full animated movie down the line. There are already programs that fill in gaps for animation poses. Boy running across field chased by robot penguins—animated, eight seconds. And so on. At that point it’s like Pixar in a box. We’ll see an explosion of directors who work alone, typing descriptions, testing camera angles, altering scenes on the fly. Do that again but more violent. Do that again but with more blood splatter.
Animation in the style of Family Guy seems a natural first step there. Solid colours, less variation, not messing with light rippling etc.
There’s a service authors use of illustrated chapter breaks, a black and white dragon snoozing, roses around knives, that sort of thing. No need to hire an illustrator now.
Conversion of all fiction novels to graphic novel format. At first it’ll be laborious, typing in scene descriptions but graphic novel art is really expensive now. I can see a publisher hiring a freelancer to produce fifty graphic novels from existing titles.
With a bit of memory, so once I choose the image of each character I want, this is an amazing game changer for publishing.
Storyboarding requires no drawing skill now. Couple sprinting down dark alley chased by robots.
Game companies can use it to rapid prototype looks and styles. They can do all that background art by typing descriptions and saving the best.
We’re going to end up with people who are famous illustrator who can’t draw but have created amazing styles using this and then made books.
Thanks so much for this post. This is wild astonishing stuff. As an author who is about to throw large sums of money at cover design, it’s incredible to think a commercial version of this could do it for a fraction of the price.
edit: just going to add some more
App design that requires art. For example many multiple choice story apps that are costly to make due to art cost.
Split-tested covers designs for pretty much anything—books, music, albums, posters. Generate, ad campaign, test clicks. An ad business will be able to throw up a 1000 completely different variations in a day.
All catalogs/brochures that currently use stock art. While choosing stock art to make things works it also sucks and is annoying with the limited range. I’m imagining a stock art company could radically expand their selection to keep people buying from them. All those searches that people have typed in are now prompts.
Illustrating wikipedia. Many articles need images to demonstrate a point and rely on contributors making them. This could open up improvements in the volume of images and quality.
Graphic novels/comic books—writers who don’t need artists essentially. To start it will be describing single panels and manually adding speech text but that’s still faster and cheaper than hiring an artist. For publishers—why pick and choose what becomes a graphic novel when you can just make every title into a graphic novel.
Youtube/video interstitial art. No more stock photos.
Licensed characters (think Paw Patrol, Disney, Dreamworks) - creation of endless poses, scenes. No more waiting for Dreamworks to produce 64 pieces of black and white line art when it may be able to take the movie frames and create images from that.
Adaptations—the 24-page storybook of Finding Nemo. The 24-page storybook of Pinocchio. The picture book of Fast and The Furious.
Looking further ahead we might even see a drop-down option of existing comics, graphic novels but in a different art style. Reading the same Spiderman story but illustrated by someone else.
Character design—for games, licensing, children’s animation. This radically expands the volume of characters that can be designed, selected and then chosen for future scenes.
With some sort of “keep this style”, “save that character” method, it really would be possible to generate a 24-page picture book in an incredibly short amount of time.
Quite frankly, knowing how it works, I’d write a picture book of a kid going through different art styles in their adventure. Chasing their puppy through the art museum and the dog runs into a painting. First Van Gogh, then Da Vinci and so on. The kid changes appearance due to the model but that works for the story.
As a commercial produce, this system would be incredible. I expect we’ll see an explosion in the number of picture books, graphic novels, posters, art designs, etsy prints, downloadable files and so on. Publishers with huge backlists would be a prime customer.
Allow it to display info on a screen. Set up a simple poleroid camera that takes a phone every X seconds.
Ask the question, take physical photos of the screen remotely.
View the photos.
Large transmission of information in analog format.