Adobe catches up in the generative AI game with a home run
Since our last update on Artificial Intelligence as a useful tool for social media graphics, two fascinating upgrades have hit the market: Adobe’s Generative Fill feature and Midjourney’s Zoom. Both are huge game changers and are worth a mention.
Ever since Midjourney was first released, there has been an ongoing discussion about the ethics behind it. As we’ve mentioned in a previous blog post, this is largely because of the type of model Midjourney and others are. Writing for Midjourney (since we’re talking about their feature), it is a “generative artificial intelligence program”, based on the principle of “Generative Adversarial Networks”. That means there are two neural networks working against each other in order to produce arguably original results. It’s the “arguably original” results that rest in the legal realm, and leads to a long-winded debate on an augmented reality version of the ship of Theseus between artists, graphic designers, and AI programmers.
Adobe and AI
It’s that above mentioned debate that probably led Adobe to be a little late to the generative party. With all these generative AI apps cropping up everywhere, it seemed that even an Android phone could outdo Adobe’s usefulness for the social content creator (and this author said so much myself). AI had long been a part of Adobe’s arsenal, using neural networks to assist with all kinds of functions, like background removal and content-aware fills, as well as helping Adobe Stock tag its inventory, but nothing with natural wording or truly “generative”.
In March 2023, Adobe announced the development of Firefly and its implementation in Photoshop. And let me tell you: Firefly is a game changer. It makes me (the author, not the official position of Smartsound Cloud) seriously regret abandoning Adobe half a year back for a cheaper alternative that I could afford as a blog writer.
Firefly flies in
Adobe developed Firefly in partnership with OpenAI, initially to help users complete tasks on a more natural language basis. But answering questions and chatting is not the coolest thing Firefly does. Rather it’s the Generative Fill that has stirred up the chat-o-sphere.
Generative Fill is a fill function that essentially allows you to fill in a gap with a simple text prompt. This is not just useful for putting random items into your pic, but also deleting objects and letting you or the AI imagine what was behind it. You can also make a vertical image into a horizontal, and have the AI fill in the sides rather than just have some awkward blurry stuff or be forced to cut out the top and bottom of the original.
There are just so many amazing things you can do with this that for content creators, it really does smash through a lot of limits. Lower-skilled designers now have a tool that can up their game, but those who already have a lot of skill… I can imagine this will set their art on fire (in a good way).
The issue with Generative Fill
The Generative Fill function runs into a lot of the same ethical quandaries as other generative image AI, like Midjourney. It was trained on previously created works, and one has to wonder just how much of that past work is used in a coherent manner that can be identified as part of that previous work. With Midjourney, I’ve come across some renderings that have watermarks, meaning it was such a close copy of the original piece it still had the watermark! (Let that be a warning to all designers/photographers/artists, use watermarks!)
Most of Firefly’s training materials come from Adobe Stock, which they own and define the license in no unclear way that they can use the pictures for whatever they want. Fair enough. But it also uses openly licensed content and public domain content where the license has expired. So supposedly the pool it’s coming from has already given the permission to use those images for any reason (or has no legal right to them). But that remains unclear, we’ll see.
Using Generative Fill for social media content
Right now, the feature is on the Beta phase and can be tried out for free here. All you have to do is paint on the object you want to remove. So for my example, I took their lighthouse pic and highlighted both the lighthouse and the rock. Now, either you can enter something in the prompt (I entered “steamboat”) or you can let the AI go into automate. You’ll sometimes get good results (for smaller object replaces) and sometimes it’s borderline ridiculous (like any generative AI). Here’s the original picture provided by Adobe as an example.
I decided to see how it handles removing the entire rock and lighthouse. I put in the prompt: “steamboat” and got this:
Good, but what’s with this funky split bow? And is it a shadow behind the ship? After reviewing a few more pics, I came to realize that the AI is attempting to fill in the entire area that I masked, and not just a reasonable area. My tip then is to let the AI do its thing for the entire area. Once you find something mostly acceptable, hit up a smaller area with the general shape of your request. And following those instructions, here is our generated steamboat:
Not perfect, and not really a steamboat, but a lot better.
Midjourney’s Zoom release
Midjourney has again rocked the world of AI with its Zoom feature. Midjourney already gave you four images based on your prompt (we’ve a blog here with more on that). But one problem I continuously ran into was the framing. Most of the time, the subject was way too close to the “camera”. Apparently I wasn’t alone with this issue, as many other people complained, which led to Midjourney’s handlers to develop this new function.
Zoom copies the zooming out function of a camera’s zoom lens. Note, you can’t zoom in with this, not yet at least. The benefits can be to either change the relative size of the subject, or to turn the design into a square by filling in the top or bottom (for example if you want the image for Instagram).
Using Zoom for social media content
The biggest advantage for Zoom is that if you want to create Instagram content, it makes it much easier to do so. Because of the tendency for Midjourney to give closeups, I’ve found it more useful to set the aspect ratio to something like 9:16 so that I can have a full banner, and then crop it down from there. But now we have the option to “crop up”, which really expands the possibilities.
To do this is pretty easy. You first have to Upscale your preferred picture. With the Upscale result, they’ve added several buttons. “Vary (Strong)”, “Vary (Subtle)”, “Zoom Out 2x”, “Zoom Out 1.5x”, “Custom Zoom” and “Make Square”. With my example (which is the cover for this blog) I started at an aspect ratio of 16:9. I’ll show you the “Make Square” result below.
Other interesting new options are the two “Vary” ones, which can really make for even more interesting results, a good kind of “reset” for when Midjourney isn’t quite giving you what you want. The Zoom gives you for results, as per usual, and you continue your refinements from there.
What the future brings
Zoom and Generative Fill are both exciting new features for the AI graphics and the social media content community. They’re both set to help graphic designers and amateurs alike in profound ways, and it will be exciting to see what people start coming up with and how they use it. It’s even more exciting to think what other opportunities AI will have and how else it will help hasten our workflows or challenge our current working models.
