AI is a Tool for Artists

Perhaps to the chagrin of Drake and The Weekend, artificial intelligence continues to adapt to its environment. In the VFX world, however, AI has not replaced any of our tools yet — but it is changing how we start the work on a new project.

One reason AI is not much help beyond that initial step is the potential for copyright issues. All of the big image diffusion companies — DALL-E, Midjourney, Stable Diffusion — train their systems on copyright-protected content. That creates a sort of intellectual-property-based Sword of Damocles hanging over every image these platforms produce. Not only that, the gray areas of intellectual property law are numerous — and gray areas in particular often require astronomical legal fees to settle or litigate.  

In contrast to the big three, Adobe has trained its diffusion model, Firefly, on only the stock footage that the Adobe company owns, licenses, or is otherwise permitted to use. This system is not yet perfect, as artists are allegedly not being paid for all of the uses they’ve been racking up. Adobe is currently working on ways to compensate artists/photographers for using their imagery, although without the artist's opinion.  

The other reason AI is not very useful after that initial stage is because it can’t really produce acceptable video yet. We're in the business of creating motion images, and the best looking visuals AI can do, currently, is a still image. The “holy grail” for VFX artists would be a means to convert AI imagery directly into 3-D models. Then the client would be able to essentially sign off on the look of an entire project much faster, and we as artists would be able to push the envelope on what’s possible.

When people say “AI is a tool,” I like to think of actual tools. Imagine the great impact the hammer must have had on civilization. We are in the middle of a similarly shifting period of time, and I think most artists are already excelling at the new tech.  

So while our tools have not been replaced, they have been dramatically affected by generative AI — along with our workflow. We’re now able to quickly explore concept images with directors and producers, even though we won’t use the imagery directly. At this stage, AI is best leveraged as a brainstorming tool that’s available to us for whatever uses we can dream up.

Originally published: 10/04/2023