Hybrid AI Workflows for Creative Production

Yeah, Jonathan Whistler, founder of Illumicloud, and I'll get into that a little bit later.

A Hybrid Advertising Demo

First, I want to show a demo of advertising that was what we call hybrid. So it's a creation between what you would call traditional, like we have a real model in there, and then using traditional tools and AI. So it's a hybrid model of AI and traditional workflows.

Let's see.

At the beginning, there was an idea. idea, to create a spot that would be both a demonstration of technology and a piece of art.

From the initial ideas and concepts, we moved on to casting a real model. We photographed her and then transferred her into AI, where we digitally reconstructed her and were able to place her freely within our cinematic world.

The script was turned into a storyboard, and the individual frames were further developed into cinematic concepts. Some of them were also converted into 3D using AI, allowing us to explore camera angles, prepare additional cuts, and design new shots.

We also used AI to direct the performance, movement, and facial expressions. These can be driven by references of a real actress, or created purely through prompts. The resulting shots can also include masks and depth information, making them ready for further traditional post -production. production.

Finally, the entire spot was edited, sound designed, and graded using a traditional post -production workflow. The music and voiceover were also created with AI, and the final film was finished in 4K.

Making Cloud Computing Invisible

Yeah, so we didn't create that. Actually, I'm the founder of Illumicloud, and then we're on a mission to empower innovators.

So any AI you see or anything you run needs to run on computers somewhere. So what we think is that cloud should be invisible. It's like a light switch. You flick the light switch and the lights come on. You don't have to be an electrician to get the lights on.

We feel the same for computing, that you shouldn't have to have a certification from AWS. You should just run your application and then the cloud works. We also think it should be affordable.

We don't believe that this cloud shortage or this compute shortage is a real thing. there's actually an abundance of compute out there. So there's 300 million devices in people's homes, GPUs, and there's billions of processors sitting in Linux processors sitting in devices. So there's not really a shortage. There's just not a clever way yet to architect this.

AI as a Tool for Human Creativity

So before we get into walking through the workflows of this commercial, which we didn't do, this is what our customer did they built that commercial or something like that but I just wanted to get a little philosophical about AI and I think this is the right venue for that

you know I was in California this summer and I saw the speaker she wrote a book that was called robot proof and then the theory of this book is that it's not a people versus the machine AI versus humans but actually how we're going to have superpowers is if we combine the power of AI with human creativity and human ingenuity so that's kind of our

philosophy and the things we build is we don't want to replace any humans what we want to do is empower them to create a better future and in the context of the the workflow I'm going to go go through is what they're doing is it's custom AI so they're using models and then fine -tuning them based on their own assets on their own data and that's where we see the unique value being

created so anyone can get a prompt and type it into open AI or Gemini but if you're using your own data in your own creativity that's how you create unique value in the this world of AI good so how do you do that just a quick question

anyone use imaging diffusion models in their work okay good anyone using comfy UI which is an open source workflow tool for creating commercials and other AI also architecture they have architecture workflows good so I'll go through this at a high level then

The AI-Assisted Commercial Workflow

Concept Development and Previsualization

basically you saw in the ad right so first they did was a concept which AI is really good at so they said okay we want to have a perfume and a woman walking through a warehouse with beautiful flowers and butterflies so they could test those concepts and what they call pre -vis pre -visibility

so that actually makes the creativity much stronger because before I used to work on movie sets where you have to lay out the storyboards. Someone would draw those or create those in a computer. And then, you know, it would take hours, days. And then you have a concept, you give it to the client. They're like, no, this is terrible or change it.

Now you could have like three or four iterations in one day as opposed to one. So you make this pre -visibility much better, much more quality and align with your vision in a much cheaper, faster way. So that's the first step.

Storyboarding and Client Feedback

And then the other thing is storyboarding. So you can take a script after you have that, put it into Claude and say hey Claude give me the shots on this so it'll break it into scenes and give you the script and then you can play around with that

and then you can go one step further in a Gemini or another tool it'll actually create the storyboards so you'll get some look they look like hand -drawn if that's your instructions like this is what the scenes are gonna look like so actually before you shoot one shot you have an idea of what the commercial will

look like you can give it to the clients you can get feedback you can adjust it and it's way more faster and efficient yes you have to pay some tokens but there are ways to

Training Models for Brand Consistency

make that more efficient as well and then what you need to do and what they did is they took they hired a model uh they took images of her you saw the makeup they already had the theme with the dress etc and then you what you want to do is you take at least 15 to 20 images and they use that to train the model with laura right so then that gives you a creative context and then

then you tell the model to run with this creative context and when you have that it's much more consistent because if you try to do that in an AI model now you'll get different faces you'll get different angles so it really doesn't really look like the same person so this is a good workflow

to put like 20 30 images it can be a product it can be a person use that to train the model and then your model is much more consistent from all different angles then what they do is they any

Any gamers in the house?

Using 3D Engines for Realistic Scenes

Good, so then what they do is they actually import these images into a gaming engine, like a Unreal Engine or Blender as an open source model as well. And then they can actually look at the scenes. You get 3D angles.

And what these gaming models understands are the 3D physics of how people work, how balls roll, you know, it understands the real world better than diffusion models do. do, because the Fusion models are like, oh, here's a bunch of similar images. Let me give you something I think you want.

But they put it into a 3D, basically gaming architecture models. And then they use that to look at camera angles, shapes, et cetera. So then the images are much better, much more realistic.

And then they direct the performance. Like you saw in the video, they can say, make her smile, make her run, make her jump. And then you can give those instructions. and based on the 3D models, like the gaming models, Unreal Engine, then it looks much more realistic and then it fits their brand.

So there's another good one where they showed some headsets or a microphone and you want 3D images of your product and then it looks really good and consistent from every angle.

Finishing, Sound, and 4K Delivery

And then of course, you take that and you drop it in, you can add sound, you can add voiceover, you can adjust it. You say, okay, I want a female voice from South Africa. I want a male voice from England. so you can play around with different scenarios all with ai and it's all relatively quick but

that's where the laura comes in you want to train it so it understands your brand it understands the creative effect you're going for and then in the end you go back to like traditional editing software like a da vinci so you jump from uh gen ai to like traditional models 3d models then you render then you jump back to like a traditional workflow for uh media and entertainment and

And then the final step is you upscale it. So then it's gotta be in 4K, so you can see it on a screen. It could be a movie screen, television screen, or the screen like we just used here. So that's the workflow.

And that goes back to the cyborg I mentioned before, where you say, okay, it's a combination. So it's not AI replacing humans. It's really humans using their existing tools, real humans using real physics, and then combining those together in a cyborg. and that's where you get real creativity and i'll argue that it's even better than

Tools and Resources for Creators

One-Click Cloud Workflows

just pure ai or non -ai creativity good so where can you learn more like i said we're a cloud platform so this is the service we provide for our customers we provide what we call a comfy one -click comfy so you save your environments so for example one of these commercials will have 30 different models you have voice you have one for light etc you have different lores that you've trained them on you'll have different assets so we save all those so the artists can just click and their whole environment is uploaded into cloud and they can start creating and that requires a lot more computing power than just a local device so we call it one click comfy and then you're up and running as an artist you can create these things with high power compute and then when you're done you turn it off so you just only pay for what you use um yeah and then if you want to

Learning Resources for AI Creative Work

to learn more in creative, even if you're just creating images for your website or something very simple, a little video, which actually in media and on LinkedIn, if you create like a five second video, it's going to get way more engagement than a, like a still image.

So I recommend looking at Higgs field. They have some, there's a Higgs field Academy on YouTube, and it'll walk you through all this stuff. A lot of it's open source, um, learn a little bit more. They actually have of skills you can download from Claude. So it'll help you prompt things with the right lighting, with the right angles. So there's a lot you can get already from the Higgs Field Academy.

And then if you really wanna go deep, go Comfy. It's a open source gen AI tool for workflow management. It's a little more complex, but it's very rich in what you're trying to build. And they have some pre -built workflows. So I know there's one for architecture, but also they have some that says, okay, if you wanna create a 10 second commercial. so if you want to learn more play around with it I highly recommend it and yeah

these are all mine and that's it any questions