AI use case: The Second Coming (of AI) by Yeats
I can't help myself but want to play with these new toys.
Since my last substack article a couple of weeks ago a LOT of new advancements have broken out into the wild.
Specifically there are now officially many text-to-video models delivering very high quality 4 second, 5 second, 8 second, even 10 second generations with remarkable consistency. Still nothing usable ‘as is’ straight out of the models for high quality film and television work (nothing I’ve seen would pass quality control) but it may be possible to post-process and refine the footage and sort of get it there after the fact using tools like Topaz or some more exotic ComfyUI workflows but probably not. Not yet anyway.
However, for social media use and proof of concept and inspiration these tools are beyond awesome, and it takes absolutely zero imagination to visualize these working at a pro level (which may still be 2-3 years away, that last 5% in quality to get to 100% is always so tough to achieve).
It is SO MUCH FUN to spin up Kling, or Luma, or Runway Gen3 and start commissioning shots. As they start to pile up in the browser it’s super fun to discover what the elves have been building.
Most fascinating of all to me so far has been the amazing voice performances one can coax out of Elevenlabs. Now with Hedra those auditory performances can be transformed into visual ones. Can AI actually act now? Well, sort of.
The process used?
voice performance first, work to coax the performance you want out of a voice generator
generate a seed image to animate, who is our interlocutor? I used midjourney.
using Hedra, bring the voice performance to life.
edit it all together and generate a few additional shots for fun in Kling (Runway Gen3 insists on a 16:9 aspect ratio, but Kling can do square as well as vertical). Kling is also possibly higher quality than Gen3 AND appears to suffer from less censorship.
Don’t forget some ambient music. Created in udio in this case.
It’s amazing how these tools come together to allow for new possibilities.
Sesational. !!! Mind if I use it ?
Wow! So cool. Thanks.