Video ID: qZM9pHKjlBE
YouTube URL: https://www.youtube.com/watch?v=qZM9pHKjlBE
Added At: 13-06-25 21:18:35
Processed: No
Sentiment: Positive
Categories: Tech, Education
Tags: AI, Machine Learning, Video Generation, Runway Gen 3, LTX Studio
Summary
This video showcases the capabilities of AI video tools, specifically Runway Gen 3 and LTX Studio. It demonstrates how these tools can be used to create title sequences, transforming between scenes, and generating entire short films. The video also highlights the features of each tool and provides examples of their uses.
Transcript
I've been experimenting with AI video tools and covering them on this channel for a long time and right now is the most exciting and fun time in AI video yet Runway and Luma Labs have been in the spotlight and I'll cover some immediate real world use cases for them but my personal favorite tool is actually not one of those making the headlines I'll get to that a little later plus some amazing lip syncing tools and some open source models there's been a lot of advancements I've got to start with Runway gen 3 since it's been dominating my timeline it is the best textto video model available to use right now something that's particularly useful with Gen 3 is text like these title sequences it is amazing at this I'll show a few of my favorite examples I've [Music] seen this is such a perfect title sequence for a cooking channel this one also showcases how good it is at fluid simulation the physics in a lot of these is really good adding some sound design would take these to the next level for sure and be production ready title sequences so I'll try one for future pedia I'm logged in already and here's a whole Suite of image and video tools that Runway offers but for Gen 3 I'll just click right here then type in a prompt a title screen with Dynamic movement the scene starts with intricate neon circuitry patterns that light up and move on a dark background suddenly the circuits converge and form the word future pedia with a glowing pulsating effect then I can choose between 5 seconds and 10 seconds I'll lower it to five then click generate that's all there is to it right now this is how that came back and that looks amazing since futurepedia is a longer word I have gotten some misspellings but it gets it right A lot of the time this prompt structure has worked really well so I used one of the sample prompts in the Gen 3 prompting guide and modified it I modified it a lot really just kept the basic structure so I'd recommend trying to use the prompt structure they give here when you're starting out camera movement establishing scene then additional details hopefully that will help you cut down on rolls they do also have a lot of solid keywords in this guide to help give you some inspiration another thing it's really good at is transforming between scenes this is another prompt you can modify for yourself that comes back with great results pretty [Music] consistently I will copy that and paste it in here I'll change this to a wormhole and into an alien civilization [Music] that's not bad with some rerolls and modifying The Prompt I'm sure I could get a better generation here's a couple other examples using this same prompt [Music] [Music] [Music] idea now I've also had a lot of Misses with runaway 2 that's the biggest caveat here the amazing results you see posted are definitely cherry-picked that's the case with all AI video models including Sora but Runway can do some amazing things especially if you have a lot of credit here's a few more of my favorite Runway examples this was really hard to narrow down [Music] stop [Music] the best image to video right now is dream machine from Luma Labs you can also do text to video which gets some good results but where Luma shines is image to video and even better than that is with key frames I'll show a couple straight image to video examples first it's easy to use upload an image then add a prompt volcano erupting contained within a drinking glass surrounded by peaceful Tranquility this was the original image then here's the result and that looks perfect I'll show a few more straight image to video examples then we'll move on to key frames these are all within one or two tries it does weird things on occasion like this astronaut grows in extra finger or here's a weirder one or how about a much weirder one most of the time they do look good without having to do too many rolls the next step is adding an ending frame and you can start doing some really cool stuff you upload a starting frame then also upload an ending frame now add a prompt for it to use to create the whole clip in between those frames this works most consistently between things that are logical to transition between if they're not then try to be extra descriptive of what you want to happen in between it works great for transformation shots that are in the same or a similar scene but you can also do more out there [Music] transitions this one looks like it could legitimately be a Pixar movie you can extend that out by just taking your ending frame and using that as the starting frame for a new generation then adding a new ending frame you can do really long sequences this way there's been these long mashups of memes that people have been posting which are fine I am definitely more interested in the creative stuff this with this girl and the Wolf is one of the coolest I've seen using that it's amazing to keep that all consistent and there's a ton of possibilities for this you can do this for free but the generations take a long time like multiple hours when I tried before signing up paying will get you to the front of the line and a lot more Generations there's one more out of China that's similar in quality to these two called cling however there is a huge weight list for it and the process to sign up is difficult without a Chinese phone number so I'm going to cover that one at the end moving on to LTX Studio they they have the most control out of any of these platforms and the fastest speed they can build out an entire short film in a few minutes how it works is you can start from scratch input an entire script or start from a simple prompt I'll go with the prompt option for this example I will paste this in it's about a futuristic city controlled by AI entities and a hacker that can communicate with them he joins a resistance they battle all that good stuff hit next and it starts working it will show the basic story and the cast then you can select a style so this is Cinematic but let's say I want an anime I can direct it more with words as well I'll add cyberpunk there's all sorts of other styles here to try I like comic book it will use these as consistent characters throughout the story which is amazing because that's usually a struggle and I can change anything about them that I want I think this will be better without a last name I could change the essence appearance the clothes I can test out different voices if I want as an organizer I start from where the world is as it is not as I would like it to be you can even face swap these with a face you upload I want to change the title to how about Neon Flux see how many people know that reference to the anime not the movie they made from it this looks good here so I'll click Start and just from that it builds out an entire short film with fully editable scenes this whole thing was generated in less time than it took either of the others to generate a single clip and I want to show the style reference feature again now that this is open because it's even cooler to do it in here I'll try uploading my own style reference so this is a mid-journey image when I upload that style every shot across every scene regenerates in the new style that's super helpful and doing it here will give you a much better idea of how it will look and that looks awesome I'm actually going to run with that style now these are just static images currently I could generate them all into videos at once or open each one to customize it that gives you the most flexibility so I'll do that I can regenerate the image or upload my own even generate or remove objects from the image I'll just get rid of this little blob [Music] here I can choose from four types of motion and adjust how much of it most of the options are pretty straightforward but this one is really amazing when you switch to custom then orbit then you can have total control over the start and end of the shot and it will generate the motion in [Music] between you can also adjust how long the clip is I want this to be like a movie trailer so 3 seconds will be plenty you can also add music sound effects it generates a voice over all of that can be customized as well if it hasn't hit home yet everything can be customized I'll actually change the voiceover a little because this ended up telling the entire story I didn't mention in the prompt that it was for a trailer I don't want to spoil the ending the track it used is awesome so I'll keep it that's a lot of the features probably enough for this showcase so I'm going to go through and generate each scene how I want it then come back all right finished all the scenes it's really nice that you can export all the files so I can bring it into premiere for more fine grained editing the other really cool option is to export it as a pitch deck I'll do that as well first here's the pitch deck it has the story all of the characters and their backgrounds the mood board and color palette then the full story board people have already made some amazing things this way where they storyboarded a short film with LTX then actually filmed and produced the entire thing they posted this one called winged on X where this turns into this like human head with bird wings it's pretty good you should go watch it then with this premere option it gives me the project file that I can edit further I already opened that and synced things up to the music a little more and made just a couple other simple edits here is the final result in the neon lit City I discovered a secret power the ability to communicate with AI Spirits joining the resistance I became a spirit Guardian fighting alongside comrades unique Tech Powers we Face a rogue AI threatening our City's balance blending ancient wisdom with cuttingedge technology as the battle intensifies the fate of our world hangs in the balance I've been having a ton of fun with LTX studio and side note I did actually meet some people from the team at the Imagine AI conference a while back at one point they were just giving a little demo at their booth and like everyone crowded around to watch they really stole the show make sure to sign up for the weit list they're letting lots of people in the link is in the description and thank you to LTX for sponsoring this video the platform I've been having the most fun with out of all of these is Korea and with this one you can do quite a bit for free it's a lot different than the other ones we've covered it's more for abstract stuff not so much for realism so it focuses more on these like trippy morphing type animations which I like a lot personally I like to use AI to create things that would be impossible to create any other way to open up entirely new creative Avenues rather than replace EX in ones so I have a ton of fun in Kaa I'll go up to generate then video they also have a creative upscaler that's really useful I'll show that one next these are the three images I want to use they're these translucent kind of bioluminescent flower jellyfish and dragon I think they look cool together so I'll click add key frame select the flower then I'll add another key frame for the jellyfish and another for the dragon you can make these longer or shorter I'll lengthen it a little bit then you can add a text prompt I just want these to morph into each other so I don't need any longer prompt I'll just say what they are then drag the length of the prompt and that's where the transitions will start at least that's where they're supposed to it's not perfect there's a couple other settings for the aspect ratio motion intensity I like it around 60 usually then you can switch the looping on or off it defaults to on where the end will morph back into the first frames so it can just endlessly Loop then you have four styles to choose from I'll start with film then click generate video and I'll generate one at each of those for comparison in a second this will do a first pass at generating then give a preview at a low quality that'll give you a good idea what it will end up looking like if it's not going in the direction you'd like you can go to where it says enhancing and select click to cancel that will save you some generation time now I actually think it's moving away from the jellyfish too fast so I will cancel I'll add another key frame and select the jellyfish again so it hangs out on that for longer then we'll generate again that cancel option is especially helpful if you're on the free plan since your Generations are more limited I am on the basic plan which gets plenty of time for me in about a minute it's all done and here's what I [Music] got I think that looks really cool now I'll generate with render animation and experimental so you can see the difference [Music] I think these are amazing really fun and unique here's another quick example I did with these [Music] images I am working on some longer stuff with tracks from suo but they're not finished yet so here's one more weird one I did I love this style I know it's not for everyone but it's really great for something like a music video or an opening sequence but these are a couple of my favorite examples I found on x [Music] [Music] oh [Music] [Music] now I also want to demo their video upscaler this is not like a traditional upscaler where it resembles the original video as much as possible just at a higher resolution it does a creative upscale similar to how magnific works but for video it stays really close but kind of reimagines everything with AI so I'll use a clip from LTX video I made this one has a face that was just completely warped we'll see if it can fix that I don't need it upscaled that much I'll leave that at 1.5 frame rate 30 FPS write out a prompt of what it thinks is in the video that looks good so I'll leave the strength and resemblance at the defaults first and start with cinematic turn Loop off and enhance this took around 30 seconds and here's the result you can see that there's a face on here now that's pretty amazing it was able to fix that I ran one at each of the different presets so you can see the difference cinematic and render are fairly similar then animated makes it more of an anime style I'll show that with a few clips from that LTX video [Music] I hope to see them expand that into more Styles because it is very useful overall I think Kaa is a ton of fun and there's some additional options to play around with in here as well lip syncing has made some big improvements recently there's tons of impressive demos that have come out that we don't have access to use but there's two platforms I want to show that we do have access to one that's completely free and one that's free for five uses per day it's hedra and live portrait hedra has some of the most expressive talking avatars I've seen it's pretty easy to use so either generate the audio or upload some of your own I'll use the classic Fight Club line now I'll upload a character you can also generate one here if you need now I just generate the video it works pretty quickly and here's what I got the first rule of Fight Club is you do not talk about Fight Club second rule of Fight Club is you do not talk about Fight Club that looks great super expressive let's do another one but with a non-human character this time I'll use the line from Taken what I do have are a very particular set of skills skills I have acquired over a very long career skills that make me a nightmare for people like you so that one looks great but it can definitely struggle with non-human characters and the less human they look the harder it is a dog just didn't work at all or even some of those gnomes especially the ones with a beard it was having a hard time with you do not talk about Fight Club Runway also has a lips syncing feature that they implemented it works really well very high quality but it's much less expressive the first rule of Fight Club is you do not talk about Fight Club second rule of Fight Club is you do not talk about Fight Club live anime is also amazing but takes a different route you upload a reference video and it will map that onto the Avatar this allows you to have more control over the expressiveness it's on hugging face so you can use it for free upload a source portrait I'll give my face a try then upload a driving video it works best with a straight on shot like the ones they have in these examples I'll just use one of those here's a pretty classic one although I won't be able to use the song that goes along with it on YouTube now click animate and in under 30 seconds here's the result that worked pretty perfectly I'll show an example that didn't work this is a CLI clip from Breaking Bad it shows his face the whole time but he does move a bit so with that it's just not able to map that onto my face at all so there's limitations here but you can still do some pretty great stuff with it things you would never be able to get through other methods currently and really quick here I want to shout out the open source community and the tools and workflows they've been developing and pioneering those have really been the foundation of some of these other tools that will mimic or outright take those workflows and Implement them into a paid model the things you can get from Korea and some of these other video models are possible with open source tools most notably comfy UI and animate diff that method is significantly more complicated although it offers more customization and control the paid platforms tend to offer a nice UI and ease of use so far all of these examples are from James G not sure if that's how you say his last name he has so many amazing Creations on Instagram I'm only showing short clips from them they're all much longer and have music to them as well so I have been following his work for a while and it is some of my favorite this marble statue dance was one of his really early ones to go viral his Instagram is full of amazing stuff but I'll show some other people too this dancing spider is [Music] awesome these have all been from driving these animations using real footage but that's not the only way to use this at all this one by Fab dream going through the evolution of money was [Music] great or these weird frog animations from boltron I love these [Music] to learn more about how to do this I am not the person to be teaching it but if you want to do a deep dive to install and run it locally and learn how to do all of that I'll have a couple resources in the description cling is the platform that's as good or better than Runway and dream machine with text to video and image to video but the weight lless is huge and the process is more complicated if you don't have a Chinese phone number so I'm going to link to a thread by tile AI he walks through the process of downloading a couple apps and getting a burner number to use I was able to sign up for the wait list using my own number on their mobile app but then the new website doesn't have an option for a non-chinese number to sign in so it is probably best to use his method and fair warning I signed up for the wait list about 2 weeks ago it said there were around 500,000 people on it I still don't have access and now the wait list says it has around a million people on it so it is free to use right now but I imagine that will change by the time people that are just now signing up for the wait list get access so it may not be worth the hassle that's why I didn't want to spend all time going over that here follow that thread if you want to try but the quality coming out of cing is amazing [Music] we are finally to a point where you can use AI to make video that's usable in the real world outside of memes not usable for everything of course there's still plenty of limitations these tools have come a very long way and are a ton of fun right now if you want to keep up with everything in AI make sure to visit futurepedia find the best AI tools stay up to date with Innovations learn how to use it all with curated tutorials there's tons of new features on the site so check it out if you haven't for a while then subscribe so more videos like this show up on your homepage thank you so much for watching I'll see you in the next one