Build an AI video generation tool yourself using this Generative AI code

Reading Time: 3 minutes

Hi, Welcome to Build it yourself.

In this tutorial, we will build an AI video generation tool.

With this tool, you can create talking head videos in which AI takes the audio and lip-syncs it with the video. Look at this example video.

Requirements:

Disclaimer: This tool creates synthetic media. So, please use it judiciously.

• A base video – shoot yourself for 10-15 seconds without speaking anything (.mp4 format)

• Take the voice you want to use (.wav format)

Note, don’t keep any space in your file name. For ex – if the file name is “my audio.wav”, then change it to “my-audio.wav”

If your video is not in mp4 format, you can use any free tool available on the internet.

AI video generation tool

Once you have both items ready, you need to go to this video retalking – https://github.com/tayaln/AI-video-creation-/blob/main/quick_demo.ipynb

Colab is an online notebook where you can run small code in the browser and see the output.

Google Colab also provides a Tesla T4 GPU for processing in its free plan, so it’s very easy to run AI models with these notebooks. 

Let’s start

Step 1 – Installation

In this step, you will download and install all the dependencies that are required to run the AI model. 

Click on the play-like icon. It will take around 12 minutes to install all the requirements.

Step 2 – Add the Pre-trained AI model.

AI is all about models. We have a model to do a specific task. For example – For text to speech we use a model. 

Here we will use a model that would sync the audio to a face in a video.

In this step, we will download the pre-trained model. It will take around 40 seconds to do so.

Step 3 – Upload your video and audio

Check the left sidebar and you will see a folder option, click on it.

It will show 2 folders – simple_data and video-retalking. Click on the arrow to see sub-folders

Click on examples. It further has 2 subfolders – audio and face.

Click on audio and you will see a couple of existing audio files. To add your file, see on the right side, there are 3 dots. Click on them and the first option is Upload.

Upload the file from your computer. It will take a few minutes depending on the size of the file. I suggest you keep the duration to 15 seconds

Similarly, click on face and you will see a few existing video files. To add your file, see on the right side, there are 3 dots. Click on them and the first option is Upload.

Upload the file from your computer. It will take slightly longer than uploading an audio file. 

Step 4 – Select your files

Once you have uploaded both files, it’s time to check them. Run the next code and it will show 2 dropdowns. One to choose video and the other to audio. 

You know what to do – select your files

Step 5 – Check your files

Once you have selected both files, we can visualize if they were uploaded successfully or not.

Run the cell and it will display both your video and audio files.

It’s time for real action. 

Step 6 – Action 

Now we will run our model and pass our input audio and video to it. The model would lip-sync the audio to the input video and give us an output. 

The process would take some time. So, if you want you can take a short tea/lunch break after running the cell.

Step 7 – Check the output

Once the model is done with its process, it will display the output. You can check it.

You can download it from the left-side “result” folder. The name of the file would be output.mp4

You just used a generative AI model to create a talking head video.

This was pretty cool. Right?  

Unleash your creativity and build your own AI video generation tool.

Why did I build a dedicated educational platform for senior IT professionals to learn AI?

Know more about the AI courses for senior IT professionals – https://www.aimletc.com/category/ai-ml-llm-online-coaching-senior-it-professionals-entrepreneurs/

Pricing AI courses for senior IT professionals – https://www.aimletc.com/ai-ml-etc-course-offerings-pricing/

If you have more questions, check out FAQs – https://www.aimletc.com/faqs-ai-courses-for-senior-it-professionals/

If you are also looking to upskill and learn AI + LLM in a very simple language, then I invite you to join this exciting journey.

Thanks for choosing to read it completely.