FaceSwap

FaceSwap,Face Swap video AI face swapping tool software

Official FaceSwap website, Face Swap video AI face swap tool software

What is FaceSwap?

FaceSwap is one of the most popular AI face swap tools on GitHub, requiring NVIDIA GPU CUDA computing power of 3.5 or above; my RTX3080 has a computing power of 8.6. Swapface AI tool is a lightweight, ultra-realistic, and real-time video face swap technology.

Official FaceSwap website:https://faceswap.dev/

Download package address:https://github.com/deepfakes/faceswap/releases

How is FaceSwap?

It uses advanced AI and computer vision technology to generate realistic face replacement effects for your videos in seconds. No complex parameter settings required; just upload videos and photos, click a button, and Swapface AI tool will automatically detect facial features and map them to the video.
Currently supports Windows system only; updates for macOS will be provided in due course. Download packages include tutorials; interested users can download and view them at their convenience.

The face replacement effects generated by Swapface are extremely realistic and natural. The AI and computer vision technology it uses can accurately detect facial features and map them, creating highly precise facial transformation effects, bringing a cinematic visual experience to your work.

FaceSwap can create a 3D face model from a single photo and then use this 3D model to approximate the face in front of a real camera. As a result, the 3D face moves or changes with the experimenter's movement and expression.

The core process is to form a 3D face model from a photo. It mainly uses the 3D face model of Candide3, the 2D facial feature point detection of DLib library, and some drawing methods of OpenGL.

FaceSwap is one of the most popular open-source AI face swap tools on GitHub, requiring NVIDIA GPU CUDA computing power of 3.5 or above; my RTX3080 has a computing power of 8.6.

Download installation package

https://github.com/deepfakes/faceswap/releases

$ wget https://github.com/deepfakes/faceswap/releases/download/v2.0.0/faceswap_setup_x64.exe

This installation program handles everything; if it fails, try several times, as it may be due to network issues.

However, it can also be manually installed step by step:

Install Git
Install Miniconda3
Create a Python 3.10.6 virtual environment
Install faceswap source code and install

$ git clone https://github.com/deepfakes/faceswap.git $ cd faceswap/ $ conda activate faceswap $ pip3 install -r requirements/requirements_cpu.txt $ pip3 install -r requirements/requirements_amd.txt $ pip3 install -r requirements/requirements_nvidia.txt $ python faceswap.py -h $ python faceswap.py gui # GUI mode

Usage instructions

It works in roughly four steps:

Collect materials: including images and videos
Extraction: Extract faces from the materials as training sets
Training: Train based on the extracted faces
Conversion: Complete the replacement based on the training model results

It can be operated via command line or GUI, with GUI recommended.

Extraction

Prepare several directories:

src/video1.mp4: Original video to be face swapped
src-extract/: Extracted original face image files
dst/video2.mp4: Replacement video for face swapping
dst-extract/: Replacement face image files
model/: Directory to save trained models
result/: Directory to save conversion results

$ python faceswap.py extract -i ./src/video1.mp4 -o ./src-extract $ python faceswap.py extract -i ./dst/video1.mp4 -o ./dst-extract

The results after extraction will not only have a large number of face image files exported per frame but also a.fsaalignment file, which records the frame position information of each face in the original video.

But the above is just a preliminary extraction of faces, and manual adjustment is required: if there are multiple faces in the original video but only one face is to be replaced, then:

In the GUI, select [Tools] > [Sort] to sort the faces, as there may be many image files, and keeping around 1000 to 10,000 image files is better.
In the GUI, select [Tools] > [Alignment] to automatically clean up the alignment files based on the cleaned-up face image files, as we will be converting the original video based on the alignment files.

After this processing, the face image file directory and alignment file should contain the information we actually intend to use for training and replacement, but if better replacement effects are desired, manual intervention is still needed:

In the GUI, select [Tools] > [Manual] to fine-tune facial information frame by frame based on the original video and alignment files, adjusting for areas where automatic face recognition is not accurate enough, and save the alignment files.
Before deleting the generated extracted face image files, re-extract the face image files based on the alignment files again, which will be the final training set image files.

Perform the same operation on both the original video and the target replacement video to achieve satisfactory face image files for training.

Training

$ python faceswap.py train -A ./src-extract -B ./dst-extract -m ./model/

This process may take a long time; in the GUI, you can preview the replacement effects to assess the results. When the loss is less than 0.02, the effect is relatively good, and training can be stopped and restarted at any time; this process can be repeated.

Conversion

$ python faceswap.py convert -i ./src/video1.mp4 -m ./model/ -o ./converted/

Since the default alignment file of the original video is in the same directory, it will automatically search for this alignment file and replace the original video based on the trained model results to obtain the converted video.

Video to Image and Composition (ffmpeg):

$ python tools.py effmpeg -h $ ffmpeg -i src/video.mp4 src-extract/video-frame-%d.png $ ffmpeg -i video-frame-%0d.png -c:v libx264 -vf "fps=25,format=yuv420p" src.mp4

When performing conversion in the GUI, selecteffmpegto automatically complete video composition.

Reposted from here:

https://mp.weixin.qq.com/s/Kh4roFAq5uOAw4VJRceDLQ