Text only | Text with Images

Cobra Forum

Plesk Panel => Web Application => Topic started by: mahesh on Jan 04, 2024, 02:00 AM

Title: AI Face Restoration using GFPGAN on Vultr Cloud GPU
Post by: mahesh on Jan 04, 2024, 02:00 AM
Introduction
Generative Facial Prior-Generative Adversarial Network (GFPGAN) is a blind face restoration model used to restore and improve images. The model works with multiple image types such as old images, low-resolution, noisy, blurry, and AI-generated images. Generative Facial Prior (GFP) handles the face restoration process through novel channel-split spatial feature transform layers. This enables a balance between realness and fidelity on generated images.

(https://pix.cobrasoft.org/images/2024/01/04/Copy-of-Add-a-heading.png)
This article explains how to perform AI Face Restoration using GFPGAN on a Vultr Cloud GPU server. You are to restore a single image from a base URL, generate, and export improved images to compare the AI face restoration result.

Prerequisites
Before you begin:

Code Select
  # su example_user
Set Up the Server Environment
In this section, set up the server environment and install the necessary packages to run the GFPGAN model as described below.

1.Create a new directory to store generated images

Code Select
$ mkdir -p /home/user/output_images
2.Install PyTorch

Code Select
$ pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu118
The above command installs PyTorch with pre-built CUDA 11.8 libraries. torch and torchvision offer a platform to develop and train deep learning models, along with tools and pre-trained models for Computer Vision tasks to speed up the process of building vision-based applications. To download the latest version, visit the PyTorch Documentation.

3.Install Jupyter Notebook

Code Select
$ pip3 install notebook
4.Allow the Jupyter Notebook port 8888 through the firewall

Code Select
$ sudo ufw allow 8888
By default, UFW is active on Vultr servers. The above command allows incoming connections to the Jupyter Notebook port 8888

5.Launch Jupyter Notebook

Code Select
$ jupyter notebook --ip=0.0.0.0
The above command starts Jupyter Notebook and listens for incoming connections on all server interfaces with the 0.0.0.0 directive

When successful, a random token displays in your command output as below:

Code Select
[I 2023-08-10 12:57:52.455 ServerApp] Jupyter Server 2.7.0 is running at:
 [I 2023-08-10 12:57:52.455 ServerApp] http://HOSTNAME:8888/tree?token=73631c92ba278d265aedeb3b199bd4d48e5ef5b2eed0ae06
 [I 2023-08-10 12:57:52.455 ServerApp]     http://127.0.0.1:8888/tree?token=73631c92ba278d265aedeb3b199bd4d48e5ef5b2eed0ae06
 [I 2023-08-10 12:57:52.455 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
 [W 2023-08-10 12:57:52.458 ServerApp] No web browser found: Error('could not locate runnable browser').
 [C 2023-08-10 12:57:52.458 ServerApp]

     To access the server, open this file in a browser:
         file:///home/user/.local/share/jupyter/runtime/jpserver-67384-open.html
     Or copy and paste one of these URLs:
         http://ControlNet-Test2:8888/tree?token=73631c92ba278d265aedeb3b199bd4d48e5ef5b2eed0ae06
         http://127.0.0.1:8888/tree?token=73631c92ba278d265aedeb3b199bd4d48e5ef5b2eed0ae06If you receive an error when starting Jupyter Notebook. End your SSH connection, and start a new session to activate Jupyter Notebook on your server.

6.Using a web browser of your choice, append the generated token to your Server IP to access Jupyter Notebook

Code Select
http://SERVER_IP_HERE:8888/tree?token=TOKEN_HERE
7.Within the Jupyter interface, click New, select Notebook from the dropdown list and create a new Python3 Kernel notebook.

(https://pix.cobrasoft.org/images/2024/01/04/Screenshot-2023-08-04-161424.png)
Install Model Dependencies
In this section, use Jupyter Notebook to install necessary GFPGAN model dependencies as described in the steps below.

1.In a new notebook cell, install the required model libraries

Code Select
!pip install realesrgan gfpgan basicsr
2.Click the run button on the main menu bar or press CTRL + ENTER to execute the above command

3.Import the model libraries

 
Code Select
from gfpgan import GFPGANer
 from basicsr.archs.rrdbnet_arch import RRDBNet
 from realesrgan import RealESRGANer
 from IPython.display import display, Image
 import requests
 import numpy as np
 import cv2
 import osBelow is what each imported library does:

4.Run the command to import all required model libraries

Set Up the Model
Initialise and set up the GFPGAN model in your Jupyter Notebook file as described in the steps below.

1.Define Checkpoints

 
Code Select
arch = 'clean'
 model_name = 'GFPGANv1.4'
 gfpgan_checkpoint = 'https://github.com/TencentARC/GFPGAN/releases/download/v1.3.4/GFPGANv1.4.pth'
 realersgan_checkpoint = 'https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth'The checkpoints defined in the above command are:

To get the latest version and update the above checkpoints, visit the official GFPGAN releases page.

2.Define RRDBNet for background enhancements

 
Code Select
rrdbnet = RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=23, num_grow_ch=32, scale=2)The following are the defined parameters:

[li]num_in_ch: Specifies the number of input channels for the neural network. num_in_ch=3 means that the network expects input images with three color channels, and represent the Red, Green, and Blue (RGB) channels[/li]
[li]num_out_ch: Specifies the number of neural network output channels. Just like num_in_ch, the num_out_ch=3 declaration represents that the network is to produce output images using RGB[/li]
[li]num_feat: Sets the number of feature channels or filters used in the model to determine the depth or complexity of the network[/li]
[li]num_block: Sets the number of RRDBs in the network. Increasing this value can make the network deeper and more powerful[/li]
[li]num_grow_ch: Affects how the network scales and transforms information[/li]
[li]scale: Sets the upscaling factor that the network can [/li]
[li]handle. scale=2 represents that the network can upscale the input images by a factor of 2[/li][/list]
3.Define the RealESRGANer object for background enhancements

Code Select
bg_upsampler = RealESRGANer(
     scale=2,
     model_path=realersgan_checkpoint,
     model=rrdbnet,
     tile=400,
     tile_pad=10,
     pre_pad=0,
     half=True
 )
The above command parameters handle the following functions:

4.Define the GFPGANer object for face restoration

 
Code Select
restorer = GFPGANer(
     model_path=gfpgan_checkpoint,
     upscale=2,
     arch=arch,
     channel_multiplier=2,
     bg_upsampler=bg_upsampler
 )
The above parameters function as below:

5.Create a function to retrieve the image

 
Code Select
def download_image(url):
     response = requests.get(url)
     image_array = np.frombuffer(response.content, dtype=np.uint8)
     image = cv2.imdecode(image_array, cv2.IMREAD_COLOR)
     return image
The above code declares a function named download_image() with one argument which hosts an image URL. The declarations inside the download_image() function include:


Run the Model
To run the GFPGAN model, import an image from a direct URL with a readable file extension such as .jpg, or .png. When ready, load the image as described in the following steps

1.Load the image using the download_image() function

Code Select
image = download_image('https://example.com/image.jpg')
For a quick addition to the model, upload your image to a public image hosting platform such as imgur and copy the direct image address to import into the function.

2.Load the image in the restorer function you defined earlier

Code Select
cropped_faces, restored_faces, restored_img = restorer.enhance(image)
The restorer function breaks the image and stores it in three data types as below:


View the Generated Output Image
1.Run the following function to load the processed image. Replace .jpg with your actual source image extension

Code Select
def cv2_imshow(image):
     _, encoded_image = cv2.imencode('.jpg', image)
     display(Image(data=encoded_image.tobytes()))
The above code creates the function cv2_imshow() with a single argument that consists of the image URL. The remaining parameters are:

2.To view the final images, run the following commands. Replace i with your desired array index value such as 0,1, or 2 depending on the number of faces in your image

 
Code Select
#display original cropped faces
 cv2_imshow(cropped_faces[i])
 #display restored cropped faces
 cv2_imshow(restored_faces[i])
 #display restored image
 cv2_imshow(restored_img)As applied in the above command, faces in the image are available in form of an array. Therefore every face has an index value (0,1,2,3 and so on). Set the number i depending on the face you want to view and the total number of faces in the image.

3.To view the final generated image, run the following command

Code Select
cv2_imshow(restored_img)
Verify that the generated image contains notable improvements as compared to your original image.

Save Generated Images
1.To save the final generated image to the output directory you created earlier, define the directory and filename as below

 
Code Select
output_directory = "/home/user/output_images/"
 output_filename = "image.jpg"
 output_filepath = os.path.join(output_directory, output_filename)
 if not os.path.exists(output_directory):
     os.makedirs(output_directory)The above code block consists of:

2.Save the image

Code Select
cv2.imwrite(output_filepath, restored_img)
Output:

Code Select
True
The above command returns True when the image saves successfully to the directory, and below is what the function does:

3.To download a copy of the generated images, launch a new terminal session and establish an SFTP connection to the server

Code Select
$ sftp user@SERVER-IP
4.When logged in, Switch to the output_images directory in your user home

Code Select
sftp> cd output_images
5.View available files in the directory

Code Select
sftp> ls
6.Download an image file of your choice. For example, to download the image.jpg file, run the following command

Code Select
sftp> get image.jpeg
7.When the download is complete, close the SFTP connection

Code Select
sftp> exit
8.Using your local computer's file explorer, find the image in your terminal's working directory

Memory Usage
In your Jupyter Notebook session, run the following command to view the GPU memory usage statistics

Code Select
!nvidia-smi
In Half-bit, your output should look like the one below:

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0    0    0      50737      C   /usr/bin/python3                 3252MiB |
+-----------------------------------------------------------------------------+
As displayed in the output, with half-bit precision, the model uses about 3 GB of VRAM.

With full-bit precision:

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0    0    0      50855      C   /usr/bin/python3                 4492MiB |
+-----------------------------------------------------------------------------+
In full-bit precision, the model uses 4.5 GB of VRAM.

Conclusion
In this article, you implemented the GFPGAN model on a Vultr Cloud GPU server. You prepared the server, installed dependency libraries, and used the model through Jupyter Notebook. For more information on how to use GFPGAN, visit the official model repository.

Text only | Text with Images