Enhancing Image Quality Using LDM Super Resolution Techniques

templates/

static/

app.py

requirements.txt

This project aims to evaluate the performance of the LDM Super Resolution model on a diverse set of images, including images with different resolutions, content, and textures.

In this project, we will learn how to convert low-resolution images into high-resolution images using LDM(Latent Diffusion Model) Super Resolution.

Introduction

A computer vision approach called image super-resolution aims to increase the resolution of low-resolution images so that they are clearer and more detailed. Applications for super-resolution include the processing of medical images, surveillance footage, and satellite images. For accurate analysis in various applications, high-resolution pictures are required, however due to several constraints, the obtained images are typically low-resolution.

The LDM Super Resolution model, a deep learning-based approach to photo super-resolution, was developed by the Hugging Face Research team. The residual network (ResNet) architecture, a type of convolutional neural network (CNN) created to address the issue of vanishing gradients in deep neural networks, serves as the foundation for the model. The ResNet architecture has been modified in the LDM Super Resolution model to take into consideration the long-term dependencies present in the images.

Performance of the LDM Super Resolution model has been tested on a variety of pictures, including ones with various resolutions, contents, and textures. The outcomes show that the LDM Super Resolution model may improve low-resolution images' resolution while preserving fine details. The model outperforms existing established photo super-resolution techniques in terms of accuracy and output quality.

Installation Requirements

Make sure Python and pip are installed on your system.
Navigate to the directory containing the requirements.txt file using the command line/terminal.
Run the command: pip install -r requirements.txt
Wait for the installation process to complete.

Import Libraries

Lets import all the required libraries in order to to solve particular task.

from flask import Flask, request, render_template
import torch
import os
from PIL import Image
from diffusers import LDMSuperResolutionPipeline

Create Flask App

Let create a Flask app by defining app=Flask(__name__).

Lets set the configuration options in the app that the input image is uploaded to "static/input" and output imgae to "static/output," respectively. The images used as input and output for the LDM super-resolution pipeline are most likely stored in these folders.

app = Flask(__name__)
app.config['OUTPUT_FOLDER'] = 'static/output'
app.config['INPUT_FOLDER'] = 'static/input'

Setup Routes & Start App

The code below sets up the routes for the Flask application and starts the application.

The indx function handles the "/" route and returns the index.html template.
The "/srgan up" route, which is set up to handle POST requests, is managed by the srgan up() method. The LDM pipeline is used by this function to super-resize images. It first ascertains whether a GPU is accessible before modifying the device variable appropriately. A pre-trained LDM super-resolution model's identification is stored in the variable model id.
The model and scheduler are then loaded, and the pipeline is then transferred to the selected device by calling the function from pretrained(model id). The input image is retrieved via the HTTP request and saved in the request.files["image"] folder. By downsizing the input image with the Pillow library's Image.resize method, the low-resolution version of the picture is produced.
By using the method pipeline(low res img, num inference steps=100, eta=1), where num inference steps and eta are set to 100 and 1, the pipeline is executed in inference mode. Using the save function from the Pillow library, the first picture in the output images from the pipeline is stored in the folder labelled "static/output." The 'predict.html' template is finally returned.
The last code app.run(debug=True) starts the Flask application in debug mode, which allows for easier error tracking during development.

@app.route("/")
def idx():
    return render_template("index.html")

@app.route("/srgan_up", methods=["POST"])
def srgan_up():
    device = "cuda" if torch.cuda.is_available() else "cpu"
    model_id = "CompVis/ldm-super-resolution-4x-openimages"

    # load model and scheduler
    pipeline = LDMSuperResolutionPipeline.from_pretrained(model_id)
    res = pipeline.to(device)

    input_image = request.files["image"]
    input_image.save(os.path.join(app.config['INPUT_FOLDER'], 'input_image.jpg'))
    image = Image.open(input_image)
    low_res_img = image.resize((128, 128))
    # run pipeline in inference (sample random noise and denoise)
    out = res(low_res_img, num_inference_steps=100, eta=1).images[0]
    # save image
    out.save(os.path.join(app.config['OUTPUT_FOLDER'], 'output_image.jpg'))
    return render_template('predict.html')


app.run(debug=True)

Run the App & Make Predictions

Now it's time to run our app. To run the flask app, hit the below commands in your terminal

export FLASK_APP=app.py 
flask run

By default, the app will be live on port 5000.

The below HTML template display on the webpage when the app runs

index.html

On this page, the user uploads the image and hit the upscale image button & moves to the next page.

After hitting the button the below page will appear that shows low resolution i.e our input image and high-resolution image which is the predicted image.

Coders Packet