Python simply implements image style migration

Jun 01, 2021 Article blog

This article was reproduced to Know ID: Charles (Bai Lu) knows his personal column

Download the W3Cschool Mobile App, 0 Foundation Anytime, Anywhere Learning Programming >> Poke this to learn

Lead

T_T seemed to have sent similar articles before, which were implemented with Keras, now with PyTorch, and at that time the content felt a little watery, so I decided...

Well, I'm really just trying to write some PyTorch practice and drop by and water a beautiful text

Use Python for image-style migration!!!

Don't like the process students can still directly pull down to the end to see the results

Let's Go!

Resources

link:

http://pytorch.org/tutorials/advanced/neural_style_tutorial.html#

Yes, this is another tutorial from PyTorch's official website.

In the relevant documents I still provide my translated version

As well as the papers involved

Develop tools

Python version: 3.6.4

Related modules:

torch module;

PIL module;

matplotlib module;

torchvision module;

and some Python's own modules.

Torch version:

0.3.0

Environment construction

Install Python and add it to the environment variable, and pip installs the relevant modules that are required.

Additional notes:

PyTorch does not support direct pip installations for the time being.

There are two options:

(1) after the installation of anaconda3 installed in the environment of anaconda3 (direct pip installation can be);

2 Use the compiled whl file installation, download link is:

https://pan.baidu.com/s/1dF6ayLr#list/path=%2Fpytorch

Additional instructions

the required pre-training model is provided T_T relevant files, and if you don't think about the download speed of the website, you can download mine.

After downloading, put it in a folder similar to the following path:

Python simply implements image style migration1

Introduction to the principle

As a pure non-artistic student, I'm sorry I really can't explain what the art style of images is. Qi Baishi and Van Gogh's paintings are certainly not a style.

So let me try to explain the style migration:

The essence of style migration is to preserve the original content on the basis of another style to present the original painting.

So how do you quantify it?

Simply put, it's probably like this:

Use CNN to extract the features of the image layer by layer (the higher and more stable the features extracted from the back of the layer, that is, the higher the semantic information that represents the image), and use the Gram matrix output from one layer or layer of the image as a loss function to measure the content/style difference between the two images (T_T is that the two images go through the same convolutional neural network, comparing the differences between one layer or several layers of output Gram matrix).

What is the Gram matrix???

Specifically, it is calculated as:

As we all know, the output of a picture in a convolutional layer is characterized by a fourth-order sag of a shape (batch_size, channels, width, height), apparently batch_size is 1.

We'll look at the characteristics of the following image:

Python simply implements image style migration2

Convert to a matrix of (batch_size,channels, widths, heights) size, and multiplying this matrix by its transpose to get a matrix of the size (batch_size'channels, batch_size'channels), which is the Gram matrix.

In fact, this is to define the image style and image content of the two concepts, and then we can achieve style migration.

The process is probably this:

Python simply implements image style migration3

Details of the implementation can be found in the source code

There is also a personal translation of the official website tutorial in the relevant documents, as well as related papers for the reference of interested parties.

How to use it

Modify the picture path at the image shown below to be your own picture path: