Optimization-based Neural Style Transfer

Overview

This page provides instructions on how to set up and run an artistic style transfer program on your machines. The following implementation is based on the optimization method proposed by Gatys et al in 2015 ([1]).

Setup & Execution

This section explains how to use the Neural Style Transfer program implemented on Python2.7 using Keras with Tensorflow as a backend.

Pre-requisites

This program has been implemented in Python 2.7. The following libraries must be installed on your system to run it:

• Numpy
• Scipy
• Keras
• h5py
• Tensorflow

You can use the requirements.txt file provided in the zip folder with the command: pip install requirements.txt for easy installation.

This program can be run on the CPU or a GPU without any modifications. It was tested on MacOS and NVIDIA GTX 970 GPU running CUDA 9.0.

Usage

Download the following zip file for the code Media:NeuralStyleTransfer.zip to your preferred directory on your machine. This program accepts multiple arguments that provide the user with the flexibility of altering inputs, outputs, and control parameters without modifying the code.

Here's a rundown on the arguments:

(Mandatory)

Without these arguments, the program will not run

• Path to content image
• Path to style images(s). One or more style images can be provided by separating each path with space.
• Path to save generated image.

At this point, you should be able to run the program. Here's how you can do that:

Open the Terminal from that directory and run the program using the following command:

 python NST.py path/to/content-image path/to/style-image(s) path/to/generated-image

Example:

python NST.py images/inputs/content/Dipping-Sun.jpg images/inputs/style/the_scream.jpg images/inputs/style/wave_kanagawa.jpg Results/blue

If you'd like to further control your program parameters, you can make use of these additional arguments:

(Optional)

These arguments have default values, and can hence be skipped if required. However, to include it in your command, just append the syntax provided below to the main command:

• Minimum image size. Default value: 400. Hence, the generated image saved has a size of 400 x 713.

  To change this, you can pass the following argument to the command:
  --image_size <new_value>
  Example: --image_size 800

• Content weight. Default value: 0.025.

  To change this, you can pass the following argument to the command:
  --content_weight <new_value>.
  Example: --content_weight 0.5

• Style weight(s). Default value: 1.0 (in case of single style transfer), 1/n for each style (in case of n-style transfer).

  To change this, you can pass the following argument to the command:
  --style_weight <new_value(s) separated by space>.
  Example: --style_weight 0.3 0.7
  Caution: Weights assigned are matched to the corresponding style image provided in the style path argument.

• Number of iterations. Default value: 10.

  To change this, you can pass the following argument to the command:
  --num_iter <new_value>.
  Example: --num_iter 25

• Model. Default value: "vgg19".

  To change this, you can pass the following argument to the command:
  --model <new_value>.
  Example: --model 'vgg16'

For example, to run a multi-style transfer program with custom weights and 30 iterations, you can run the following command from your directory:

python NST.py images/inputs/content/Dipping-Sun.jpg images/inputs/style/the_scream.jpg images/inputs/style/wave_kanagawa.jpg Results/generated --num_iter 30 --style_weight 0.3 0.7 --content_weight 0.1

Results

The results are saved to the path provided at the beginning through the mandatory argument - Path to save generated image. The program saves the intermediate output after each iteration along with the final result (which is just the output after the final iteration).