Upscaling_UltraQuick_CLIP_Guided_Diffusion_HQ_256x256_and_512x512.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Upscaling-UltraQuick CLIP Guided Diffusion HQ 256x256 and 512x512.ipynb",
      "private_outputs": true,
      "provenance": [],
      "collapsed_sections": [
        "4jxCQbtInUCN"
      ],
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sadnow/ESRGAN-UltraFast-CLIP-Guided-Diffusion-Colab/blob/main/Upscaling_UltraQuick_CLIP_Guided_Diffusion_HQ_256x256_and_512x512.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1YwMUyt9LHG1"
      },
      "source": [
        "# Generates images from text prompts with CLIP guided diffusion.\n",
        "\n",
        "By Katherine Crowson (https://github.com/crowsonkb, https://twitter.com/RiversHaveWings). It uses either OpenAI's 256x256 unconditional ImageNet or Katherine Crowson's fine-tuned 512x512 diffusion model (https://github.com/openai/guided-diffusion), together with CLIP (https://github.com/openai/CLIP) to connect text prompts with images.\n",
        "\n",
        "Modified by Daniel Russell (https://github.com/russelldc, https://twitter.com/danielrussruss) to include (hopefully) optimal params for quick generations in 15-100 timesteps rather than 1000, as well as more robust augmentations.\n",
        "\n",
        "**Update**: Sep 19th 2021\n",
        "\n",
        "\n",
        "Further improvements from Dango233 and nsheppard helped improve the quality of diffusion in general, and especially so for shorter runs like this notebook aims to achieve.\n",
        "\n",
        "Katherine's original notebook can be found here:\n",
        "https://colab.research.google.com/drive/1QBsaDAZv8np29FPbvjffbE1eytoJcsgA\n",
        "\n",
        "---\n",
        "\n",
        "**Update**: 9/27/21\n",
        "Further UI simplifications and Real-ESRGAN integration made by sadnow on 9/27/21. Updates can be found at https://github.com/sadnow/ESRGAN-UltraFast-CLIP-Guided-Diffusion-Colab if available.\n",
        "\n",
        "- Notebook now does 256x diffusion and then upscales to 1024x1024.\n",
        "- Real-ESRGAN upscaling at the end of every batch\n",
        "- Added a \"project name\" setting and changed temp folder location\n",
        "- Added a supper settings slider for simplicity and an optional override toggle\n",
        "\n",
        "---\n",
        "\n",
        "**Update**: Alpha 0.95 - 9/29/21 - The Init Image Update\n",
        "\n",
        "- Added form support for init images (supports both URL and filepath) and image_prompt\n",
        "- Added A100 support (untested, credit goes to sportracer48)\n",
        "- preemptive error correction for user input values \n",
        "- Documentation additions\n",
        "\n",
        "**I highly recommend trying out a silhouette as an init_image ^^**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qZ3rNuAWAewx",
        "cellView": "form"
      },
      "source": [
        "#@markdown ##Check GPU\n",
        "#@markdown ***This Notebook supports \"Run All. Once everything is set to your liking, you can do `Ctrl+F9`!***\n",
        "import torch\n",
        "# Check the GPU status\n",
        "device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n",
        "print('Using device:', device)\n",
        "\n",
        "enable_error_checking = False#@param {type:\"boolean\"}\n",
        "if enable_error_checking:\n",
        "  !nvidia-smi\n",
        "else:\n",
        "  !nvidia-smi\n",
        "  !nvidia-smi -i 0 -e 0\n",
        "#@markdown *Special thanks to sportracer48 and his Discord channel. If you want to make AI animations, he has the meats in closed beta.* https://www.patreon.com/sportsracer48"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yZsjzwS0YGo6",
        "cellView": "form"
      },
      "source": [
        "!pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html\n",
        "\n",
        "def dir_make(a):\n",
        "  import os.path\n",
        "  from os import path\n",
        "  if not path.exists(a):\n",
        "    print(\"Creating\"+a+\"...\")\n",
        "    !mkdir -p $a\n",
        "\n",
        "def dir_check(a):\n",
        "  import os.path\n",
        "  from os import path\n",
        "  if not path.exists(a):\n",
        "    return False\n",
        "  else:\n",
        "    return True\n",
        "\n",
        "def file_check(a):\n",
        "  import os.path\n",
        "  from os import path\n",
        "  if not path.isfile(a):\n",
        "    return False\n",
        "  else:\n",
        "    return True\n",
        "\n",
        "def image_upscale(model_path,scale,input,output):\n",
        "    %cd /content/Real-ESRGAN/\n",
        "    !python inference_realesrgan.py --model_path $model_path --netscale $scale --face_enhance --input $input --output $output --ext jpg\n",
        "    %cd /content/\n",
        "\n",
        "from google.colab import drive\n",
        "\n",
        "#@title Choose model here:\n",
        "#diffusion_model = \"256x256_diffusion_uncond\" #@param [\"256x256_diffusion_uncond\", \"512x512_diffusion_uncond_finetune_008100\"]\n",
        "diffusion_model = \"256x256_diffusion_uncond\"\n",
        "\n",
        "#@markdown If you connect your Google Drive, you can save the final image of each run on your drive.\n",
        "\n",
        "google_drive = True #@param {type:\"boolean\"}\n",
        "\n",
        "#@markdown You can use your mounted Google Drive to load the model checkpoint file if you've already got a copy downloaded there. This will save time (and resources!) when you re-visit this notebook in the future.\n",
        "\n",
        "#@markdown Click here if you'd like to save the diffusion model checkpoint file to (and/or load from) your Google Drive:\n",
        "yes_please = True #@param {type:\"boolean\"}\n",
        "\n",
        "_drive_location = '/content/drive/MyDrive/AI/Diffusion/' #@param{type:\"string\"}\n",
        "contains_slash = (_drive_location.endswith('/'))\n",
        "if not contains_slash:\n",
        "  _drive_location = _drive_location + '/'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4jxCQbtInUCN"
      },
      "source": [
        "# Install and import dependencies"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mQE-fIMnYKYK"
      },
      "source": [
        "#@title Download diffusion model\n",
        "\n",
        "model_path = '/content/'\n",
        "if google_drive:\n",
        "    from google.colab import drive\n",
        "    drive.mount('/content/drive')\n",
        "    if yes_please:\n",
        "        dir_make(_drive_location)\n",
        "        model_path = _drive_location\n",
        "\n",
        "\n",
        "if diffusion_model == '256x256_diffusion_uncond':\n",
        "    !wget --continue 'https://openaipublic.blob.core.windows.net/diffusion/jul-2021/256x256_diffusion_uncond.pt' -P {model_path}\n",
        "elif diffusion_model == '512x512_diffusion_uncond_finetune_008100':\n",
        "    !wget --continue 'https://the-eye.eu/public/AI/models/512x512_diffusion_unconditional_ImageNet/512x512_diffusion_uncond_finetune_008100.pt' -P {model_path}\n",
        "\n",
        "if google_drive and not yes_please:\n",
        "    model_path = _drive_location"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-_UVMZCIAq_r",
        "cellView": "form"
      },
      "source": [
        "def install_ESRGAN():\n",
        "    %cd /content/\n",
        "    print(\"Installing libraries for Real-ESRGAN upscaling.\")\n",
        "    !git clone https://github.com/xinntao/Real-ESRGAN.git\n",
        "    %cd /content/Real-ESRGAN\n",
        "    !pip install basicsr -q\n",
        "    !pip install facexlib -q\n",
        "    !pip install gfpgan -q\n",
        "    !pip install -r requirements.txt -q\n",
        "    %cd /content/Real-ESRGAN\n",
        "    !python setup.py develop -q\n",
        "    !wget -nc https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P experiments/pretrained_models\n",
        "    !wget -nc https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth -P experiments/pretrained_models\n",
        "    !wget -nc https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth -P experiments/pretrained_models\n",
        "    print(\"Finished Installing libraries for Real-ESRGAN upscaling.\")\n",
        "    %cd /content/\n",
        "    #\n",
        "\n",
        "#@markdown ##Git and pip install\n",
        "!git clone https://github.com/openai/CLIP\n",
        "!git clone https://github.com/crowsonkb/guided-diffusion\n",
        "!pip install -e ./CLIP\n",
        "!pip install -e ./guided-diffusion\n",
        "!pip install lpips datetime\n",
        "install_ESRGAN()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JmbrcrhpBPC6"
      },
      "source": [
        "#@markdown ##imports\n",
        "import gc\n",
        "import io\n",
        "import math\n",
        "import sys\n",
        "from IPython import display\n",
        "import lpips\n",
        "from PIL import Image, ImageOps\n",
        "import requests\n",
        "import torch\n",
        "from torch import nn\n",
        "from torch.nn import functional as F\n",
        "import torchvision.transforms as T\n",
        "import torchvision.transforms.functional as TF\n",
        "from tqdm.notebook import tqdm\n",
        "sys.path.append('./CLIP')\n",
        "sys.path.append('./guided-diffusion')\n",
        "import clip\n",
        "from guided_diffusion.script_util import create_model_and_diffusion, model_and_diffusion_defaults\n",
        "from datetime import datetime\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import random"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FpZczxnOnPIU",
        "cellView": "form"
      },
      "source": [
        "\n",
        "# https://gist.github.com/adefossez/0646dbe9ed4005480a2407c62aac8869\n",
        "\n",
        "def interp(t):\n",
        "    return 3 * t**2 - 2 * t ** 3\n",
        "\n",
        "def perlin(width, height, scale=10, device=None):\n",
        "    gx, gy = torch.randn(2, width + 1, height + 1, 1, 1, device=device)\n",
        "    xs = torch.linspace(0, 1, scale + 1)[:-1, None].to(device)\n",
        "    ys = torch.linspace(0, 1, scale + 1)[None, :-1].to(device)\n",
        "    wx = 1 - interp(xs)\n",
        "    wy = 1 - interp(ys)\n",
        "    dots = 0\n",
        "    dots += wx * wy * (gx[:-1, :-1] * xs + gy[:-1, :-1] * ys)\n",
        "    dots += (1 - wx) * wy * (-gx[1:, :-1] * (1 - xs) + gy[1:, :-1] * ys)\n",
        "    dots += wx * (1 - wy) * (gx[:-1, 1:] * xs - gy[:-1, 1:] * (1 - ys))\n",
        "    dots += (1 - wx) * (1 - wy) * (-gx[1:, 1:] * (1 - xs) - gy[1:, 1:] * (1 - ys))\n",
        "    return dots.permute(0, 2, 1, 3).contiguous().view(width * scale, height * scale)\n",
        "\n",
        "def perlin_ms(octaves, width, height, grayscale, device=device):\n",
        "    out_array = [0.5] if grayscale else [0.5, 0.5, 0.5]\n",
        "    # out_array = [0.0] if grayscale else [0.0, 0.0, 0.0]\n",
        "    for i in range(1 if grayscale else 3):\n",
        "        scale = 2 ** len(octaves)\n",
        "        oct_width = width\n",
        "        oct_height = height\n",
        "        for oct in octaves:\n",
        "            p = perlin(oct_width, oct_height, scale, device)\n",
        "            out_array[i] += p * oct\n",
        "            scale //= 2\n",
        "            oct_width *= 2\n",
        "            oct_height *= 2\n",
        "    return torch.cat(out_array)\n",
        "\n",
        "def create_perlin_noise(octaves=[1, 1, 1, 1], width=2, height=2, grayscale=True):\n",
        "    out = perlin_ms(octaves, width, height, grayscale)\n",
        "    if grayscale:\n",
        "        out = TF.resize(size=(side_x, side_y), img=out.unsqueeze(0))\n",
        "        out = TF.to_pil_image(out.clamp(0, 1)).convert('RGB')\n",
        "    else:\n",
        "        out = out.reshape(-1, 3, out.shape[0]//3, out.shape[1])\n",
        "        out = TF.resize(size=(side_x, side_y), img=out)\n",
        "        out = TF.to_pil_image(out.clamp(0, 1).squeeze())\n",
        "\n",
        "    out = ImageOps.autocontrast(out)\n",
        "    return out\n",
        "\n",
        "#################################################################################################################\n",
        "def fetch(url_or_path):\n",
        "    if str(url_or_path).startswith('http://') or str(url_or_path).startswith('https://'):\n",
        "        r = requests.get(url_or_path)\n",
        "        r.raise_for_status()\n",
        "        fd = io.BytesIO()\n",
        "        fd.write(r.content)\n",
        "        fd.seek(0)\n",
        "        return fd\n",
        "    return open(url_or_path, 'rb')\n",
        "\n",
        "\n",
        "def parse_prompt(prompt):\n",
        "    if prompt.startswith('http://') or prompt.startswith('https://'):\n",
        "        vals = prompt.rsplit(':', 2)\n",
        "        vals = [vals[0] + ':' + vals[1], *vals[2:]]\n",
        "    else:\n",
        "        vals = prompt.rsplit(':', 1)\n",
        "    vals = vals + ['', '1'][len(vals):]\n",
        "    return vals[0], float(vals[1])\n",
        "\n",
        "def sinc(x):\n",
        "    return torch.where(x != 0, torch.sin(math.pi * x) / (math.pi * x), x.new_ones([]))\n",
        "\n",
        "def lanczos(x, a):\n",
        "    cond = torch.logical_and(-a < x, x < a)\n",
        "    out = torch.where(cond, sinc(x) * sinc(x/a), x.new_zeros([]))\n",
        "    return out / out.sum()\n",
        "\n",
        "def ramp(ratio, width):\n",
        "    n = math.ceil(width / ratio + 1)\n",
        "    out = torch.empty([n])\n",
        "    cur = 0\n",
        "    for i in range(out.shape[0]):\n",
        "        out[i] = cur\n",
        "        cur += ratio\n",
        "    return torch.cat([-out[1:].flip([0]), out])[1:-1]\n",
        "\n",
        "def resample(input, size, align_corners=True):\n",
        "    n, c, h, w = input.shape\n",
        "    dh, dw = size\n",
        "\n",
        "    input = input.reshape([n * c, 1, h, w])\n",
        "\n",
        "    if dh < h:\n",
        "        kernel_h = lanczos(ramp(dh / h, 2), 2).to(input.device, input.dtype)\n",
        "        pad_h = (kernel_h.shape[0] - 1) // 2\n",
        "        input = F.pad(input, (0, 0, pad_h, pad_h), 'reflect')\n",
        "        input = F.conv2d(input, kernel_h[None, None, :, None])\n",
        "\n",
        "    if dw < w:\n",
        "        kernel_w = lanczos(ramp(dw / w, 2), 2).to(input.device, input.dtype)\n",
        "        pad_w = (kernel_w.shape[0] - 1) // 2\n",
        "        input = F.pad(input, (pad_w, pad_w, 0, 0), 'reflect')\n",
        "        input = F.conv2d(input, kernel_w[None, None, None, :])\n",
        "\n",
        "    input = input.reshape([n, c, h, w])\n",
        "    return F.interpolate(input, size, mode='bicubic', align_corners=align_corners)\n",
        "\n",
        "class MakeCutouts(nn.Module):\n",
        "    def __init__(self, cut_size, cutn, skip_augs=False):\n",
        "        super().__init__()\n",
        "        self.cut_size = cut_size\n",
        "        self.cutn = cutn\n",
        "        self.skip_augs = skip_augs\n",
        "        self.augs = T.Compose([\n",
        "            T.RandomHorizontalFlip(p=0.5),\n",
        "            T.Lambda(lambda x: x + torch.randn_like(x) * 0.01),\n",
        "            T.RandomAffine(degrees=15, translate=(0.1, 0.1)),\n",
        "            T.Lambda(lambda x: x + torch.randn_like(x) * 0.01),\n",
        "            T.RandomPerspective(distortion_scale=0.4, p=0.7),\n",
        "            T.Lambda(lambda x: x + torch.randn_like(x) * 0.01),\n",
        "            T.RandomGrayscale(p=0.15),\n",
        "            T.Lambda(lambda x: x + torch.randn_like(x) * 0.01),\n",
        "            # T.ColorJitter(brightness=0.1, contrast=0.1, saturation=0.1, hue=0.1),\n",
        "        ])\n",
        "\n",
        "    def forward(self, input):\n",
        "        input = T.Pad(input.shape[2]//4, fill=0)(input)\n",
        "        sideY, sideX = input.shape[2:4]\n",
        "        max_size = min(sideX, sideY)\n",
        "\n",
        "        cutouts = []\n",
        "        for ch in range(cutn):\n",
        "            if ch > cutn - cutn//4:\n",
        "                cutout = input.clone()\n",
        "            else:\n",
        "                size = int(max_size * torch.zeros(1,).normal_(mean=.8, std=.3).clip(float(self.cut_size/max_size), 1.))\n",
        "                offsetx = torch.randint(0, abs(sideX - size + 1), ())\n",
        "                offsety = torch.randint(0, abs(sideY - size + 1), ())\n",
        "                cutout = input[:, :, offsety:offsety + size, offsetx:offsetx + size]\n",
        "\n",
        "            if not self.skip_augs:\n",
        "                cutout = self.augs(cutout)\n",
        "            cutouts.append(resample(cutout, (self.cut_size, self.cut_size)))\n",
        "            del cutout\n",
        "\n",
        "        cutouts = torch.cat(cutouts, dim=0)\n",
        "        return cutouts\n",
        "\n",
        "\n",
        "def spherical_dist_loss(x, y):\n",
        "    x = F.normalize(x, dim=-1)\n",
        "    y = F.normalize(y, dim=-1)\n",
        "    return (x - y).norm(dim=-1).div(2).arcsin().pow(2).mul(2)\n",
        "\n",
        "\n",
        "def tv_loss(input):\n",
        "    \"\"\"L2 total variation loss, as in Mahendran et al.\"\"\"\n",
        "    input = F.pad(input, (0, 1, 0, 1), 'replicate')\n",
        "    x_diff = input[..., :-1, 1:] - input[..., :-1, :-1]\n",
        "    y_diff = input[..., 1:, :-1] - input[..., :-1, :-1]\n",
        "    return (x_diff**2 + y_diff**2).mean([1, 2, 3])\n",
        "\n",
        "\n",
        "def range_loss(input):\n",
        "    return (input - input.clamp(-1, 1)).pow(2).mean([1, 2, 3])\n",
        "\n",
        "##########################################################################################################\n",
        "\n",
        "#@markdown ##def do_run()\n",
        "def do_run():\n",
        "    loss_values = []\n",
        " \n",
        "    if seed is not None:\n",
        "        np.random.seed(seed)\n",
        "        random.seed(seed)\n",
        "        torch.manual_seed(seed)\n",
        "        torch.cuda.manual_seed_all(seed)\n",
        "        torch.backends.cudnn.deterministic = True\n",
        " \n",
        "    make_cutouts = MakeCutouts(clip_size, cutn, skip_augs=skip_augs)\n",
        "    target_embeds, weights = [], []\n",
        " \n",
        "    for prompt in text_prompts:\n",
        "        txt, weight = parse_prompt(prompt)\n",
        "        txt = clip_model.encode_text(clip.tokenize(prompt).to(device)).float()\n",
        "        print(\"Tokenized text_prompt is: \",txt)\n",
        "\n",
        "        if fuzzy_prompt:\n",
        "            for i in range(25):\n",
        "                target_embeds.append((txt + torch.randn(txt.shape).cuda() * rand_mag).clamp(0,1))\n",
        "                weights.append(weight)\n",
        "        else:\n",
        "            target_embeds.append(txt)\n",
        "            weights.append(weight)\n",
        " \n",
        "    for prompt in image_prompts:\n",
        "        path, weight = parse_prompt(prompt)\n",
        "        img = Image.open(fetch(path)).convert('RGB')\n",
        "        img = TF.resize(img, min(side_x, side_y, *img.size), T.InterpolationMode.LANCZOS)\n",
        "        batch = make_cutouts(TF.to_tensor(img).to(device).unsqueeze(0).mul(2).sub(1))\n",
        "        embed = clip_model.encode_image(normalize(batch)).float()\n",
        "        if fuzzy_prompt:\n",
        "            for i in range(25):\n",
        "                target_embeds.append((embed + torch.randn(embed.shape).cuda() * rand_mag).clamp(0,1))\n",
        "                weights.extend([weight / cutn] * cutn)\n",
        "        else:\n",
        "            target_embeds.append(embed)\n",
        "            weights.extend([weight / cutn] * cutn)\n",
        " \n",
        "    target_embeds = torch.cat(target_embeds)\n",
        "    weights = torch.tensor(weights, device=device)\n",
        "    if weights.sum().abs() < 1e-3:\n",
        "        raise RuntimeError('The weights must not sum to 0.')\n",
        "    weights /= weights.sum().abs()\n",
        " \n",
        "    init = None\n",
        "    if init_image is not None:\n",
        "        init = Image.open(fetch(init_image)).convert('RGB')\n",
        "        init = init.resize((side_x, side_y), Image.LANCZOS)\n",
        "        init = TF.to_tensor(init).to(device).unsqueeze(0).mul(2).sub(1)\n",
        "    \n",
        "    if perlin_init:\n",
        "        if perlin_mode == 'color':\n",
        "            init = create_perlin_noise([1.5**-i*0.5 for i in range(12)], 1, 1, False)\n",
        "            init2 = create_perlin_noise([1.5**-i*0.5 for i in range(8)], 4, 4, False)\n",
        "        elif perlin_mode == 'gray':\n",
        "           init = create_perlin_noise([1.5**-i*0.5 for i in range(12)], 1, 1, True)\n",
        "           init2 = create_perlin_noise([1.5**-i*0.5 for i in range(8)], 4, 4, True)\n",
        "        else:\n",
        "           init = create_perlin_noise([1.5**-i*0.5 for i in range(12)], 1, 1, False)\n",
        "           init2 = create_perlin_noise([1.5**-i*0.5 for i in range(8)], 4, 4, True)\n",
        "        \n",
        "        # init = TF.to_tensor(init).add(TF.to_tensor(init2)).div(2).to(device)\n",
        "        init = TF.to_tensor(init).add(TF.to_tensor(init2)).div(2).to(device).unsqueeze(0).mul(2).sub(1)\n",
        "        del init2\n",
        " \n",
        "    cur_t = None\n",
        " \n",
        "    def cond_fn(x, t, y=None):\n",
        "        with torch.enable_grad():\n",
        "            x = x.detach().requires_grad_()\n",
        "            n = x.shape[0]\n",
        "            my_t = torch.ones([n], device=device, dtype=torch.long) * cur_t\n",
        "            out = diffusion.p_mean_variance(model, x, my_t, clip_denoised=False, model_kwargs={'y': y})\n",
        "            fac = diffusion.sqrt_one_minus_alphas_cumprod[cur_t]\n",
        "            x_in = out['pred_xstart'] * fac + x * (1 - fac)\n",
        "            x_in_grad = torch.zeros_like(x_in)\n",
        "            for i in range(cutn_batches):\n",
        "                clip_in = normalize(make_cutouts(x_in.add(1).div(2)))\n",
        "                image_embeds = clip_model.encode_image(clip_in).float()\n",
        "                dists = spherical_dist_loss(image_embeds.unsqueeze(1), target_embeds.unsqueeze(0))\n",
        "                dists = dists.view([cutn, n, -1])\n",
        "                losses = dists.mul(weights).sum(2).mean(0)\n",
        "                loss_values.append(losses.sum().item()) # log loss, probably shouldn't do per cutn_batch\n",
        "                x_in_grad += torch.autograd.grad(losses.sum() * clip_guidance_scale, x_in)[0] / cutn_batches\n",
        "            tv_losses = tv_loss(x_in)\n",
        "            range_losses = range_loss(out['pred_xstart'])\n",
        "            sat_losses = torch.abs(x_in - x_in.clamp(min=-1,max=1)).mean()\n",
        "            loss = tv_losses.sum() * tv_scale + range_losses.sum() * range_scale + sat_losses.sum() * sat_scale\n",
        "            if init is not None and init_scale:\n",
        "                init_losses = lpips_model(x_in, init)\n",
        "                loss = loss + init_losses.sum() * init_scale\n",
        "            x_in_grad += torch.autograd.grad(loss, x_in)[0]\n",
        "            grad = -torch.autograd.grad(x_in, x, x_in_grad)[0]\n",
        "        if clamp_grad:\n",
        "            magnitude = grad.square().mean().sqrt()\n",
        "            return grad * magnitude.clamp(max=0.05) / magnitude\n",
        "        return grad\n",
        " \n",
        "    if model_config['timestep_respacing'].startswith('ddim'):\n",
        "        sample_fn = diffusion.ddim_sample_loop_progressive\n",
        "    else:\n",
        "        sample_fn = diffusion.p_sample_loop_progressive\n",
        " \n",
        "    for i in range(n_batches):\n",
        "        cur_t = diffusion.num_timesteps - skip_timesteps - 1\n",
        " \n",
        "        if model_config['timestep_respacing'].startswith('ddim'):\n",
        "            samples = sample_fn(\n",
        "                model,\n",
        "                (batch_size, 3, side_y, side_x),\n",
        "                clip_denoised=clip_denoised,\n",
        "                model_kwargs={},\n",
        "                cond_fn=cond_fn,\n",
        "                progress=True,\n",
        "                skip_timesteps=skip_timesteps,\n",
        "                init_image=init,\n",
        "                randomize_class=randomize_class,\n",
        "                eta=eta,\n",
        "            )\n",
        "        else:\n",
        "            samples = sample_fn(\n",
        "                model,\n",
        "                (batch_size, 3, side_y, side_x),\n",
        "                clip_denoised=clip_denoised,\n",
        "                model_kwargs={},\n",
        "                cond_fn=cond_fn,\n",
        "                progress=True,\n",
        "                skip_timesteps=skip_timesteps,\n",
        "                init_image=init,\n",
        "                randomize_class=randomize_class,\n",
        "            )\n",
        "\n",
        "        for j, sample in enumerate(samples):\n",
        "            display.clear_output(wait=True)\n",
        "            cur_t -= 1\n",
        "            if j % display_rate == 0 or cur_t == -1:\n",
        "                for k, image in enumerate(sample['pred_xstart']):\n",
        "                    tqdm.write(f'Batch {i}, step {j}, output {k}:')\n",
        "                    current_time = datetime.now().strftime('%y%m%d-%H%M%S_%f')\n",
        "                    filename = f'progress_batch{i:05}_iteration{j:05}_output{k:05}_{current_time}.png'\n",
        "                    image = TF.to_pil_image(image.add(1).div(2).clamp(0, 1))\n",
        "                    image.save('/content/image_storage/' + filename)\n",
        "                    display.display(display.Image('/content/image_storage/' + filename))\n",
        "                    if google_drive and cur_t == -1:\n",
        "                        final_filename = output_folder_images + filename\n",
        "                        print(final_filename)\n",
        "                        #print('THIS VISIBLE? TEST!')\n",
        "                        if upscale_multiplier >= 2:\n",
        "                          temp_image_filename = temp_image_storage + _project_name + filename\n",
        "                          image.save(temp_image_filename)\n",
        "                          #gc.collect()\n",
        "                          #image_upscale(realesrgan_mpath,upscale_value,temp_image_filename,'final_filename')\n",
        "                          #def image_upscale(realesrgan_mpath,scale,temp_image_filename,_output_folder_images):\n",
        "                          %cd /content/Real-ESRGAN/\n",
        "                          !python inference_realesrgan.py --model_path $realesrgan_mpath --netscale $upscale_value --face_enhance --input $temp_image_filename --output $output_folder_images --ext jpg\n",
        "                          if upscale_multiplier == 4:\n",
        "                            gc.collect()\n",
        "                            import os\n",
        "                            doubleup_path = os.path.splitext(final_filename)[0]\n",
        "                            doubleup_path = doubleup_path + '_out.jpg'\n",
        "                            print(\"finalfilename was\",final_filename)\n",
        "                            print(\"Thingy is \",doubleup_path)\n",
        "                            !python inference_realesrgan.py --model_path '/content/Real-ESRGAN/experiments/pretrained_models/RealESRGAN_x2plus.pth' --netscale 2 --face_enhance --input $doubleup_path --output $output_folder_images --ext jpg\n",
        "                          %cd /content/\n",
        "                        else:\n",
        "                          image.save(output_folder_images + filename)\n",
        "\n",
        "                        \n",
        " \n",
        "        plt.plot(np.array(loss_values), 'r')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9zY-8I90LkC6"
      },
      "source": [
        "# Settings & Generation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fpbody2NCR7w",
        "cellView": "form"
      },
      "source": [
        "timestep_respacing = 'ddim25' #@param ['25','50','150','250','500','1000','ddim25','ddim50','ddim150','ddim250','ddim500','ddim1000']\n",
        "#@markdown *Modify this value to decrease the number of iterations/prompt.\n",
        "# timestep_respacing = '25'\n",
        "diffusion_steps = 1000\n",
        "\n",
        "model_config = model_and_diffusion_defaults()\n",
        "if diffusion_model == '512x512_diffusion_uncond_finetune_008100':\n",
        "    model_config.update({\n",
        "        'attention_resolutions': '32, 16, 8',\n",
        "        'class_cond': False,\n",
        "        'diffusion_steps': diffusion_steps,\n",
        "        'rescale_timesteps': True,\n",
        "        'timestep_respacing': timestep_respacing,\n",
        "        'image_size': 512,\n",
        "        'learn_sigma': True,\n",
        "        'noise_schedule': 'linear',\n",
        "        'num_channels': 256,\n",
        "        'num_head_channels': 64,\n",
        "        'num_res_blocks': 2,\n",
        "        'resblock_updown': True,\n",
        "        'use_fp16': True,\n",
        "        'use_scale_shift_norm': True,\n",
        "    })\n",
        "elif diffusion_model == '256x256_diffusion_uncond':\n",
        "    model_config.update({\n",
        "        'attention_resolutions': '32, 16, 8',\n",
        "        'class_cond': False,\n",
        "        'diffusion_steps': diffusion_steps,\n",
        "        'rescale_timesteps': True,\n",
        "        'timestep_respacing': timestep_respacing,\n",
        "        'image_size': 256,\n",
        "        'learn_sigma': True,\n",
        "        'noise_schedule': 'linear',\n",
        "        'num_channels': 256,\n",
        "        'num_head_channels': 64,\n",
        "        'num_res_blocks': 2,\n",
        "        'resblock_updown': True,\n",
        "        'use_fp16': True,\n",
        "        'use_scale_shift_norm': True,\n",
        "    })\n",
        "side_x = side_y = model_config['image_size']\n",
        "\n",
        "model, diffusion = create_model_and_diffusion(**model_config)\n",
        "model.load_state_dict(torch.load(f'{model_path}{diffusion_model}.pt', map_location='cpu'))\n",
        "model.requires_grad_(False).eval().to(device)\n",
        "for name, param in model.named_parameters():\n",
        "    if 'qkv' in name or 'norm' in name or 'proj' in name:\n",
        "        param.requires_grad_()\n",
        "if model_config['use_fp16']:\n",
        "    model.convert_to_fp16()\n",
        "\n",
        "################################################\n",
        "\n",
        "clip_model = clip.load('ViT-B/16', jit=False)[0].eval().requires_grad_(False).to(device)\n",
        "clip_size = clip_model.visual.input_resolution\n",
        "normalize = T.Normalize(mean=[0.48145466, 0.4578275, 0.40821073], std=[0.26862954, 0.26130258, 0.27577711])\n",
        "lpips_model = lpips.LPIPS(net='vgg').to(device)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U0PwzFZbLfcy",
        "cellView": "form"
      },
      "source": [
        "#@markdown ##Simple Settings\n",
        "_text_prompt =  \"Sad panda vector drawing stylized low-poly art, 3D HD panda bear trending on artstation\" #@param {type:\"string\"}\n",
        "_project_name = 'panda' #@param{type:\"string\"}\n",
        "\n",
        "#_run_upscaler = True #@param{type:\"boolean\"}\n",
        "_run_upscaler = True\n",
        "#--------------------------\n",
        "#Output image directory handling\n",
        "output_folder_images = _drive_location + _project_name\n",
        "contains_slash = (output_folder_images.endswith('/'))\n",
        "if not contains_slash:\n",
        "  print(\"Adding slash to _drive_location...\")\n",
        "  output_folder_images = output_folder_images + '/'\n",
        "dir_make(output_folder_images)\n",
        "temp_image_storage = '/content/image_storage/'  #for non-upscaled images\n",
        "dir_make(temp_image_storage)\n",
        "#-------------------------------------------------\n",
        "\n",
        "text_prompts = [\n",
        "    # \"an abstract painting of 'ravioli on a plate'\",\n",
        "    # 'cyberpunk wizard on top of a skyscraper, trending on artstation, photorealistic depiction of a cyberpunk wizard',\n",
        "    _text_prompt]\n",
        "    # 'cyberpunk wizard',\n",
        "\n",
        "\n",
        "n_batches =  10000#@param{type:\"integer\"}\n",
        "_easySettings = 1 #@param {type:\"slider\", min:0, max:2, step:0.1}\n",
        "#@markdown `_easySettings` controls a few of the below advanced settings. Experiment!\n",
        "\n",
        "#@markdown Leave the `_init_image` blank if you want to generate from scratch.\n",
        "#------------------------------------------------\n",
        "#@markdown ---\n",
        "##@markdown ##Bulk/Batch Settings\n",
        "\n",
        "batch_size =  1\n",
        "\n",
        "#------------------------------------------------\n",
        "#@markdown ##Intermediate Settings (Optional)\n",
        "\n",
        "_init_image = 'https://www.pngitem.com/pimgs/m/16-161339_giant-panda-bear-silhouette-drawing-clip-art-silhouette.png' #@param{type:\"string\"}\n",
        "if _init_image == '':\n",
        "  init_image = None\n",
        "else:\n",
        "  init_image = _init_image  # None - URL or local path\n",
        "\n",
        "_init_scale =  300#@param{type:\"integer\"}\n",
        "default_init_scale = 1000 #in case user forgets to set\n",
        "if _init_image == '':\n",
        "  print(\"No init image detected. Setting init_scale to 0...\")\n",
        "  init_scale = 0\n",
        "else:\n",
        "  init_scale = _init_scale\n",
        "  if init_scale == 0:\n",
        "    print(\"WARNING: init_image is set but there is no init_scale! Setting to \",default_init_scale)\n",
        "    init_scale = default_init_scale\n",
        "\n",
        "_image_prompts = \"\" #@param{type:\"string\"}\n",
        "if not _image_prompts == \"\":\n",
        "  image_prompts = [\n",
        "      _image_prompts,\n",
        "  ]\n",
        "\n",
        "\n",
        "  \n",
        "\n",
        "#_saturation_scale = 0 #@param{type:\"integer\"}\n",
        "#sat_scale = _saturation_scale\n",
        "  # 0 - Controls how much saturation is allowed. From nshepperd's JAX notebook.\n",
        "\n",
        "_noise_amount = 0 #@param {type:\"slider\", min:0.00, max:1, step:0.05}\n",
        "if _noise_amount >=0.05: \n",
        "  add_random_noise = True\n",
        "else:\n",
        "  add_random_noise = False\n",
        "rand_mag = _noise_amount # 0.1 - Controls the magnitude of the random noise\n",
        "if not init_image == '':\n",
        "  perlin_init = add_random_noise #Option to start with random perlin noise\n",
        "else:\n",
        "  print(\"WARNING: init_images do not work with _noise options! Disabling noise...\")\n",
        "  perlin_init = False\n",
        "#@markdown A possible good value for `_init_scale` is 1000. If set to `0` it will default back to 1000 when an init image is present.\n",
        "\n",
        "#@markdown `_noise` settings are deactivated if an `init_image` is detected.\n",
        "\n",
        "##@markdown Set `saturation_scale` to `0` to use native saturation control (default). \n",
        "\n",
        "#@markdown ---\n",
        "#----------------------------------------------\n",
        "\n",
        "##@markdown ---\n",
        "#@markdown ##Advanced Settings (Optional)\n",
        "\n",
        "\n",
        "# 350/50/50/32 and 500/0/0/64 have worked well for 25 timesteps on 256px\n",
        "# Also, sometimes 1 cutn actually works out fine\n",
        "#clip_guidance_scale = 5000 # 1000 - Controls how much the image should look like the prompt.\n",
        "#clip_guidance_scale = 5000 # 1000 - Controls how much the image should look like the prompt.\n",
        "#tv_scale = 150 # 150 - Controls the smoothness of the final output.\n",
        "#range_scale = 150 # 150 - Controls how far out of range RGB values are allowed to be.\n",
        "cutn = 16 #16\n",
        "  #Controls how many crops to take from the image.\n",
        "cutn_batches = 8 #@param [2,4,8,16] {type:\"raw\"}\n",
        "_noise_mode = 'mixed' #@param ['mixed','gray','color']\n",
        "perlin_mode = _noise_mode # 'mixed' ('gray', 'color')\n",
        "override_easySettings = False #@param{type:\"boolean\"}\n",
        "if override_easySettings:\n",
        "  _clip_guidance_scale = 1.3 #@param {type:\"slider\", min:0, max:2, step:0.1}\n",
        "  _tv_scale = 0.4 #@param {type:\"slider\", min:0, max:2, step:0.1}\n",
        "  _range_scale = 0.5 #@param {type:\"slider\", min:0, max:2, step:0.1}\n",
        "else:\n",
        "  _clip_guidance_scale = _easySettings\n",
        "  _tv_scale = _easySettings\n",
        "  _range_scale = _easySettings\n",
        "\n",
        "clip_guidance_scale = 5000 * _clip_guidance_scale\n",
        "tv_scale = 150 * _tv_scale\n",
        "range_scale = 150 * _range_scale\n",
        "\n",
        "# 2 - Accumulate CLIP gradient from multiple batches of cuts [Can help with OOM errors / Low VRAM]\n",
        "\n",
        "realesrgan_mpath='/content/Real-ESRGAN/experiments/pretrained_models/RealESRGAN_x4plus.pth' #@param ['/content/Real-ESRGAN/experiments/pretrained_models/RealESRGAN_x4plus_anime_6B.pth','/content/Real-ESRGAN/experiments/pretrained_models/RealESRGAN_x4plus.pth','/content/Real-ESRGAN/experiments/pretrained_models/RealESRGAN_x2plus.pth']\n",
        "if realesrgan_mpath == '/content/Real-ESRGAN/experiments/pretrained_models/RealESRGAN_x2plus.pth':\n",
        "  upscale_value=\"2\"\n",
        "else:\n",
        "  upscale_value=\"4\"\n",
        "\n",
        "#upscale_multiplier = \"2\" #@param [2,4]\n",
        "upscale_multiplier = \"2\"\n",
        "upscale_multiplier = int(upscale_multiplier)\n",
        "\n",
        "\n",
        "\n",
        "skip_timesteps = 6 # 0 - Controls the starting point along the diffusion timesteps\n",
        "\n",
        "\n",
        "skip_augs = False # False - Controls whether to skip torchvision augmentations\n",
        "randomize_class = True # True - Controls whether the imagenet class is randomly changed each iteration\n",
        "clip_denoised = False # False - Determines whether CLIP discriminates a noisy or denoised image\n",
        "clamp_grad = True # True - Experimental: Using adaptive clip grad in the cond_fn\n",
        "\n",
        "seed = None\n",
        "# seed = random.randint(0, 2**32) # Choose a random seed and print it at end of run for reproduction\n",
        "\n",
        "fuzzy_prompt = False # False - Controls whether to add multiple noisy prompts to the prompt losses\n",
        "\n",
        "eta = 0.5 # 0.0 - DDIM hyperparameter\n",
        "\n",
        "#@markdown `cutn_batches` might increase quality at the cost of speed. Lowering it may help with VRAM issues.\n",
        "\n",
        "################################################\n",
        "\n",
        "\n",
        "display_rate = 1\n",
        "# 1 - Controls how many consecutive batches of images are generated\n",
        "\n",
        "gc.collect()\n",
        "torch.cuda.empty_cache()\n",
        "try:\n",
        "    do_run()\n",
        "except KeyboardInterrupt:\n",
        "    gc.collect()\n",
        "    torch.cuda.empty_cache()\n",
        "    pass\n",
        "finally:\n",
        "    print('seed', seed)\n",
        "    print('Output(s) saved to ',output_folder_images)\n",
        "    gc.collect()\n",
        "    torch.cuda.empty_cache()\n",
        "\n",
        "###############################################"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xc1W6_SAAvJU"
      },
      "source": [
        "# Prompt Engineering (Documentation)\n",
        "\n",
        "Collapse this section if it makes you feel cramped-"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u4Ut78Lol7EN"
      },
      "source": [
        "**Example of weights:** `Representation of chronic anxiety:2, illustrated by Luigi Serafini:1, inspired by the Codex Seraphinianus:1`\n",
        "\n",
        "*If you use `:` (weights), you must weigh out everything or you'll receive an error. For example, if you have three things and one has a weight of `:2`, the other two things need to have a weight of `:1`.\n",
        "\n",
        "*I am currently unsure whether `|` (pipes) do the same thing as commas.*\n",
        "\n",
        "---\n",
        "\n",
        "Diffusion prompting might seem trickier to master than VQGAN models, but it still allows for some level of control. Sometimes less it better.\n",
        "\n",
        "It doesn't always need the `trending on artstation` type of lingo; sometimes it will benefit more from something like `photorealistic 4k nature replication`.\n",
        "\n",
        "Diffusion is kind of unique in how it often doesn't duplicate the desired subject. For example, `cat photo` will usually only give you 1 cat.\n",
        "\n",
        "---\n",
        "\n",
        "### A Giant List of Terms to Try\n",
        "\n",
        "Credit for this list goes to @Atman on Discord and/or whoever else contributed to the Pastebin.\n",
        "\n",
        "*'8k resolution'\n",
        ",'pencil sketch'\n",
        ",'8K 3D'\n",
        ",'creative commons attribution'\n",
        ",'deviantart'\n",
        ",'CryEngine'\n",
        ",'Unreal Engine'\n",
        ",'concept art'\n",
        ",'photoillustration'\n",
        ",'pixiv'\n",
        ",'Flickr'\n",
        ",'ArtStation HD'\n",
        ",'Behance HD'\n",
        ",'HDR'\n",
        ",'anime'\n",
        ",'filmic'\n",
        ",'Stock photo'\n",
        ",'Ambient occlusion'\n",
        ",'Global illumination'\n",
        ",'Chalk art'\n",
        ",'Low poly'\n",
        ",'Booru'\n",
        ",'Polycount'\n",
        ",'Acrylic art'\n",
        ",'Hyperrealism'\n",
        ",'Zbrush Central'\n",
        ",'Rendered in Cinema4D'\n",
        ",'Rendered in Maya'\n",
        ",'Photo taken with Nikon D750'\n",
        ",'Tilt shift'\n",
        ",'Mixed media'\n",
        ",'Depth of field'\n",
        ",'DSLR'\n",
        ",'Detailed painting'\n",
        ",'Volumetric lighting'\n",
        ",'Storybook illustration'\n",
        ",'Unsplash contest winner'\n",
        ",'#vfxfriday'\n",
        ",'Ultrafine detail'\n",
        ",'20 megapixels'\n",
        ",'Photo taken with Fujifilm Superia'\n",
        ",'Photo taken with Ektachrome'\n",
        ",'matte painting'\n",
        ",'reimagined by industrial light and magic'\n",
        ",'Watercolor'\n",
        ",'CGSociety'\n",
        ",'childs drawing'\n",
        ",'marble sculpture'\n",
        ",'airbrush art'\n",
        ",'renaissance painting'\n",
        ",'Velvia'\n",
        ",'Provia'\n",
        ",'photo taken with Provia'\n",
        ",'prerendered graphics'\n",
        ",'criterion collection'\n",
        ",'dye-transfer'\n",
        ",'stipple'\n",
        ",'Parallax'\n",
        ",'Bryce 3D'\n",
        ",'Terragen'\n",
        ",'(2013) directed by cinematography by'\n",
        ",'Bokeh'\n",
        ",'1990s 1995'\n",
        ",'1970s 1975'\n",
        ",'1920s 1925'\n",
        ",'charcoal drawing'\n",
        ",'commission for'\n",
        ",'furaffinity'\n",
        ",'flat shading'\n",
        ",'ink drawing'\n",
        ",'artwork'\n",
        ",'oil on canvas'\n",
        ",'macro photography'\n",
        ",'hall of mirrors'\n",
        ",'polished'\n",
        ",'sunrays shine upon it'\n",
        ",'aftereffects'\n",
        ",'iridescent'\n",
        ",'#film'\n",
        ",'datamosh'\n",
        ",'(1962) directed by cinematography'\n",
        ",'holographic'\n",
        ",'dutch golden age'\n",
        ",'digitally enhanced'\n",
        ",'National Geographic photo'\n",
        ",'Associated Press photo'\n",
        ",'matte background'\n",
        ",'Art on Instagram'\n",
        ",'#myportfolio'\n",
        ",'digital illustration'\n",
        ",'stock photo'\n",
        ",'aftereffects'\n",
        ",'speedpainting'\n",
        ",'colorized'\n",
        ",'detailed'\n",
        ",'psychedelic'\n",
        ",'wavy'\n",
        ",'groovy'\n",
        ",'movie poster'\n",
        ",'pop art'\n",
        ",'made of beads and yarn'\n",
        ",'made of feathers'\n",
        ",'made of crystals'\n",
        ",'made of liquid metal'\n",
        ",'made of glass'\n",
        ",'made of cardboard'\n",
        ",'made of vines'\n",
        ",'made of cheese'\n",
        ",'made of flowers'\n",
        ",'made of insects'\n",
        ",'made of mist'\n",
        ",'made of paperclips'\n",
        ",'made of rubber'\n",
        ",'made of plastic'\n",
        ",'made of wire'\n",
        ",'made of trash'\n",
        ",'made of wrought iron'\n",
        ",'made of all of the above'\n",
        ",'tattoo'\n",
        ",'woodcut'\n",
        ",'American propaganda'\n",
        ",'Soviet propaganda'\n",
        ",'PS1 graphics'\n",
        ",'Fine art'\n",
        ",'HD mod'\n",
        ",'Photorealistic'\n",
        ",'Poster art'\n",
        ",'Constructivism'\n",
        ",'pre-Raphaelite'\n",
        ",'Impressionism'\n",
        ",'Lowbrow'\n",
        ",'RTX on'\n",
        ",'chiaroscuro'\n",
        ",'Egyptian art'\n",
        ",'Fauvism'\n",
        ",'shot on 70mm'\n",
        ",'Art Deco'\n",
        ",'Picasso'\n",
        ",'Da Vinci'\n",
        ",'Academic art'\n",
        ",'3840x2160'\n",
        ",'Photocollage'\n",
        ",'Cubism'\n",
        ",'Surrealist'\n",
        ",'THX Sound'\n",
        ",'ZBrush'\n",
        ",'Panorama'\n",
        ",'smooth'\n",
        ",'DC Comics'\n",
        ",'Marvel Comics'\n",
        ",'Ukiyo-e'\n",
        ",'Flemish Baroque'\n",
        ",'vray tracing'\n",
        ",'pixel perfect'\n",
        ",'quantum wavetracing'\n",
        ",'Zbrush central contest winner'\n",
        ",'ISO 200'\n",
        ",'Bob Ross'\n",
        ",'32k HUHD'\n",
        ",'Photocopy'\n",
        ",'DeviantArt HD'\n",
        ",'infrared'\n",
        ",'Angelic photograph'\n",
        ",'Demonic photograph'\n",
        ",'Biomorphic'\n",
        ",'Windows Vista'\n",
        ",'Skeuomorphic'\n",
        ",'Physically based rendering'\n",
        ",'Trance compilation CD'\n",
        ",'Concert poster'\n",
        ",'Steampunk'\n",
        ",'Sketchfab'\n",
        ",'Goth'\n",
        ",'Wiccan'\n",
        ",'trending on artstation'\n",
        ",'featured on artstation'\n",
        ",'artstation HQ'\n",
        ",'artstation contest winner'\n",
        ",'ultra HD'\n",
        ",'high quality photo'\n",
        ",'instax'\n",
        ",'ilford HP5'\n",
        ",'infrared'\n",
        ",'Lomo'\n",
        ",'Matte drawing'\n",
        ",'matte photo'\n",
        ",'glowing neon'\n",
        ",'Xbox 360 graphics'\n",
        ",'flickering light'\n",
        ",'Playstation 5 screenshot'\n",
        ",'Kodak Gold 200'\n",
        ",'by Edward Hopper'\n",
        ",'rough'\n",
        ",'maximalist'\n",
        ",'minimalist'\n",
        ",'Kodak Ektar'\n",
        ",'Kodak Portra'\n",
        ",'geometric'\n",
        ",'cluttered'\n",
        ",'Rococo'\n",
        ",'destructive'\n",
        ",'by James Gurney'\n",
        ",'by Thomas Kinkade'\n",
        ",'by Vincent Di Fate'\n",
        ",'by Jim Burns'\n",
        ",'androgynous'\n",
        ",'masculine'\n",
        ",'genderless'\n",
        ",'feminine'\n",
        ",'extremely gendered, masculine and feminine'\n",
        ",'4k result'\n",
        ",'#pixelart'\n",
        ",'voxel art'\n",
        ",'wimmelbilder'\n",
        ",'dystopian art'\n",
        ",'apocalypse art'\n",
        ",'apocalypse landscape'\n",
        ",'2D game art'\n",
        ",'Windows XP'\n",
        ",'y2k aesthetic'\n",
        ",'#screenshotsaturday'\n",
        ",'seapunk'\n",
        ",'vaporwave'\n",
        ",'Ilya Kuvshinov'\n",
        ",'Paul Cezanne'\n",
        ",'Henry Moore'\n",
        ",'phallic'\n",
        ",'creepypasta'\n",
        ",'retrowave'\n",
        ",'synthwave'\n",
        ",'outrun'*"
      ]
    }
  ]
}