Examples/GrowingNeuralCellularAutomata/GrowingNeuralCellularAutomata.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "GrowingNeuralCellularAutomata.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Swift",
      "language": "swift",
      "name": "swift"
    },
    "language_info": {
      "file_extension": ".swift",
      "mimetype": "text/x-swift",
      "name": "swift",
      "version": ""
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9TV7IYeqifSv"
      },
      "source": [
        "##### Copyright 2020 The TensorFlow Authors. [Licensed under the Apache License, Version 2.0](#scrollTo=ByZjmtFgB_Y5)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kZRlD4utdPuX"
      },
      "source": [
        "%install-location $cwd/swift-install\n",
        "%install '.package(url: \"https://github.com/tensorflow/swift-models\", .branch(\"main\"))' ModelSupport\n",
        "print(\"\\u{001B}[2J\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tRIJp_4m_Afz"
      },
      "source": [
        "// #@title Licensed under the Apache License, Version 2.0 (the \"License\"); { display-mode: \"form\" }\n",
        "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "// you may not use this file except in compliance with the License.\n",
        "// You may obtain a copy of the License at\n",
        "//\n",
        "// https://www.apache.org/licenses/LICENSE-2.0\n",
        "//\n",
        "// Unless required by applicable law or agreed to in writing, software\n",
        "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "// See the License for the specific language governing permissions and\n",
        "// limitations under the License."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sI1ZtrdiA4aY"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        " <td>\n",
        "  <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/swift-models/blob/main/Examples/GrowingNeuralCellularAutomata/GrowingNeuralCellularAutomata.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        " </td>\n",
        " <td>\n",
        "  <a target=\"_blank\" href=\"https://github.com/tensorflow/swift-models/blob/main/Examples/GrowingNeuralCellularAutomata/GrowingNeuralCellularAutomata.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        " </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8sa42_NblqRE"
      },
      "source": [
        "# Growing Neural Cellular Automata\n",
        "\n",
        "This is an implementation in Swift for TensorFlow of the experiments described in [\"Growing Neural Cellular Automata\"](https://distill.pub/2020/growing-ca/) by Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson, and Michael Levin. Currently, only Experiment 1 has been completed.\n",
        "\n",
        "In this publication, cellular automata have a rule that is trained via gradient descent to cause a single cell to grow into a larger image, stabilize at a final shape, and repair damage to that image. The rule used for updates on each step is defined by a simple neural network, trained using gradient descent to produce a rule that can grow into a target image."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W7MpNcIwIIy8"
      },
      "source": [
        "## Device setup and model parameters\n",
        "\n",
        "We'll start by importing the appropriate modules:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OHRTNQJo1TxT"
      },
      "source": [
        "import Foundation\n",
        "import TensorFlow\n",
        "import ModelSupport"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W5UYUqsW1oo8"
      },
      "source": [
        "Next, we'll configure the accelerator the tensor operations will run on. For best compatibility (TPU + GPU), we'll use XLA through Swift for TensorFlow's X10 backend. The eager-mode runtime can also be used, and may provide better performance on GPUs at present:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FCMWR11NIIy-"
      },
      "source": [
        "let device = Device.defaultTFEager\n",
        "// let device = Device.defaultXLA\n",
        "// device"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rh4WcQzIIvGa"
      },
      "source": [
        "To aid us in displaying images within the notebook, we'll use Swift's Python interoperability to set up an image display function."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W0IeaDuyIvaJ"
      },
      "source": [
        "import PythonKit\n",
        "\n",
        "%include \"EnableIPythonDisplay.swift\"\n",
        "IPythonDisplay.shell.enable_matplotlib(\"inline\")\n",
        "let display = Python.import(\"IPython.core.display\")\n",
        "\n",
        "func showImageFile(_ filename: String) {\n",
        "  display.Image(Python.open(filename, \"rb\").read()).display()\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HrlMNOinIIy_"
      },
      "source": [
        "The following contains all model parameters used during training:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NrRQhQaHaJm9"
      },
      "source": [
        "// The height and width to use when resizing the input image.\n",
        "let imageSize = 40\n",
        "// The padding to add around the input image after resizing.\n",
        "let padding = 16\n",
        "// The number of state channels for each cell.\n",
        "let stateChannels = 16\n",
        "// The batch size during training.\n",
        "let batchSize = 4\n",
        "// The fraction of cells to fire at each update.\n",
        "let cellFireRate: Float = 0.5\n",
        "// The number of training iterations.\n",
        "let iterations = 2000\n",
        "// The minimum number of steps.\n",
        "let minimumSteps = 64\n",
        "// The maximum number of steps.\n",
        "let maximumSteps = 96\n",
        "// The number of steps to run through during inference.\n",
        "let inferenceSteps = 200"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rU0WY_sJodio"
      },
      "source": [
        "## Configuring the cell update rule\n",
        "\n",
        "The cell update rule is computed by a neural network that takes in the current state (batch size x height x width x state channels) and outputs a new state to use for the next time step. At its first stage, horizontal and vertical Sobel kernels are applied to the 3x3 neighborhood around a cell, and those results, as well as the cell's current state, are passed into the network. By default, a cell's state consists of red, green, blue, and alpha color components along with 12 hidden parameters.\n",
        "\n",
        "The network itself has two 1x1 convolutional layers, with a ReLU activation after the first. Only a fraction of the cells are updated at a given time step, and any cell with an alpha value less than 10% is considered \"dead\" and ignored.\n",
        "\n",
        "The following diagram from [\"Growing Neural Cellular Automata\"](https://distill.pub/2020/growing-ca/) explains the model structure and function:\n",
        "\n",
        "![Cell rule model](https://distill.pub/2020/growing-ca/figures/model.svg)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i4MSHK5YDiAQ"
      },
      "source": [
        "The first component we'll implement will be the perception function. We'll cache the horizontal and vertical Sobel kernel tensors so that they can be reused on each iteration and save on host -> device transfers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kqXILiXhq-iM"
      },
      "source": [
        "let horizontalSobelKernel = Tensor<Float>(\n",
        "  shape: [3, 3, 1, 1], scalars: [-1.0, 0.0, 1.0, -2.0, 0.0, 2.0, -1.0, 0.0, 1.0], on: device) / 8.0\n",
        "let horizontalSobelFilter = horizontalSobelKernel.broadcasted(to: [3, 3, stateChannels, 1])\n",
        "let verticalSobelKernel = Tensor<Float>(\n",
        "  shape: [3, 3, 1, 1], scalars: [-1.0, -2.0, -1.0, 0.0, 0.0, 0.0, 1.0, 2.0, 1.0], on: device) / 8.0\n",
        "let verticalSobelFilter = verticalSobelKernel.broadcasted(to: [3, 3, stateChannels, 1])\n",
        "let identityKernel = Tensor<Float>(\n",
        "  shape: [3, 3, 1, 1], scalars: [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], on: device)\n",
        "let identityFilter = identityKernel.broadcasted(to: [3, 3, stateChannels, 1])\n",
        "let perceptionFilter = Tensor(\n",
        "  concatenating: [horizontalSobelFilter, verticalSobelFilter, identityFilter], alongAxis: 3)\n",
        "\n",
        "@differentiable\n",
        "func perceive(_ input: Tensor<Float>) -> Tensor<Float> {\n",
        "  return depthwiseConv2D(\n",
        "    input, filter: perceptionFilter, strides: (1, 1, 1, 1), padding: .same)\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SMOU77oLiyFx"
      },
      "source": [
        "As a convenience, we'll implement extensions to Tensor that separate out just the RGBA color channels from the larger cell state, as well as mask active cells:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ve4_tuycixJ-"
      },
      "source": [
        "extension Tensor where Scalar: Numeric {\n",
        "  @differentiable(where Scalar: TensorFlowFloatingPoint)\n",
        "  var colorComponents: Tensor {\n",
        "    precondition(self.rank == 3 || self.rank == 4)\n",
        "    if self.rank == 3 {\n",
        "      return self.slice(\n",
        "        lowerBounds: [0, 0, 0], sizes: [self.shape[0], self.shape[1], 4])\n",
        "    } else {\n",
        "      return self.slice(\n",
        "        lowerBounds: [0, 0, 0, 0], sizes: [self.shape[0], self.shape[1], self.shape[2], 4])\n",
        "    }\n",
        "  }\n",
        "\n",
        "  func mask(condition: (Tensor) -> Tensor<Bool>) -> Tensor {\n",
        "    let satisfied = condition(self)\n",
        "    return Tensor(zerosLike: self)\n",
        "      .replacing(with: Tensor(onesLike: self), where: satisfied)\n",
        "  }\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cwNDwzS2QgS1"
      },
      "source": [
        "Next, we need the ability to mask off only the \"living\" cells  (those with an alpha channel above 0.1) and their neighbors:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fEAEyUExXT3I"
      },
      "source": [
        "@differentiable\n",
        "func livingMask(_ input: Tensor<Float>) -> Tensor<Float> {\n",
        "  let alphaChannel = input.slice(\n",
        "    lowerBounds: [0, 0, 0, 3], sizes: [input.shape[0], input.shape[1], input.shape[2], 1])\n",
        "  let localMaximum =\n",
        "    maxPool2D(alphaChannel, filterSize: (1, 3, 3, 1), strides: (1, 1, 1, 1), padding: .same)\n",
        "  return withoutDerivative(at: input) { _ in localMaximum.mask { $0 .> 0.1 } }\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1pMewsl0VgnJ"
      },
      "source": [
        "The cell update rule itself is encapsulated in a custom Layer, and can be called like a function. The steps in the rule are applied within `callAsFunction()`, and they follow the diagram above: "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eNzOdly3QY_P"
      },
      "source": [
        "struct CellRule: Layer {\n",
        "  @noDerivative let fireRate: Float\n",
        "  var conv1: Conv2D<Float>\n",
        "  var conv2: Conv2D<Float>\n",
        "\n",
        "  init(stateChannels: Int, fireRate: Float) {\n",
        "    self.fireRate = fireRate\n",
        "    self.conv1 = Conv2D<Float>(filterShape: (1, 1, stateChannels * 3, 128))\n",
        "    self.conv2 = Conv2D<Float>(\n",
        "      filterShape: (1, 1, 128, stateChannels), useBias: false, filterInitializer: zeros())\n",
        "  }\n",
        "\n",
        "  @differentiable\n",
        "  func callAsFunction(_ input: Tensor<Float>) -> Tensor<Float> {\n",
        "    // Perform the update and determine the change to be applied to the cell state.\n",
        "    let perception = perceive(input)\n",
        "    let dx = conv2(relu(conv1(perception)))\n",
        "\n",
        "    // Only fire a certain percentage of cells at each time step.\n",
        "    let updateDistribution = Tensor<Float>(\n",
        "      randomUniform: [input.shape[0], input.shape[1], input.shape[2], 1], on: input.device)\n",
        "    let updateMask = withoutDerivative(at: input) { _ in\n",
        "      updateDistribution.mask { $0 .< fireRate }\n",
        "    }\n",
        "\n",
        "    let updatedState = input + (dx * updateMask)\n",
        "\n",
        "    // Mask off \"dead\" cells in the new state and use the combined mask to zero out \"dead\" cells.\n",
        "    let combinedLivingMask = livingMask(input) * livingMask(updatedState)\n",
        "    return updatedState * combinedLivingMask\n",
        "  }\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Te7sNNx9c_am"
      },
      "source": [
        "## Training the cell rule\n",
        "\n",
        "The training loop starts from a single black cell in the center of the grid and applies the cell rule for between `minimumSteps` and `maximumSteps`. The resulting state is then compared along the red, green, and blue channels against a target image and loss calculated via mean squared error. A gradient is determined from this and the Adam optimizer updates the cell rules.\n",
        "\n",
        "The first step in this process is initializing our cell rule model and Adam optimizer, then moving both onto the appropriate accelerator device:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3yF4vWcBJTiz"
      },
      "source": [
        "var cellRule = CellRule(stateChannels: stateChannels, fireRate: cellFireRate)\n",
        "cellRule.move(to: device)\n",
        "var optimizer = Adam(for: cellRule, learningRate: 2e-3)\n",
        "optimizer = Adam(copying: optimizer, to: device)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ERP_1BL9JSvu"
      },
      "source": [
        "We'll load our target image into a Tensor, pad it, and convert that to a batch:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MaN7fM-lAe7r"
      },
      "source": [
        "let imageData = try! Data(contentsOf: URL(string: \"https://github.com/googlefonts/noto-emoji/raw/master/png/128/emoji_u1f98e.png\")!)\n",
        "try! imageData.write(to: URL(fileURLWithPath: \"lizard.png\"))\n",
        "\n",
        "let hostInputImage = Image(contentsOf: URL(fileURLWithPath: \"lizard.png\")).premultipliedAlpha()\n",
        "let resizedHostInputImage = hostInputImage.resized(to: (imageSize, imageSize))\n",
        "let inputImage = Tensor(copying: resizedHostInputImage.tensor, to: device) / 255.0\n",
        "let paddedImage = inputImage.padded(forSizes: [\n",
        "  (before: padding, after: padding), (before: padding, after: padding), (before: 0, after: 0),\n",
        "])\n",
        "let paddedImageBatch = paddedImage.broadcasted(to: [\n",
        "  batchSize, paddedImage.shape[0], paddedImage.shape[1], paddedImage.shape[2],\n",
        "])\n",
        "\n",
        "try paddedImage.scaled(by: 255.0).overlaidOnWhite()\n",
        "  .saveImage(directory: \"output\", name: \"targetimage\", format: .png)\n",
        "\n",
        "showImageFile(\"lizard.png\")\n",
        "showImageFile(\"output/targetimage.png\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w3lmTRCWT5sS"
      },
      "source": [
        "The initial cell state is set up once and then re-used:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JGG9FXE8K-3G"
      },
      "source": [
        "var initialState = Tensor(zerosLike: paddedImage).padded(forSizes: [\n",
        "  (before: 0, after: 0), (before: 0, after: 0), (before: 0, after: stateChannels - 4),\n",
        "])\n",
        "initialState[initialState.shape[0] / 2][initialState.shape[1] / 2][3] = Tensor<Float>(1.0, on: device)\n",
        "let initialBatch = initialState.broadcasted(to: [\n",
        "  batchSize, initialState.shape[0], initialState.shape[1], initialState.shape[2],\n",
        "])\n",
        "LazyTensorBarrier()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_BtTLecWL8N-"
      },
      "source": [
        "We normalize gradients to stabilize training:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VgDtt0cGMCQp"
      },
      "source": [
        "func normalizeGradient(_ gradient: CellRule.TangentVector) -> CellRule.TangentVector {\n",
        "  var outputGradient = gradient\n",
        "  for kp in gradient.recursivelyAllWritableKeyPaths(to: Tensor<Float>.self) {\n",
        "    let norm = sqrt(gradient[keyPath: kp].squared().sum())\n",
        "    outputGradient[keyPath: kp] = gradient[keyPath: kp] / (norm + 1e-8)\n",
        "  }\n",
        "  \n",
        "  return outputGradient\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XAOk0to6tbE1"
      },
      "source": [
        "Due to the way that the X10 backend traces out subgraphs, we need to prevent the iterated cell computation from being fully unrolled on the backward pass. To do this, we'll introduce a passthrough function that has a custom derivative which stops the trace at that point:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VUk-Qgs3t2UK"
      },
      "source": [
        "@inlinable\n",
        "@differentiable\n",
        "func clipBackwardsTrace(_ input: Tensor<Float>) -> Tensor<Float> {\n",
        "  return input\n",
        "}\n",
        "\n",
        "@inlinable\n",
        "@derivative(of: clipBackwardsTrace)\n",
        "func _vjpClipBackwardsTrace(\n",
        "  _ input: Tensor<Float>\n",
        ") -> (value: Tensor<Float>, pullback: (Tensor<Float>) -> Tensor<Float>) {\n",
        "  return (input, { \n",
        "    LazyTensorBarrier()\n",
        "    return $0\n",
        "    }\n",
        "  )\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LbHMo7AKnmI6"
      },
      "source": [
        "In order to visualize the evolution of the cell states over time with a given rule, we'll set up a function that can perform inference for a series of time steps and record the result directly as an animated GIF:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HENf0G5pn9X0"
      },
      "source": [
        "func recordGrowth(\n",
        "  seed: Tensor<Float>, rule: CellRule, steps: Int, directory: String, filename: String\n",
        ") throws {\n",
        "  var state = seed\n",
        "  var states: [Tensor<Float>] = []\n",
        "  LazyTensorBarrier()\n",
        "  for _ in 0..<steps {\n",
        "    state = rule(state)\n",
        "    let sampledState = state[0]\n",
        "    LazyTensorBarrier()\n",
        "    states.append(sampledState.colorComponents * 255.0)\n",
        "  }\n",
        "  try states.saveAnimatedImage(directory: directory, name: filename, delay: 1)\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kZawqO2oLQBH"
      },
      "source": [
        "Finally, we train the model in a loop for `iterations`, and capture and display a representative end state every 10 iterations.\n",
        "\n",
        "Note: currently Swift Jupyter kernels do not support inline display of images during training, so the animated results will be displayed once the calculation has finished. The intermediate results are present as GIF in the output/ directory (on Colab, accessible via the file browser on the left-hand side of the screen)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W9bUsiOxVf_v"
      },
      "source": [
        "for iteration in 0..<iterations {\n",
        "  let startTime = Date()\n",
        "  let steps = Int.random(in: minimumSteps...maximumSteps)\n",
        "  var loggingState = initialState\n",
        "  let (loss, ruleGradient) = valueWithGradient(at: cellRule) { model -> Tensor<Float> in\n",
        "    var state = initialBatch\n",
        "    for _ in 0..<steps {\n",
        "      state = clipBackwardsTrace(state)\n",
        "      state = model(state)\n",
        "      LazyTensorBarrier()\n",
        "    }\n",
        "\n",
        "    loggingState = state[0]\n",
        "    return meanSquaredError(predicted: state.colorComponents, expected: paddedImageBatch)\n",
        "  }\n",
        "  optimizer.update(&cellRule, along: normalizeGradient(ruleGradient))\n",
        "  LazyTensorBarrier()\n",
        "\n",
        "  let lossScalar = loss.scalarized()\n",
        "  print(\n",
        "    \"Iteration: \\(iteration), loss: \\(lossScalar), log loss: \\(log10(lossScalar)), time: \\(Date().timeIntervalSince(startTime)) s\")\n",
        "\n",
        "  // Note: currently Swift Jupyter kernels cannot display images while a calculation is ongoing.\n",
        "  /*\n",
        "  if (iteration % 10) == 0 {\n",
        "    let filename = String(format: \"iteration%03d\", iteration)\n",
        "    try recordGrowth(\n",
        "      seed: initialState.expandingShape(at: 0), rule: cellRule, steps: inferenceSteps,\n",
        "      directory: \"output\", filename: filename)\n",
        "    display.clear_output(wait: true)\n",
        "    showImageFile(\"output/\\(filename).gif\")\n",
        "  }\n",
        "  */\n",
        "\n",
        "  if ((iteration + 1) % 2000) == 0 {\n",
        "    optimizer.learningRate = optimizer.learningRate * 0.1\n",
        "  }\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7ED0HGZW2gWY"
      },
      "source": [
        "Once the model has trained, we can perform inference and capture the evolution of the cell state over time:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jpcOByipc75O"
      },
      "source": [
        "try recordGrowth(\n",
        "  seed: initialState.expandingShape(at: 0), rule: cellRule, steps: inferenceSteps,\n",
        "  directory: \"output\", filename: \"growth\")\n",
        "showImageFile(\"output/growth.gif\")"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}