examples/run_advanced.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0754861b",
   "metadata": {},
   "source": [
    "Linajea Tracking Example\n",
    "=====================\n",
    "\n",
    "This example show all steps necessary to generate the final tracks, from training the network to finding the optimal ILP weights on the validation data to computing the tracks on the test data.\n",
    "Before you start, make sure that the `linajea` package is installed. For information on how to do that, please refer to the readme (https://github.com/funkelab/linajea/blob/master/README.md).\n",
    "\n",
    "- train network\n",
    "- predict on validation data\n",
    "- grid search weights for ILP\n",
    "  - solve once per set of weights\n",
    "  - evaluate once per set of weights\n",
    "  - select set with fewest errors\n",
    "- predict on test data\n",
    "- solve on test data with optimal weights\n",
    "- evaluate on test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5c7284e",
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "import logging\n",
    "import multiprocessing\n",
    "import os\n",
    "import shutil\n",
    "import sys\n",
    "import time\n",
    "import types\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from linajea.config import (dump_config,\n",
    "                            SolveParametersConfig,\n",
    "                            TrackingConfig)\n",
    "from linajea.utils import getNextInferenceData\n",
    "import linajea.evaluation\n",
    "from linajea.process_blockwise import (extract_edges_blockwise,\n",
    "                                       predict_blockwise,\n",
    "                                       solve_blockwise)\n",
    "from linajea.training import train\n",
    "import linajea.utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dffaf22c",
   "metadata": {},
   "outputs": [],
   "source": [
    "logging.basicConfig(\n",
    "        level=logging.INFO,\n",
    "        format='%(asctime)s %(name)s %(levelname)-8s %(message)s')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78bf70e7",
   "metadata": {},
   "source": [
    "Experiment\n",
    "-----------------\n",
    "\n",
    "To start a new experiment create a new folder and copy the configuration file(s) you want to use into this folder.\n",
    "For this example we have already done this for you (`example_advanced`). Then change the current working directory to that folder.\n",
    "Make sure that the file paths contained in the configuration files point to the correct destination, for instance that they are adapted to your directory structure. And that `config.general.setup_dir` is set to the folder you just created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "286d8576",
   "metadata": {},
   "outputs": [],
   "source": [
    "setup_dir = \"example_advanced\"\n",
    "os.chdir(setup_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85fb4241",
   "metadata": {},
   "source": [
    "Setup\n",
    "--------\n",
    "\n",
    "Make sure that the `linajea` package is installed and that the correct kernel is selected in the jupyter notebook.\n",
    "\n",
    "\n",
    "### Data\n",
    "\n",
    "Set `download_data` to `True` and execute the next cell to download a subset of: *3D+time nuclei tracking dataset of confocal fluorescence microscopy time series of C. elegans embryos* (https://zenodo.org/record/6460303).\n",
    "\n",
    "Per step (train, validation, test) we will use multiple samples now (For demonstration purposes each 'sample' is a different set of frames from the whole dataset). For training we use frames 20-40 and frames 50-90 (while excluding frames 65-75) from the `mskcc_emb1` volume, for validation we use frames 30-45 and frames 50-65 from the `mskcc_emb2` volume und to test we use frames 50-65 from the `mskcc_emb3` volume.\n",
    "\n",
    "You can of course use your own data, it has to be in a compatible format (see https://github.com/funkelab/linajea/examples/README.md for more information)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8a76dcf",
   "metadata": {},
   "outputs": [],
   "source": [
    "download_data = False\n",
    "if download_data:\n",
    "    !wget -c https://figshare.com/ndownloader/files/36873687 -N --show-progress\n",
    "    !ln -s 36873687 mskcc_emb1_fr20-40.zip\n",
    "    !wget -c https://figshare.com/ndownloader/files/36873690 -N --show-progress\n",
    "    !mv 36873690 mskcc_emb1_fr20-40_tracks.csv\n",
    "\n",
    "    !wget -c https://figshare.com/ndownloader/files/36873714 -N --show-progress\n",
    "    !ln -s 36873714 mskcc_emb1_fr50-60_fr65-75_fr80-90.zip\n",
    "    !wget -c https://figshare.com/ndownloader/files/36873711 -N --show-progress\n",
    "    !mv 36873711 mskcc_emb1_fr50-90_tracks.csv\n",
    "\n",
    "    !wget -c https://figshare.com/ndownloader/files/36873756 -N --show-progress\n",
    "    !ln -s 36873756 mskcc_emb2_fr30-45.zip\n",
    "    !wget -c https://figshare.com/ndownloader/files/36873759 -N --show-progress\n",
    "    !mv 36873759 mskcc_emb2_fr30-45_tracks.csv\n",
    "  \n",
    "    !wget -c https://figshare.com/ndownloader/files/36873786 -N --show-progress\n",
    "    !ln -s 36873786 mskcc_emb2_fr50-65.zip\n",
    "    !wget -c https://figshare.com/ndownloader/files/36873789 -N --show-progress\n",
    "    !mv 36873789 mskcc_emb2_fr50-65_tracks.csv\n",
    "\n",
    "    !wget -c https://figshare.com/ndownloader/files/36873849 -N --show-progress\n",
    "    !ln -s 36873849 mskcc_emb3_fr50-65.zip\n",
    "    !wget -c https://figshare.com/ndownloader/files/36873852 -N --show-progress\n",
    "    !mv 36873852 mskcc_emb3_fr50-65_tracks.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f584c1e",
   "metadata": {},
   "source": [
    "### Database\n",
    "\n",
    "MongoDB is used to store the computed results. A `mongod` server has to be running before executing the remaining cells.\n",
    "See https://www.mongodb.com/docs/manual/administration/install-community/ for a guide on how to install it (Linux/Windows/MacOS).\n",
    "Alternatively you might want to create a singularity image (https://github.com/singularityhub/mongo). This can be used locally, too, but will be necessary if you want to run the code on an HPC cluster and there is no server installed already.\n",
    "\n",
    "Set `setup_databases` to `True` to add the ground truth tracks to the database server. Make sure to set `db_host` to the correct server (if you run it locally you can usually just set it to `\"localhost\"`). If you use different data you also have to adapt `csv_tracks_file` and `db_name`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c718d8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "setup_databases = False\n",
    "db_host = \"localhost\"\n",
    "if setup_databases:\n",
    "    csv_tracks_file = \"mskcc_emb1_fr20-40_tracks.csv\"\n",
    "    db_name = \"linajea_mskcc_emb1_fr20-40_gt\"\n",
    "    linajea.utils.add_tracks_to_database(\n",
    "        csv_tracks_file,\n",
    "        db_name,\n",
    "        db_host)\n",
    "    \n",
    "    csv_tracks_file = \"mskcc_emb1_fr50-90_tracks.csv\"\n",
    "    db_name = \"linajea_mskcc_emb1_fr50-60_fr65-75_fr80-90_gt\"\n",
    "    linajea.utils.add_tracks_to_database(\n",
    "        csv_tracks_file,\n",
    "        db_name,\n",
    "        db_host)\n",
    "    \n",
    "    csv_tracks_file = \"mskcc_emb2_fr30-45_tracks.csv\"\n",
    "    db_name = \"linajea_mskcc_emb2_fr30-45_gt\"\n",
    "    linajea.utils.add_tracks_to_database(\n",
    "        csv_tracks_file,\n",
    "        db_name,\n",
    "        db_host)\n",
    "    \n",
    "    csv_tracks_file = \"mskcc_emb2_fr50-65_tracks.csv\"\n",
    "    db_name = \"linajea_mskcc_emb2_fr50-65_gt\"\n",
    "    linajea.utils.add_tracks_to_database(\n",
    "        csv_tracks_file,\n",
    "        db_name,\n",
    "        db_host)\n",
    "    \n",
    "    csv_tracks_file = \"mskcc_emb3_fr50-65_tracks.csv\"\n",
    "    db_name = \"linajea_mskcc_emb3_fr50-65_gt\"\n",
    "    linajea.utils.add_tracks_to_database(\n",
    "        csv_tracks_file,\n",
    "        db_name,\n",
    "        db_host)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "428b2706",
   "metadata": {},
   "source": [
    "Configuration\n",
    "--------------------\n",
    "\n",
    "All parameters to control the pipeline (e.g. model architecture, data augmentation, training parameters, ILP weights) are contained in a configuration file (in the TOML format https://toml.io)\n",
    "\n",
    "You can use a single monolithic configuration file or separate configuration files for a subset of the steps of the pipeline, as long as the parameters required for the respective steps are there.\n",
    "\n",
    "Familiarize yourself with the example configuration files and have a look at the documentation for the configuration to see what is needed. Most parameters have sensible defaults; usually setting the correct paths and the data configuration is all that is needed to start. See `run_simple.ipynb` for a simpler example setup that can only handle one sample/volume per dataset and that requires manual selection of the data used in the individual steps.\n",
    "\n",
    "In this setup `train_data`, `val_data` and `test_data` have to be set once and depending on the processing step the correct data is selected automatically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a28ca24d",
   "metadata": {},
   "outputs": [],
   "source": [
    "config_file = \"config.toml\"\n",
    "config = TrackingConfig.from_file(config_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79e4a402",
   "metadata": {},
   "source": [
    "Training\n",
    "------------\n",
    "\n",
    "To start training simply pass the configuration object to the train function. Make sure that the training data and parameters such as the number of iterations/setps are set correctly.\n",
    "\n",
    "To train until convergence will take from several hours to multiple days."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2ae694b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# done in child process to automatically free cuda resources\n",
    "p = multiprocessing.Process(target=train, args=(config,))\n",
    "p.start()\n",
    "p.join()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7983522",
   "metadata": {},
   "source": [
    "As training until convergence will take a while we provide a pretrained model that can be used to test the following steps of the tracking pipeline.\n",
    "To use the pretained model set `use_pretrained` to `True` and execute the next cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "30f3152f",
   "metadata": {},
   "outputs": [],
   "source": [
    "use_pretrained = False\n",
    "if use_pretrained:\n",
    "    urls = [\"https://figshare.com/ndownloader/files/36939976\",\n",
    "            \"https://figshare.com/ndownloader/files/36939550\"]\n",
    "    assert len(config.validate_data.checkpoints) <= len(urls), \\\n",
    "        f\"only {len(urls)} pretrained checkpoints available for testing\"\n",
    "    for url, chkpt in zip(urls, config.validate_data.checkpoints):\n",
    "        !wget $url -N --show-progress\n",
    "        shutil.copy2(url.split(\"/\")[-1], f\"train_net_checkpoint_{chkpt}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "213c022b",
   "metadata": {},
   "source": [
    "Validation\n",
    "--------------\n",
    "\n",
    "After the training is completed we first have to determine the optimal ILP weights.\n",
    "This is achieved by first creating the prediction on the validation data and then performing a grid search by solving the ILP and evaluating the results repeatedly.\n",
    "\n",
    "`getNextInferenceData` can be used to loop over all samples in a dataset, it returns a generator.\n",
    "If `validation` is set to `True` in `args` the validation data is used, otherwise the test data. Other details (e.g. which training checkpoint to use, which database to store the results in to use) are determined automatically based on the configuration file. Internally it adds an `inference_data` entry that is used by the postprocessing functions such as `*_blockwise` and `evaluate_setup`. This entry is updated automatically after each iteration to point to the correct sample (and to use the correct checkpoint)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3679e44f",
   "metadata": {},
   "outputs": [],
   "source": [
    "val_args = types.SimpleNamespace(\n",
    "    config=config_file, validation=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a240c93",
   "metadata": {},
   "source": [
    "### Predict Validation Data\n",
    "\n",
    "First we predict the `cell_indicator` and `movement_vectors` on the validation data. Make sure that `args.validation` is set to `True`, then execute the next cell. The extracted maxima of the `cell_indicator` map correspond to potential cells in our candidate graph.\n",
    "\n",
    "This command starts a number of workers (`predict.job.num_workers`) in the background, each worker tries to access a GPU. Do not start more workers than GPUs available. By default the workers are started locally. If you are working on a compute cluster (`lsf` supported, `slurm` and `gridengine` experimental) set `predict.job.run_on` to the respective string value, the code will communicate with the cluster scheduler and allocate the appropriate jobs.\n",
    "\n",
    "Depending on the number of workers used (see config file) and the size of the data this can take a while. If there is no progress for a while check the log files in `<setup_dir>/daisy_logs/linajea_prediction`!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4e9124e3",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for inf_config in linajea.utils.getNextInferenceData(val_args):\n",
    "    predict_blockwise(inf_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ba90ec2",
   "metadata": {},
   "source": [
    "### Extract Edges Validation Data\n",
    "\n",
    "In the next step we extract potential edges for our candidate graph. For each cell candidate, look for neighboring cells in the next time frame and insert an edge candidate for each into the database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e8e3ca0",
   "metadata": {},
   "outputs": [],
   "source": [
    "for inf_config in linajea.utils.getNextInferenceData(val_args):\n",
    "    extract_edges_blockwise(inf_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b032f671",
   "metadata": {},
   "source": [
    "### ILP Weights Grid Search\n",
    "\n",
    "Cell/Node and edge candidates form together our candidate graph. By solving the ILP we extract tracks from this graph. However the ILP is parameterized by a set of weights. First we have to find the optimal values for these weights. To achieve this we perform a grid search over a predefined search space. For each set of parameter candidates we solve the ILP once on the validation data.\n",
    "\n",
    "\n",
    "#### Solve on Validation Data\n",
    "\n",
    "Make sure that `solve.grid_search` is set to `True` and that the search space (`solve.parameters_search_grid`) is defined. The parameter sets to try are then generated automatically. (Alternatively you can set `solve.parameters` to a list of parameter sets to try manually, as in `example_basic`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b79eae3",
   "metadata": {},
   "outputs": [],
   "source": [
    "config.solve.grid_search = True\n",
    "config.solve.parameters = None\n",
    "val_args.config = dump_config(config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a4fc5ae",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for inf_config in linajea.utils.getNextInferenceData(val_args, is_solve=True):\n",
    "    linajea.process_blockwise.solve_blockwise(inf_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1a28d14",
   "metadata": {},
   "source": [
    "#### Evaluate on Validation Data\n",
    "\n",
    "And as a last validation step we evaluate the performance for each set of parameter candidates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0bcc596e",
   "metadata": {},
   "outputs": [],
   "source": [
    "for inf_config in linajea.utils.getNextInferenceData(val_args, is_evaluate=True):\n",
    "    linajea.evaluation.evaluate_setup(inf_config)\n",
    "    parameters = inf_config.solve.parameters[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d427d9a7",
   "metadata": {},
   "source": [
    "#### Determine best ILP weights\n",
    "\n",
    "The set of weights/parameters resulting in the best performance (fewest number of errors) will then be used to get the performance on the test set. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1c7fe27",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "config.solve.grid_search = False\n",
    "val_args.config = dump_config(config)\n",
    "\n",
    "score_columns = ['fn_edges', 'identity_switches',\n",
    "                 'fp_divisions', 'fn_divisions']\n",
    "if not config.general.sparse:\n",
    "    score_columns = ['fp_edges'] + score_columns\n",
    "\n",
    "sort_by = \"sum_errors\"\n",
    "results = {}\n",
    "\n",
    "samples = set()\n",
    "for inf_config in linajea.utils.getNextInferenceData(val_args):\n",
    "    sample = inf_config.inference_data.data_source.datafile.filename\n",
    "    checkpoint = inf_config.inference_data.checkpoint\n",
    "    cell_score_threshold = inf_config.inference_data.cell_score_threshold\n",
    "    samples.add(sample)\n",
    "    print(\"getting results for:\", sample, checkpoint, cell_score_threshold)\n",
    "    res = linajea.evaluation.get_results_sorted(\n",
    "        inf_config,\n",
    "        filter_params={\"val\": True},\n",
    "        score_columns=score_columns,\n",
    "        sort_by=sort_by)\n",
    "\n",
    "    res = res.assign(checkpoint=checkpoint).assign(cell_score_threshold=cell_score_threshold)\n",
    "    results[(os.path.basename(sample), checkpoint, cell_score_threshold)] = res.reset_index()\n",
    "\n",
    "results = pd.concat(list(results.values())).reset_index()\n",
    "del results['param_id']\n",
    "del results['_id']\n",
    "\n",
    "by = [\n",
    "    \"matching_threshold\",\n",
    "    \"weight_node_score\",\n",
    "    \"selection_constant\",\n",
    "    \"track_cost\",\n",
    "    \"weight_division\",\n",
    "    \"division_constant\",\n",
    "    \"weight_child\",\n",
    "    \"weight_continuation\",\n",
    "    \"weight_edge_score\",\n",
    "    \"checkpoint\",\n",
    "    \"cell_score_threshold\"\n",
    "]\n",
    "if \"cell_state_key\" in results:\n",
    "    by.append(\"cell_state_key\")\n",
    "\n",
    "results = results.groupby(by, dropna=False, as_index=False).agg(\n",
    "    lambda x: -1 if len(x) != len(samples) else sum(x))\n",
    "results = results[results.sum_errors != -1]\n",
    "results.sort_values(sort_by, ascending=True, inplace=True)\n",
    "\n",
    "parameters.weight_node_score = float(results.at[0, 'weight_node_score'])\n",
    "parameters.selection_constant = float(results.at[0, 'selection_constant'])\n",
    "parameters.track_cost = float(results.at[0, 'track_cost'])\n",
    "parameters.weight_edge_score = float(results.at[0, 'weight_edge_score'])\n",
    "parameters.weight_division = float(results.at[0, 'weight_division'])\n",
    "parameters.weight_child = float(results.at[0, 'weight_child'])\n",
    "parameters.weight_continuation = float(results.at[0, 'weight_continuation'])\n",
    "\n",
    "cell_score_threshold = float(results.at[0, 'cell_score_threshold'])\n",
    "checkpoint = int(results.at[0, 'checkpoint'])\n",
    "print(\"Best parameters:\\n\", parameters)\n",
    "print(\"Best model checkpoint:\\n\", checkpoint)\n",
    "print(f\"(used cell_score_threshold : {cell_score_threshold})\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e92993ea",
   "metadata": {},
   "source": [
    "Test\n",
    "------\n",
    "\n",
    "Now that we know which ILP weights to use we can create the candidate graph on the test data and compute the tracks. \n",
    "\n",
    "First load the test configuration file and set the parameters to the previously determined values (alternatively set the values manually directly in the configuration file). \n",
    "Make sure that `args.validation` is set to `False` and that `solve.grid_search` and `solve.random_search` are not set or set to `False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e396ec9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "config.solve.parameters = [parameters]\n",
    "config.solve.grid_search = False\n",
    "config.solve.random_search = False\n",
    "\n",
    "config.test_data.cell_score_threshold = cell_score_threshold\n",
    "config.test_data.checkpoint = checkpoint\n",
    "\n",
    "test_args = types.SimpleNamespace()\n",
    "test_args.config = dump_config(config)\n",
    "test_args.validation = False"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d174b602",
   "metadata": {},
   "source": [
    "### Predict Test Data\n",
    "\n",
    "Now that we know which ILP weights to use we can predict the `cell_indicator` and `movement_vectors` on the test data and compute the tracks. Make sure that `args.validation` is set to `False` and `solve.grid_search` and `solve.random_search` are set to `False`.\n",
    "If there is no progress for a while check the log files in `<setup_dir>/daisy_logs/linajea_prediction`!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73142808",
   "metadata": {},
   "outputs": [],
   "source": [
    "for inf_config in linajea.utils.getNextInferenceData(test_args):\n",
    "    predict_blockwise(inf_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82c29d00",
   "metadata": {},
   "source": [
    "### Extract Edges on Test Data\n",
    "\n",
    "In the next step we extract again the potential edges for our candidate graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bdfbfc6a",
   "metadata": {},
   "outputs": [],
   "source": [
    "for inf_config in linajea.utils.getNextInferenceData(test_args):\n",
    "    extract_edges_blockwise(inf_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25c0660d",
   "metadata": {},
   "source": [
    "### Solve on Test Data\n",
    "\n",
    "Then we can solve the ILP on the test data. We select the weights that resulted in the lowest overall number of errors on the validation data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a141c33",
   "metadata": {},
   "outputs": [],
   "source": [
    "for inf_config in linajea.utils.getNextInferenceData(test_args, is_solve=True):\n",
    "    solve_blockwise(inf_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4448cb8d",
   "metadata": {},
   "source": [
    "### Evaluate on Test Data\n",
    "\n",
    "And finally we can evaluate the performance of our tracks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "03698bce",
   "metadata": {},
   "outputs": [],
   "source": [
    "for inf_config in linajea.utils.getNextInferenceData(test_args, is_evaluate=True):\n",
    "    report = linajea.evaluation.evaluate_setup(inf_config)\n",
    "    for k, v in report.get_short_report().items():\n",
    "        print(f\"\\t{k: <32}: {v}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}