hello_world_submit_job.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PGE Executor\n",
    "This notebook executes the SDS job that was created by the *hello_world_pge_create* notebook. In this driver the user can select his/her own parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. Initialization\n",
    "Perform imports, initialize the SDS API if necessary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from pathlib import Path\n",
    "from pprint import pprint\n",
    "import otello\n",
    "\n",
    "if not os.path.exists(f\"{Path.home()}/.config/otello/config.yml\"):\n",
    "    otello.client.initialize()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. Create a handle to the SDS endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = otello.mozart.Mozart()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. Specify and initialize the job type.\n",
    "Note that here you may need to update the tag (i.e. the portion following the colon) to match the repository branch that was used in *hello_world_pge_create* notebook. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'queues': {'queues': ['factotum-job_worker-large',\n",
       "   'factotum-job_worker-small',\n",
       "   'import_prov_es',\n",
       "   'nisar-job_worker-large',\n",
       "   'nisar-job_worker-small'],\n",
       "  'recommended': ['factotum-job_worker-small']},\n",
       " 'default': 'factotum-job_worker-small'}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "job_type='job-hello_world_notebook:develop'\n",
    "hw_job_def = m.get_job_types()[job_type]\n",
    "hw_job_def.initialize()\n",
    "hw_job_def.get_queues()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. Show some information about the job, specifically, the input parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Job Type: job-hello_world_notebook:develop\n",
      "Label: Hello world notebook PGE\n",
      "\n",
      "\n",
      "Tunable Parameters:\n",
      "\tname: name\n",
      "\tdefault: jimmie\n",
      "\n",
      "\tname: start_orbit_number\n",
      "\tdefault: 5020A\n",
      "\n",
      "\tname: end_orbit_number\n",
      "\tdefault: 5028D\n",
      "\n",
      "\tname: periapsis\n",
      "\tdefault: 3668.75\n",
      "\n",
      "\tname: apoapsis\n",
      "\tdefault: 18699.23\n",
      "\n",
      "\n",
      "Dataset Parameters:\n",
      "\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(hw_job_def.describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5. Display the current values of the input parameters (i.e. the default values)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'apoapsis': 18699.23,\n",
      " 'end_orbit_number': '5028D',\n",
      " 'name': 'jimmie',\n",
      " 'periapsis': 3668.75,\n",
      " 'start_orbit_number': '5020A'}\n"
     ]
    }
   ],
   "source": [
    "pprint(hw_job_def.get_input_params())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 6. Specify custom parameter values.\n",
    "\n",
    "Here is where user-defined parameter values are specified, making sure to remain consistent with value types as indicated in the *hello_world_notebook*. Default values are used where none are provided (as we've done here with *start_orbit_number*). *set_input_params* is called to pass the parameter values to the job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'apoapsis': 4299.68,\n",
      " 'end_orbit_number': '5042D',\n",
      " 'name': 'Barney Rubble',\n",
      " 'periapsis': 7428.99,\n",
      " 'start_orbit_number': '5020A'}\n"
     ]
    }
   ],
   "source": [
    "params = {\n",
    "    'name' : 'Barney Rubble',\n",
    "    'end_orbit_number' : '5042D',\n",
    "    'apoapsis' : 4299.68,\n",
    "    'periapsis' : 7428.99\n",
    "}\n",
    "    \n",
    "hw_job_def.set_input_params(params)\n",
    "pprint(hw_job_def.get_input_params())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 7. Submit the job\n",
    "A job tag (useful for filtering on the job console) and job queue are specified. Either are optional. Job submission is performed asynchronously, so this call will return almost immediately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "job_run = hw_job_def.submit_job(tag=\"hello_from_jupyter\", queue=\"nisar-job_worker-small\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 8. Wait for the job to complete.\n",
    "Information about the job state will print periodically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3c547eb0-68ff-4f01-975f-8d1f5d3d6db7: job-completed 2021-02-03T23:59:29.605204\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'job-completed'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "job_run.wait_for_completion()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once completed, the results can be viewed on the job console, which should appear similar to the following:\n",
    "\n",
    "``` 2021-02-03 21:40:43 [INFO] Executing notebook with kernel: python3\n",
    "2021-02-03 21:40:43 [INFO] Executing Cell 1---------------------------------------`\n",
    "2021-02-03 21:40:43 [INFO] Ending Cell 1------------------------------------------\n",
    "Executing:  12%|█▎        | 1/8 [00:00<00:06,  1.09cell/s]2021-02-03 21:40:43 [INFO] Executing Cell 2---------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Ending Cell 2------------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Executing Cell 3---------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Ending Cell 3------------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Executing Cell 4---------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Ending Cell 4------------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Executing Cell 5---------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Hello, Barney Rubble\n",
    "Processing for orbits from 5020A to 5041D\n",
    "\n",
    "2021-02-03 21:40:43 [INFO] Ending Cell 5------------------------------------------\n",
    " 2021-02-03 21:40:43 [INFO] Executing Cell 6---------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] apoapsis = 9766.43 of type <class 'float'>\n",
    "periapsis = 3668.75 of type <class 'float'>\n",
    "diff is 6097.68\n",
    "\n",
    "2021-02-03 21:40:43 [INFO] Ending Cell 6------------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Executing Cell 7---------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Goodbye, Barney Rubble\n",
    "\n",
    "2021-02-03 21:40:43 [INFO] Ending Cell 7------------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Executing Cell 8---------------------------------------\n",
    "2021-02-03 21:40:43 [INFO] Ending Cell 8------------------------------------------\n",
    "Executing: 100%|██████████| 8/8 [00:01<00:00,  6.57cell/s]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}