\n",
+ "Below we go through a few examples for how to use the MAST API to find observations planned for JWST (the GTO and ERS programs). We are going to do this by using [astroquery](https://astroquery.readthedocs.io/en/latest/mast/mast.html) on the MAST CAOM database. To avoid unintentional duplications, JWST proposers are required to check their proposed observations against those already approved. These tools may be useful in that process, and to find out more about what counts as a duplicate observations, see the [JWST documentation](https://jwst-docs.stsci.edu/display/JPP/JWST+Duplication+Checking).\n",
+ "\n",
+ "We begin by writing a couple of functions to query the MAST. The first function does a cone search on the database of JWST observations that have calibration level equal to -1 (proposed observations). You need only provide the RA, Dec and radius of your search. The second function does a name look-up using the Mast.Name.Lookup service to determine RA and Dec.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import astropy\n",
+ "from astroquery.mast import Mast\n",
+ "\n",
+ "def filteredConeSearch(ra,dec,radius,service=\"\",myfilters=None, returnNum=False):\n",
+ " \"\"\"\n",
+ " This function performs a cone search on the MAST CAOM database\n",
+ " and returns whether any observations overlap with the cone search\n",
+ " and other filters provided. This only searches planned observations.\n",
+ " \n",
+ " Args:\n",
+ " ra: right ascension in degrees\n",
+ " dec: declination in degrees\n",
+ " radius: radius of cone search in arc seconds\n",
+ " myfilters: Dictionary of what you want to filter on, \n",
+ " if None, it searhes mission=JWST and calib_level=-1\n",
+ " service: For testing you can change the MAST service to the testbed here.\n",
+ " returnNum: False. Set to True if you only want the number of observations returned.\n",
+ " \n",
+ " Returns:\n",
+ " results dictionary unless there are more than 1000 observations or less than 1.\n",
+ " \n",
+ " \"\"\"\n",
+ " \n",
+ " \n",
+ " if service==\"\":\n",
+ " service=\"Mast.Caom.Filtered.Position\"\n",
+ " \n",
+ " if myfilters!=None:\n",
+ " filters=myfilters\n",
+ " else:\n",
+ " filters = [\n",
+ " {\"paramName\":\"calib_level\",\n",
+ " \"values\":[\"-1\"],},\n",
+ " {\"paramName\":\"obs_collection\",\"values\":[\"JWST\"]}\n",
+ " ]\n",
+ " cone_search=\"%f, %f, %f\" % (ra,dec,radius/3600)\n",
+ " \n",
+ " #First see how many observations there are using COUNT_BIG(*)\n",
+ " params = { \"columns\":\"COUNT_BIG(*)\",\n",
+ " \"filters\":filters,\n",
+ " \"position\":cone_search\n",
+ " }\n",
+ " \n",
+ " result=Mast.service_request(service,params)\n",
+ " numbObs=int(result[0][0])\n",
+ " if (numbObs > 1000) | (numbObs == 0) | (returnNum):\n",
+ " return result\n",
+ " else:\n",
+ " print(\"Found: %u\" % numbObs)\n",
+ " \n",
+ " params = { \"columns\":\"*\",\n",
+ " \"filters\":filters,\n",
+ " \"position\":cone_search\n",
+ " }\n",
+ " \n",
+ " result=Mast.service_request(service,params)\n",
+ " \n",
+ " return result\n",
+ "\n",
+ "\n",
+ "def getMASTCoords(name):\n",
+ " \"\"\"\n",
+ " Use Mast.Name.Lookup to get the ra/dec of your target.\n",
+ " \"\"\"\n",
+ " service = 'Mast.Name.Lookup'\n",
+ " params ={'input':name,\n",
+ " 'format':'json'}\n",
+ " response = Mast.service_request_async(service,params)\n",
+ " result = response[0].json()\n",
+ " coord=result['resolvedCoordinate']\n",
+ " ra = coord[0]['ra']\n",
+ " dec = coord[0]['decl']\n",
+ " \n",
+ " return(ra,dec)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Example 1. Look-up TRAPPIST-1 and return all JWST observations on that target."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "target_name=\"Trappist-1\"\n",
+ "\n",
+ "(ra,dec) = getMASTCoords(target_name)\n",
+ "print(\"RA: %f, Dec: %f\" % (ra,dec))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Find planned JWST observations with the function we wrote above. \n",
+ "Print out the columns that may be of interest. Notice, it is still up to you to determine if these observations count as a duplicate with those you were planning. For instance, it does not provide the timing information necessary to determine which TRAPPIST-1 planet they are targetting. In some cases, the target_name or proposal title (obs_title) contains this information."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "service=\"Mast.Caom.Filtered.Position\"\n",
+ "radius = 10 #in arc seconds.\n",
+ "result=filteredConeSearch(ra,dec,radius,service=service)\n",
+ "print(result['target_name','proposal_pi','instrument_name','filters','t_exptime','proposal_id'])\n",
+ "print(\"\\nObservation Titles\")\n",
+ "print(result['obs_title'])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Example 1 cont. -- Find HST observations for Trappist-1 using the same function.\n",
+ "We need to build our own filters dictionary to send into the function. \n",
+ "In this case we want HST observations taken with the WFC3/IR instrument and the F139M filter. Note, limiting your search by the database \"filters\" column can be tricky for JWST because there is frequently more than one filter listed in the filters field. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "filters = [{\"paramName\":\"obs_collection\",\n",
+ " \"values\":[\"HST\"]},\n",
+ " { \"paramName\":\"instrument_name\",\n",
+ " \"values\":[\"WFC3/IR\"]},\n",
+ " {\"paramName\" : \"filters\",\n",
+ " \"values\" : [\"F139M\"]}]\n",
+ "\n",
+ "service=\"Mast.Caom.Filtered.Position\"\n",
+ "\n",
+ "result=filteredConeSearch(ra,dec,radius,service=service,myfilters=filters)\n",
+ "\n",
+ "print(result['target_name','proposal_pi','instrument_name','filters','t_exptime'])\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Example 2. Search for JWST planned observations around a list of stars with known planetary disks.\n",
+ "We will only ask for data taken with NIRCam. And we will ask our function to only return the number of observations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "pdlist = [\"Vega\", \"Deneb\",\"Fomalhaut\",\"HL Tauri\",\"Eta Corvi\",\"HD 15115\"]\n",
+ "\n",
+ "filters= [ {\"paramName\": \"calib_level\",\n",
+ " \"values\" :[\"-1\"],\n",
+ " \"paramName\": \"instrument_name\",\n",
+ " \"values\":[\"NIRCam\"]} \n",
+ "]\n",
+ "\n",
+ "service=\"Mast.Caom.Filtered.Position\"\n",
+ "\n",
+ "radius = 100 #arcseconds for cone search\n",
+ "numlist=[]\n",
+ "for pd in pdlist:\n",
+ " (ra,dec) = getMASTCoords(pd)\n",
+ " result=filteredConeSearch(ra,dec,radius,service=service,myfilters=filters,returnNum=True)\n",
+ " numbObs=int(result[0][0])\n",
+ " numlist.append(numbObs)\n",
+ "\n",
+ "print(pdlist)\n",
+ "print(numlist)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Check Details\n",
+ "Now I list the details of the observations for Fomalhaut."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "(ra,dec) = getMASTCoords('Fomalhaut')\n",
+ "result = filteredConeSearch(ra,dec,radius,service=service,myfilters=filters)\n",
+ "print(result['target_name','proposal_pi','instrument_name','filters','t_exptime'])"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.7"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/MAST/JWST/PlannedObs_Requests/Querying_JWST_in_mashup.ipynb b/notebooks/MAST/JWST/PlannedObs_Requests/Querying_JWST_in_mashup.ipynb
new file mode 100644
index 00000000..0121c7b8
--- /dev/null
+++ b/notebooks/MAST/JWST/PlannedObs_Requests/Querying_JWST_in_mashup.ipynb
@@ -0,0 +1,1064 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
Using the API to Query Planned JWST Observations in MAST
\n",
+ "
By Peter Forshay (pforshay@stsci.edu)
\n",
+ "
In this Notebook we'll take a look at how to use the MAST API to find planned JWST observations from GTO and ERS programs. We'll start with a basic filtered query to find all planned observations before moving on to constructing more specific requests:
Be aware that MAST queries can take some time to run and get a response, so please be patient when running cells back to back. If you encounter any errors, try to run the cell again when the previous cells have completed execution.
\n",
+ "
IMPORTANT DISCLAIMER: To avoid unintentional duplications, JWST proposers are required to check their proposed observations against those already approved. The tools provided here may be able to assist a user in identifying POTENTIAL observing conflicts for JWST programs. It remains the PI's responsibility to determine whether or not the result provided by these tools constitutes an actual conflict, in accordance with the JWST Duplicate Observations Policy.
Here is where we begin to customize the code to perform our own queries:
\n",
+ "
The first search below sends a filtered query to count all planned observations, designated by a calibration level of -1. In the 'data' entry, we can see 'Column1' returns with the number of results for the submitted query.
\n",
+ "
\n",
+ "
\"paramName\":\"calib_level\" allows us to filter based on calibration level.
\n",
+ "
\"values\":[\"-1\"] defines a planned observation.
First off, in order to send a filtered position search via the API, we'll need to submit the position in question in degrees. Converting a set of coordinates to degrees is made pretty easy by using the astropy.coordinates SkyCoord class.
Now that we have our RA and Dec available in degrees, we can define a radius (also in degrees) and submit a filtered position query. Recommended search radii for various JWST observing modes are provided within the JWST Duplication Policy information found on JDox. By default, we keep the \"columns\":\"COUNT_BIG(*)\" to simply return a count of the results, in case we hit a large number of entries.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def filtered_position_query(coordinates, radius=0.2, count=True):\n",
+ " \"\"\"\n",
+ " Construct a filtered position mashup request to send to the mastQuery module. Return \n",
+ " either the results of the query or just the results count.\n",
+ " \n",
+ " :param coordinates: Expects a pair of coordinates in degrees.\n",
+ " :type coordinates: tuple\n",
+ " \n",
+ " :param radius: Defines the radius to search around the designated coordinates within.\n",
+ " Also in degrees.\n",
+ " :type radius: float\n",
+ " \n",
+ " :param count: Flag to designate whether a full query is submitted, or just the count\n",
+ " results. Also affects the returned product. Defaults to True.\n",
+ " :type count: boolean\n",
+ " \"\"\"\n",
+ " \n",
+ " # Unpack the coordinates tuple\n",
+ " ra_deg, dec_deg = coordinates\n",
+ " \n",
+ " # Determine whether this is a full query or just a count\n",
+ " if count:\n",
+ " columns = \"COUNT_BIG(*)\" # \"COUNT_BIG(*)\" will only get a count of the results\n",
+ " else:\n",
+ " columns = \"*\"\n",
+ " \n",
+ " # Construct the mashup request\n",
+ " service = \"Mast.Caom.Filtered.Position\"\n",
+ " filters = [{\"paramName\":\"calib_level\", \"values\":[\"-1\"]},\n",
+ " {\"paramName\":\"obs_collection\", \"values\":[\"JWST\"]}\n",
+ " ]\n",
+ " position = \"{0}, {1}, {2}\".format(ra_deg, dec_deg, radius)\n",
+ " mashupRequest = {\"service\":service,\n",
+ " \"format\":\"json\",\n",
+ " \"params\":{\"columns\":columns, \n",
+ " \"filters\":filters,\n",
+ " \"position\":position\n",
+ " }\n",
+ " }\n",
+ "\n",
+ " # Send the query\n",
+ " headers,outString = mastQuery(mashupRequest)\n",
+ " queryResults = json.loads(outString)\n",
+ " \n",
+ " # Return either the full query results or just the results count\n",
+ " if count:\n",
+ " data = queryResults['data']\n",
+ " count = data[0]['Column1']\n",
+ " return count\n",
+ " else:\n",
+ " return queryResults"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "TEST_COUNT = filtered_position_query(SAMPLE_COORDS)\n",
+ "print(TEST_COUNT)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
In the above result we see we get 152 results, so we can go ahead and submit the full request.
If we display the first few results from the returned 'data' dictionary, we see we have a number of different properties available from these search results.
Now that we have some full query results available, we'll want to extract some of the relevant information. First, we'll want to see a list of any nearby pointings found by our position query. Next, we'll also want to see a list of which programs these pointings are associated with.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def analyze_query_results(our_target, queryResults):\n",
+ " \"\"\"\n",
+ " Perform analysis of the query results. Identify the nearby planned pointings and\n",
+ " identify the programs involved.\n",
+ " \n",
+ " :param our_target: Expects a pair of coordinates in degrees.\n",
+ " :type our_target: tuple\n",
+ " \n",
+ " :param queryResults: Full results from a MAST query.\n",
+ " :type queryResults: dictionary\n",
+ " \"\"\"\n",
+ " \n",
+ " # Set up initial variables\n",
+ " data = queryResults['data']\n",
+ " ra_target, dec_target = our_target\n",
+ " targets = {}\n",
+ " programs = []\n",
+ " \n",
+ " # Create a dictionary of all unique coordinate pairs along with a count of how many times they\n",
+ " # are found\n",
+ " for current in data:\n",
+ " current_program = current['proposal_id']\n",
+ " current_ra = current['s_ra']\n",
+ " current_dec = current['s_dec']\n",
+ " current_coords = (current_ra, current_dec)\n",
+ " if current_coords in targets.keys():\n",
+ " targets[current_coords] += 1\n",
+ " else:\n",
+ " targets[current_coords] = 1\n",
+ " programs.append(current_program)\n",
+ "\n",
+ " # For each unique coordinate pair, calculate the distance from the target and display our \n",
+ " # results\n",
+ " for x in sorted(targets.keys()):\n",
+ " num_obs = targets[x]\n",
+ " unique_ra, unique_dec = x\n",
+ " result = \"Found {0} planned observations at {1}, {2}\".format(num_obs, \n",
+ " unique_ra, \n",
+ " unique_dec)\n",
+ " distance_ra = abs(unique_ra - ra_target)\n",
+ " distance_dec = abs(unique_dec - dec_target)\n",
+ " distance = SkyCoord(distance_ra, distance_dec, frame=\"icrs\", unit='deg')\n",
+ " if distance_ra < 0.001 and distance_dec < 0.001: # Account for rounding differences\n",
+ " result += \" (target match)\"\n",
+ " else:\n",
+ " result += \" ({0} away)\".format(distance.to_string('hmsdms'))\n",
+ " print(result)\n",
+ " \n",
+ " # Output a link for each program found\n",
+ " for p in sorted(list(set(programs))):\n",
+ " address = \"https://jwst.stsci.edu/observing-programs/program-information?id={0}\".format(p)\n",
+ " print(\"Found planned observations in {0}: {1}\".format(p, address))\n",
+ " \n",
+ " # Return the dictionary of coordinates and number of observations\n",
+ " return targets"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "TEST_RESULTS = analyze_query_results(SAMPLE_COORDS, TEST_QUERY)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
Instead of simply listing associated proposal ID's, our module creates a link to the program information page for any associated proposals found in the area. On these pages, the user may access additional information on the program or the associated APT file itself. Examining the information provided on these program pages is a critical step to investigating any potential sources of conflict for your JWST observations.
This gives us a basic idea of how many observations are currently planned in the vicinity, but this does not necessarily mean these observations will conflict. We still need to check which instruments and configurations are being used. Much of this information is contained in the query results we've obtained, and would be more easily digestable in a CSV table.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import csv\n",
+ "\n",
+ "def write_to_csv_file(queryResults, filename):\n",
+ " \"\"\"\n",
+ " Write MAST query results to a .csv file.\n",
+ " \n",
+ " :param queryResults: Full results from a MAST query.\n",
+ " :type queryResults: dictionary\n",
+ " \n",
+ " :param filename: The desired filename to save the CSV table as.\n",
+ " :type filename: string\n",
+ " \"\"\"\n",
+ " \n",
+ " # Column names are stored in the 'fields' dictionary\n",
+ " fields = queryResults['fields']\n",
+ " header = [entry['name'] for entry in fields]\n",
+ "\n",
+ " # Use the DictWriter class to write the data dictionary to a .csv file\n",
+ " directory = os.getcwd()\n",
+ " filename = \"/\".join([directory, filename])\n",
+ " data = queryResults['data']\n",
+ " with open(filename, 'w') as output:\n",
+ " writer = csv.DictWriter(output, fieldnames=header)\n",
+ " writer.writeheader()\n",
+ " w = [writer.writerow(obs) for obs in data]\n",
+ " \n",
+ " # Return the filename created\n",
+ " print(\"Saved {0}\".format(filename))\n",
+ " return filename"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Choose a filename for the resulting CSV table\n",
+ "SAVE_AS = 'planned_obs_test.csv'\n",
+ "SAVED = write_to_csv_file(TEST_QUERY, SAVE_AS)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
Process multiple targets
\n",
+ "
We now have a CSV table with all parameters available through the Portal for all observations found within a 0.2 degree radius of a set of sample coordinates. The API allows us to now take this one step further and bring all these modules together and check multiple sets of coordinates back-to-back.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def check_multiple_targets(coordinates_list, coordinates_format, write):\n",
+ " \"\"\"\n",
+ " Handle the full process for checking a list of targets. This includes converting\n",
+ " to degrees if necessary, sending an initial count query, sending the full MAST\n",
+ " query, analyzing the results, and writing the results to a file.\n",
+ " \n",
+ " :param coordinates_list: Expects a list of coordinate tuples, not necessarily in\n",
+ " degrees.\n",
+ " :type coordinates_list: list\n",
+ " \n",
+ " :param coordinates_format: Which format the list of coordinates is provided in\n",
+ " will determine whether or not convert_to_degrees is run.\n",
+ " :type coordinates_format: string\n",
+ " \n",
+ " :param write: Flag to save results to a .csv file.\n",
+ " :type write: boolean\n",
+ " \"\"\"\n",
+ " \n",
+ " results = []\n",
+ " \n",
+ " # Iterate through our list of coordinate tuples\n",
+ " for target in coordinates_list:\n",
+ " query_results = \"None\"\n",
+ " print(\"...checking {0}...\".format(target))\n",
+ " \n",
+ " # Convert each pair of coordinates to degrees if necessary\n",
+ " if coordinates_format.lower() == \"deg\":\n",
+ " in_degrees = target\n",
+ " else:\n",
+ " in_degrees = convert_to_degrees(target[0], target[1])\n",
+ " \n",
+ " # Submit an initial count query\n",
+ " count = filtered_position_query(in_degrees)\n",
+ " \n",
+ " # If the count is within a valid range, submit the full query\n",
+ " if count > 0 and count < 50000:\n",
+ " query_results = filtered_position_query(in_degrees, count=False)\n",
+ " nearby_targets = analyze_query_results(in_degrees, query_results)\n",
+ " \n",
+ " # Generate a filename and write CSV table if 'write' enabled\n",
+ " if write:\n",
+ " filename = \"results_{0}_{1}.csv\".format(target[0], target[1])\n",
+ " filename = write_to_csv_file(query_results, filename)\n",
+ " \n",
+ " # Skip if too many results are found\n",
+ " elif count > 50000:\n",
+ " print(\"More than 50,000 results found! Please narrow your query.\")\n",
+ " \n",
+ " # Skip if no results are found\n",
+ " elif count == 0:\n",
+ " print(\"No nearby observations found for {0}\".format(target))\n",
+ " \n",
+ " results.append(query_results)\n",
+ " \n",
+ " # Return the list of all query result dictionaries\n",
+ " return results"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "OUR_COORDINATES_LIST = [('04h16m09.370s', '-24d04m20.50s'),\n",
+ " ('05h42m15s', '+48d22m43s'),\n",
+ " ('19h45m01.190s', '-14d45m15.79s'),\n",
+ " ('20h20m20.20s', '+20d20m20.20s')\n",
+ " ]\n",
+ "OUR_QUERY_RESULTS = check_multiple_targets(OUR_COORDINATES_LIST, \"icrs\", write=False) # Change the 'write' flag to save .csv files"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
Getting coordinates from stationary target names
\n",
+ "
Using the API also gives us access to the name-resolver service, which allows us to input a list of target names instead of coordinates. The returned coordinates are already in degrees, so we can also skip the convert_to_degrees module.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def resolve_target_names(target_list):\n",
+ " \"\"\"\n",
+ " Look up a list of target names using the MAST lookup service and return\n",
+ " a list of coordinates.\n",
+ " \n",
+ " :param target_list: The list of targets to be resolved.\n",
+ " :type target_list: list\n",
+ " \"\"\"\n",
+ " \n",
+ " # Set up an empty list for coordinate results and iterate through the list of target names\n",
+ " coordinates_list = []\n",
+ " for target_name in target_list:\n",
+ " \n",
+ " # Make a resolver request with the current target name\n",
+ " resolverRequest = {'service':'Mast.Name.Lookup',\n",
+ " 'params':{'input':target_name,\n",
+ " 'format':'json'}\n",
+ " }\n",
+ " headers, resolvedObjectString = mastQuery(resolverRequest)\n",
+ " resolvedObject = json.loads(resolvedObjectString)\n",
+ " \n",
+ " # If the target name was not found, we will run into IndexErrors when we try to set these \n",
+ " # variables\n",
+ " try:\n",
+ " target_ra = resolvedObject['resolvedCoordinate'][0]['ra']\n",
+ " target_dec = resolvedObject['resolvedCoordinate'][0]['decl']\n",
+ " canonical_name = resolvedObject['resolvedCoordinate'][0]['canonicalName']\n",
+ " except IndexError:\n",
+ " print(\"{0} not found\".format(target_name))\n",
+ " continue\n",
+ " \n",
+ " # Add the coordinates as a tuple to the list\n",
+ " target_coords = (target_ra, target_dec)\n",
+ " coordinates_list.append(target_coords)\n",
+ " print(\"Found {0} at {1}\".format(canonical_name, target_coords))\n",
+ " \n",
+ " # Return the list of coordinate tuples\n",
+ " return coordinates_list"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "OUR_TARGETS_LIST = ['M92',\n",
+ " '30 Doradus',\n",
+ " 'Dumbbell Nebula',\n",
+ " 'Fake Test Galaxy'\n",
+ " ]\n",
+ "NEW_COORDINATES_LIST = resolve_target_names(OUR_TARGETS_LIST)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
With our list of names resolved we can send our new list of coordinates to our previous query module."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "NEW_QUERY_RESULTS = check_multiple_targets(NEW_COORDINATES_LIST, \"deg\", write=False) # Change the 'write' flag to save .csv files"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
Searching for moving targets (keyword searches on target_name)
\n",
+ "
Moving targets are a bit trickier since we do not have a single set of coordinates to search against. Instead, we'll look for matching entries in the target_name field and we'll use the freeText paramter with wildcards to find any close matches. Again, we'll begin with an initial \"COUNT_BIG(*)\" query to get a results count first.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def filtered_keyword_query(keyword, count=True):\n",
+ " \"\"\"\n",
+ " Construct a filtered mashup request to send to the mastQuery function. This\n",
+ " will search for a given keyword within the 'target_name' field and return\n",
+ " any results including wildcards.\n",
+ " \n",
+ " :param keyword: The keyword to search for within 'target_name'.\n",
+ " :type keyword: string\n",
+ " \n",
+ " :param count: Flag to submit a full query or a count query. Sends count\n",
+ " by default.\n",
+ " :type count: boolean\n",
+ " \"\"\"\n",
+ " \n",
+ " # Use count flag to determine the columns queried\n",
+ " if count:\n",
+ " columns = \"COUNT_BIG(*)\" # \"COUNT_BIG(*)\" will only get a count of the results\n",
+ " else:\n",
+ " columns = \"*\"\n",
+ " \n",
+ " # Construct the mashup request\n",
+ " service = \"Mast.Caom.Filtered\"\n",
+ " filters = [{\"paramName\":\"calib_level\", \"values\":[\"-1\"]},\n",
+ " {\"paramName\":\"obs_collection\", \"values\":[\"JWST\"]},\n",
+ " {\"paramName\":\"target_name\", \"values\":[],\"freeText\":\"%\"+keyword+\"%\"}\n",
+ " ]\n",
+ " mashupRequest = {\"service\":service,\n",
+ " \"format\":\"json\",\n",
+ " \"params\":{\"columns\":columns,\n",
+ " \"filters\":filters\n",
+ " }\n",
+ " }\n",
+ "\n",
+ " # Submit the MAST query\n",
+ " headers,outString = mastQuery(mashupRequest)\n",
+ " queryResults = json.loads(outString)\n",
+ " \n",
+ " # Return the results, depending on the type of query submitted\n",
+ " if count:\n",
+ " data = queryResults['data']\n",
+ " count = data[0]['Column1']\n",
+ " return count\n",
+ " else:\n",
+ " return queryResults"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "JUPITER_COUNT = filtered_keyword_query(\"Jupiter\")\n",
+ "print(JUPITER_COUNT)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
We find 126 target names containing \"Jupiter\", so we'll go ahead and submit the full query.
With query results in hand we'll build a module to analyze our moving target output. We'll want lists of the target names we matched with and which programs these are found in.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def keyword_matches(queryResults):\n",
+ " \"\"\"\n",
+ " Create a list of the unique target names that were returned by the filtered_keyword_query.\n",
+ " Also generate a link to each nearby program found.\n",
+ " \n",
+ " :param queryResults: Full results from a MAST query.\n",
+ " :type queryResults: dictionary\n",
+ " \"\"\"\n",
+ " \n",
+ " # Set up initial variables\n",
+ " data = queryResults['data']\n",
+ " targets = []\n",
+ " programs = []\n",
+ " \n",
+ " # For each query result, pull out the target_name and proposal_id values\n",
+ " for obs in data:\n",
+ " current_target = obs['target_name']\n",
+ " current_program = obs['proposal_id']\n",
+ " targets.append(current_target)\n",
+ " programs.append(current_program)\n",
+ " \n",
+ " # Get the unique sets of target_name and proposal_id values\n",
+ " unique_targets = sorted(list(set(targets)))\n",
+ " unique_programs = sorted(list(set(programs)))\n",
+ " \n",
+ " # Output our results\n",
+ " print(\"Matched the keyword to: {0}\".format(unique_targets))\n",
+ " for p in unique_programs:\n",
+ " address = \"https://jwst.stsci.edu/observing-programs/program-information?id={0}\".format(p)\n",
+ " print(\"Found planned observations in {0}: {1}\".format(p, address))\n",
+ " \n",
+ " # Return the list of unique target names\n",
+ " return unique_targets"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "JUPITER_MATCHES = keyword_matches(JUPITER_QUERY)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
Copying our multiple targets module from above, with a few tweaks we can now run back-to-back queries on a list of moving target names. We can also incorporate the write_to_csv_file module to automatically save all our results for further inspection.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def check_multiple_keywords(keywords_list, write):\n",
+ " \"\"\"\n",
+ " Perform multiple filtered MAST queries based on a list of keywords to look\n",
+ " for within the 'target_name' field. Analyze the results and write them to\n",
+ " a .csv file if requested.\n",
+ " \n",
+ " :param keywords_list: List of target keywords to search for in the 'target_name'\n",
+ " field.\n",
+ " :type keywords_list: list\n",
+ " \n",
+ " :param write: Flag whether or not the results are save in a .csv file.\n",
+ " :type write: boolean\n",
+ " \"\"\"\n",
+ " \n",
+ " results = []\n",
+ " \n",
+ " # Iterate through our list of target_name keywords\n",
+ " for target in keywords_list:\n",
+ " query_results = \"None\"\n",
+ " print(\"...checking {0}...\".format(target))\n",
+ " \n",
+ " # Submit an initial count query\n",
+ " count = filtered_keyword_query(target)\n",
+ " \n",
+ " # If the count is within a valid range, submit the full query\n",
+ " if count > 0 and count < 50000:\n",
+ " query_results = filtered_keyword_query(target, count=False)\n",
+ " targ_names = keyword_matches(query_results)\n",
+ " \n",
+ " # Generate a filename and write CSV table if 'write' enabled\n",
+ " if write:\n",
+ " filename = \"results_{0}.csv\".format(target)\n",
+ " filename = write_to_csv_file(query_results, filename)\n",
+ " \n",
+ " # Skip if too many results are found\n",
+ " elif count > 50000:\n",
+ " print(\"More than 50,000 results found! Please narrow your query.\")\n",
+ " \n",
+ " # Skip if no results are found\n",
+ " elif count == 0:\n",
+ " print(\"No target_name matches found for {0}\".format(target))\n",
+ " \n",
+ " results.append(query_results)\n",
+ " \n",
+ " # Return the list of query result dictionaries\n",
+ " return results"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "OUR_MOVING_TARGETS = [\"Jupiter\",\n",
+ " \"Titan\",\n",
+ " \"Ceres\",\n",
+ " \"Sun\"\n",
+ " ]\n",
+ "MOVING_MATCHES = check_multiple_keywords(OUR_MOVING_TARGETS, write=False) # Change the 'write' flag to save .csv files"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
Visualizing results with Aladin
\n",
+ "
Now that we have some idea which of our targets may have planned JWST observations nearby, it may help to plot the existing planned observations in the area. The MAST Portal handles this automatically, but we can also make a simple display with footprints here using Aladin Lite. HOWEVER, these footprints do not currently take into account some factors such as telescope rotation/orientation, or offsets from the main pointing. These footprints only represent an approximation of the instrument FOV's.
\n",
+ "
First, we'll need a function to extract the provided region information from our previous query results.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def get_polygons(queryResults):\n",
+ " \"\"\"\n",
+ " Extract 's_region' information from a set of MAST query results. Only\n",
+ " returns the unique set of footprints found.\n",
+ " \n",
+ " :param queryResults: Full results from a MAST query.\n",
+ " :type queryResults: dictionary\n",
+ " \"\"\"\n",
+ " \n",
+ " # We're only interested in the 'data' dictionary from the query results here\n",
+ " data = queryResults['data']\n",
+ " polygons = []\n",
+ " \n",
+ " # Iterate through each observation\n",
+ " for obs in data:\n",
+ " region = obs['s_region']\n",
+ " individual = region[7:].split(\" \") # Cut off prepending 'POLYGON'\n",
+ " individual = list(filter(None, individual)) # Eliminate empty entries from extra spaces\n",
+ " shape = []\n",
+ " \n",
+ " # Pair off the entries into RA, Dec coordinates\n",
+ " while len(individual) > 1:\n",
+ " coordinate = [individual[0], individual[1]]\n",
+ " shape.append(coordinate)\n",
+ " individual = individual[2:]\n",
+ " \n",
+ " # We can ignore repeated footprints\n",
+ " if shape in polygons:\n",
+ " continue\n",
+ " else:\n",
+ " print(shape)\n",
+ " polygons.append(shape)\n",
+ " \n",
+ " # Return the list of coordinate lists\n",
+ " return polygons"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "RESULTS_TO_PLOT = OUR_QUERY_RESULTS[2]\n",
+ "TEST_REGIONS = get_polygons(RESULTS_TO_PLOT)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
With our coordinates and footprint information, we'll construct a string to embed an Aladin Lite script into HTML.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def make_aladin_html(coordinates, footprints=None):\n",
+ " \"\"\"\n",
+ " Construct an HTML-formatted string to embed an Aladin Lite viewer script,\n",
+ " which will center on a set of given coordinates and draw any provided\n",
+ " footprints in overlays.\n",
+ " \n",
+ " :param coordinates: A single set of coordinates in degrees.\n",
+ " :type coordinates: tuple\n",
+ " \n",
+ " :param footprints: A list of any polygon coordinates found in the 's_region'\n",
+ " field from MAST query results.\n",
+ " :type footprints: list\n",
+ " \"\"\"\n",
+ " \n",
+ " # Format an RA/Dec string for Aladin\n",
+ " ra = str(coordinates[0])\n",
+ " dec = str(coordinates[1])\n",
+ " coords = ra + \", \" + dec\n",
+ " \n",
+ " # Create a unique div ID, allowing for multiple windows\n",
+ " div = \"aladin-lite-\" + ra.replace(\".\", \"\") + dec.replace(\".\", \"\")\n",
+ " \n",
+ " # Begin constructing the HTML string with Aladin embedded\n",
+ " aladin_string = \"\"\"\n",
+ " \n",
+ "\n",
+ " \n",
+ " \n",
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ " \"\n",
+ " \n",
+ " # Return the full constructed string\n",
+ " return aladin_string"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "COORDINATES_TO_PLOT = OUR_COORDINATES_LIST[2]\n",
+ "PLOT_IN_DEGREES = convert_to_degrees(COORDINATES_TO_PLOT[0], COORDINATES_TO_PLOT[1])\n",
+ "ALADIN_HTML = make_aladin_html(PLOT_IN_DEGREES, TEST_REGIONS)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
Next, we'll need to access IPython's core library for running HTML code inside a notebook. Then we can use that to execute the HTML code we constructed and view the resulting plot, which retains Aladin Lite's pan and zoom capabilities as well as the 'Export view as PNG' feature.
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from IPython.core.display import HTML\n",
+ "HTML(ALADIN_HTML)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.7"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/MAST/TESS/beginner_ticid_query/ticid_query.ipynb b/notebooks/MAST/TESS/beginner_ticid_query/ticid_query.ipynb
new file mode 100644
index 00000000..f98acfb8
--- /dev/null
+++ b/notebooks/MAST/TESS/beginner_ticid_query/ticid_query.ipynb
@@ -0,0 +1,192 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "# Beginner: Search the TESS Input Catalog for a Specific TIC ID\n",
+ "\n",
+ "This notebook tutorial demonstrates how to retrieve the row of information for a single TIC Identification number. This tutorial uses Python and the astroquery module. Information on how to install astroquery for MAST can be found on the [Astroquery site](http://astroquery.readthedocs.io/).\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from astroquery.mast import Catalogs"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Find the row in the TIC where the value of \"ID\" matches the integer held in ticid"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "ticid = 12345678\n",
+ "data = Catalogs.query_criteria(catalog=\"Tic\", ID=ticid)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The value is held in an astropy table.\n",
+ "Use `show_in_notebook` to display it in a user-friendly way."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "Table masked=True length=1\n",
+ "
![\"STScI](\"https://raw.githubusercontent.com/spacetelescope/notebooks/master/assets/stsci_pri_combo_mark_horizonal_white_bkgd.png\")
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.3"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}