__nbid__ = '0038'
__author__ = 'Brian Merino <brian.merino@noirlab.edu>, Vinicius Placco <vinicius.placco@noirlab.edu>'
-__version__ = '20241209' # yyyymmdd; version datestamp of this notebook
+__version__ = '20250912' # yyyymmdd; version datestamp of this notebook
__keywords__ = ['flamingos-2','gemini','browndwarf','dwarf','dragons']
Showcase how to reduce Flamingos-2 imaging data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS (Py3.7)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.
Showcase how to reduce Flamingos-2 imaging data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS-4.0.0 (DL,Py3.12)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
+DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines.
In this notebook, we present an example of a DRAGONS Jupyter notebook that works in the Data Lab environment to reduce example Gemini South Flamingos-2 Y-band imaging data fully. This notebook will not present all of the details of the many options available to adjust or optimize the DRAGONS Flamingos-2 data reduction process; rather, it will just show one example of a standard reduction of a Flamingos-2 imaging dataset.
The data used in this notebook example is Flamingos-2 Y band imaging from the Gemini archive of the brown dwarf W0413-4750 from the Gemini South program "A Study of the 450K Transition from T to Y Dwarf, and of the 350K Y Dwarfs", PI: S. Leggett, program ID GS-2013B-Q-15.
@@ -7623,23 +7625,23 @@import os
-import warnings
-import glob
-import shutil
+import os
+import warnings
+import glob
+import shutil
-from gempy.adlibrary import dataselect
-from gempy.utils import logutils
+from gempy.adlibrary import dataselect
+from gempy.utils import logutils
-from recipe_system import cal_service
-from recipe_system.reduction.coreReduce import Reduce
+from recipe_system import cal_service
+from recipe_system.reduction.coreReduce import Reduce
-from astropy.io import fits
-from astropy.wcs import WCS
-from astropy.utils.exceptions import AstropyWarning
+from astropy.io import fits
+from astropy.wcs import WCS
+from astropy.utils.exceptions import AstropyWarning
-import matplotlib.pyplot as plt
-from matplotlib.colors import Normalize
+import matplotlib.pyplot as plt
+from matplotlib.colors import Normalize
warnings.simplefilter('ignore', category=AstropyWarning)
@@ -7667,7 +7669,7 @@ Prepare the working directoryIn [ ]:
def clean_up(save_reduced=0):
+def clean_up(save_reduced=0):
#Does the calibrations directory already exist?
caldb_Exist = os.path.exists('./calibrations')
@@ -8204,6 +8206,66 @@ Display the stacked image
+
+Display the final product¶
+
+
+final_image = "S20131121S0075_image.fits"
+hdu_final = fits.open(final_image)
+wcs_final = WCS(hdu_final[1].header)
+hdu_final.info()
+data_final = hdu_final[1].data
+The output stack units are in electrons (header keyword BUNIT=electrons). The output stack is stored in a multi-extension FITS (MEF) file. The science signal is in the “SCI” extension, the variance is in the “VAR” extension, and the data quality plane (mask) is in the “DQ” extension.
+Note: The upper-left quadrant of this science sequence is rather messy. This is caused by the PWFS2 guide probe (see Emission from PWFS2 guide probe). Photometry in this portion of the image is likely to be seriously compromised.
+plt.imshow(data_final,origin='lower',vmin=0.01,vmax=160)
+
+plt.scatter(1150,1126,marker='o',facecolors='none',s=20,edgecolors='red')
+plt.text(1095,1000,'W0413-4750',c='red')
+plt.show()
+__nbid__ = '0071'
__author__ = 'Brian Merino <brian.merino@noirlab.edu>, Vinicius Placco <vinicius.placco@noirlab.edu>'
-__version__ = '20250515' # yyyymmdd; version datestamp of this notebook
+__version__ = '20250912' # yyyymmdd; version datestamp of this notebook
__keywords__ = ['GHOST','Gemini','stars','DRAGONS']
Showcase how to reduce GHOST spectroscopy data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS-3.2.2 (DL,Py3.10.14)". The steps include downloading data from the Gemini archive, setting up the DRAGONS calibration service, processing biases, flats, and arcs, creating master flats and slit-flats, reducing the standards and science data, and finally creating the final reduced spectra for GHOST's red and blue arms.
Showcase how to reduce GHOST spectroscopy data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS-4.0.0 (DL,Py3.12)". The steps include downloading data from the Gemini archive, setting up the DRAGONS calibration service, processing biases, flats, and arcs, creating master flats and slit-flats, reducing the standards and science data, and finally creating the final reduced spectra for GHOST's red and blue arms.
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
+DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines. It is important to note that when a DRAGONS command is executed, the output will be displayed inside the cell. Make sure to scroll through the output to ensure no errors are missed.
We present an example of a DRAGONS Jupyter notebook that works in the Data Lab environment to reduce Gemini South GHOST blue:2x2 and red:2x2 spectroscopy data fully. This notebook will not present all of the details of the many options available to adjust or optimize the DRAGONS GHOST data reduction process; rather, it will just show one example of a standard reduction of a GHOST spectroscopic dataset.
The data used in this notebook example is GHOST blue:2x2 and red:2x2 spectroscopy data from the Gemini archive of the star XX Oph from the GHOST commissioning run. Because the data used is from a commissioning run, there is no program information available, but you can find more information about GHOST's red and blue IFUs on the GHOST instrument page.
@@ -7964,32 +7964,6 @@caldb.add_cal(glob.glob('raw/bpm*.fits')[0])
-caldb.add_cal(glob.glob('raw/bpm*.fits')[1])
-Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as calibrations. They are downloadable from the archive instead of being packaged with the software. They are automatically associated like any other calibrations. This means that the user now must download the BPMs along with the other calibrations and add the BPMs to the local calibration manager.
+See Getting Bad Pixel Masks from the archive in Tips and Tricks to learn about the various ways to get the BPMs from the archive.
+To add the BPM included in the data package to the local calibration database:
+caldb.add_cal(glob.glob('raw/bpm*.fits')[0])
+caldb.add_cal(glob.glob('raw/bpm*.fits')[1])
+# clean_up(save_reduced=0)
+#clean_up(save_reduced=1)
+
+
__nbid__ = '0039'
__author__ = 'Brian Merino <brian.merino@noirlab.edu>, Susan Ridgway <susan.ridgway@noirlab.edu>, Vinicius Placco <vinicius.placco@noirlab.edu>'
-__version__ = '20241209' # yyyymmdd; version datestamp of this notebook
+__version__ = '20250912' # yyyymmdd; version datestamp of this notebook
__keywords__ = ['gmos','gemini','galaxy','dragons']
Showcase how to reduce GMOS imaging data using the Gemini DRAGONS package on the Data Lab science platform, using a custom DRAGONS kernel "DRAGONS (Py3.7)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, bias, and science frames, and creating a single combined stacked image.
Showcase how to reduce GMOS imaging data using the Gemini DRAGONS package on the Data Lab science platform, using a custom DRAGONS kernel "DRAGONS-4.0.0 (DL,Py3.12)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, bias, and science frames, and creating a single combined stacked image.
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
+DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines.
In this notebook, we present an example of a DRAGONS Jupyter notebook that works in the Data Lab environment to reduce example Gemini South GMOS G-band imaging data fully. This notebook will not present all the details of the many options available to adjust or optimize the DRAGONS GMOS data reduction process; instead, it will just show one example of a standard reduction of a GMOS imaging dataset.
The data used in this notebook example is GMOS G band imaging from the Gemini archive of the galaxy NGC 5018 from the Gemini South program "The Evolutionary History of NGC 5018", PI: L. Sesto, program ID GS-2018A-Q-207. The final reduced science image combines five science frames of 460 seconds each that were dithered between each exposure.
@@ -7622,23 +7624,23 @@import warnings
-import glob
-import os
-import shutil
+import warnings
+import glob
+import os
+import shutil
-from gempy.adlibrary import dataselect
-from gempy.utils import logutils
+from gempy.adlibrary import dataselect
+from gempy.utils import logutils
-from recipe_system import cal_service
-from recipe_system.reduction.coreReduce import Reduce
+from recipe_system import cal_service
+from recipe_system.reduction.coreReduce import Reduce
-from astropy.io import fits
-from astropy.wcs import WCS
-from astropy.utils.exceptions import AstropyWarning
+from astropy.io import fits
+from astropy.wcs import WCS
+from astropy.utils.exceptions import AstropyWarning
-import matplotlib.pyplot as plt
-from matplotlib.colors import LogNorm
+import matplotlib.pyplot as plt
+from matplotlib.colors import LogNorm
warnings.simplefilter('ignore', category=AstropyWarning)
@@ -7666,7 +7668,7 @@ Prepare the working directoryIn [ ]:
def clean_up(save_reduced=0):
+def clean_up(save_reduced=0):
#Does the calibrations directory already exist?
caldb_Exist = os.path.exists('./calibrations')
@@ -7746,6 +7748,11 @@ About the datasetS20180423S0050-054
0 s, r-band
+
+BPM
+bpm_20220303_gmos-n_Ham_22_full_12amp.fits
+
@@ -7790,7 +7797,8 @@ Downloading the datahttp://archive.gemini.edu/file/S20180423S0051.fits
http://archive.gemini.edu/file/S20180423S0052.fits
http://archive.gemini.edu/file/S20180423S0053.fits
-http://archive.gemini.edu/file/S20180423S0054.fits\
+http://archive.gemini.edu/file/S20180423S0054.fits
+https://archive.gemini.edu/file/bpm_20220303_gmos-n_Ham_22_full_12amp.fits\
" > gmos_im.list
Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as calibrations. They are downloadable from the archive instead of being packaged with the software. They are automatically associated like any other calibrations. This means that the user now must download the BPMs along with the other calibrations and add the BPMs to the local calibration manager.
+See Getting Bad Pixel Masks from the archive in Tips and Tricks to learn about the various ways to get the BPMs from the archive.
+To add the BPM included in the data package to the local calibration database:
+for bpm in dataselect.select_data(all_files, ['BPM']):
+ caldb.add_cal(bpm)
+We start the data reduction by creating a master bias for the science data. It can be created and added to the calibration database using the commands below. When the cell is done running, the master bias file will be saved with the name of the first bias and the suffix _bias.fits.
plt.figure(figsize = (15,15))
+plt.figure(figsize = (10,8))
plt.subplot(projection=wcs)
plt.imshow(image_data,cmap='gray',norm=LogNorm(vmin=3000, vmax=60000),origin='lower')
plt.xlabel('Right Ascension [hh:mm:ss]',fontsize=14,fontweight='bold')
diff --git a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_Imaging_Galaxy/GMOS_Imaging_Galaxy.ipynb b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_Imaging_Galaxy/GMOS_Imaging_Galaxy.ipynb
index 60495801e..3610e7bed 100644
--- a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_Imaging_Galaxy/GMOS_Imaging_Galaxy.ipynb
+++ b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_Imaging_Galaxy/GMOS_Imaging_Galaxy.ipynb
@@ -8,7 +8,7 @@
"source": [
"__nbid__ = '0039'\n",
"__author__ = 'Brian Merino , Susan Ridgway , Vinicius Placco '\n",
- "__version__ = '20241209' # yyyymmdd; version datestamp of this notebook\n",
+ "__version__ = '20250912' # yyyymmdd; version datestamp of this notebook\n",
"__keywords__ = ['gmos','gemini','galaxy','dragons']"
]
},
@@ -36,6 +36,7 @@
"* [Downloading data for reduction](#Downloading_Data)\n",
"* [Set up the DRAGONS logger](#DRAGONS_logger)\n",
"* [Create File Lists](#File_Lists)\n",
+ "* [Bad Pixel Mask](#BPM)\n",
"* [Create Master Bias](#Master_Bias)\n",
"* [Create Master Flat Field](#Master_Flat)\n",
"* [Create Master Fringe Frame](#Master_Fringe)\n",
@@ -45,11 +46,11 @@
"\n",
"\n",
"# Goals\n",
- "Showcase how to reduce GMOS imaging data using the Gemini DRAGONS package on the Data Lab science platform, using a custom DRAGONS kernel `\"DRAGONS (Py3.7)\"`. The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, bias, and science frames, and creating a single combined stacked image.\n",
+ "Showcase how to reduce GMOS imaging data using the Gemini DRAGONS package on the Data Lab science platform, using a custom DRAGONS kernel `\"DRAGONS-4.0.0 (DL,Py3.12)\"`. The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, bias, and science frames, and creating a single combined stacked image.\n",
"\n",
"\n",
"# Summary\n",
- "DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.\n",
+ "DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.\n",
"\n",
"The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines. \n",
"\n",
@@ -206,7 +207,8 @@
"| :--- | :--- | :---: |\n",
"| Science | S20180419S0098-102 | 460 s, g-band |\n",
"| Twilight Flats | S20180419S0207-211 | g-band |\n",
- "| Bias | S20180423S0050-054 | 0 s, r-band |"
+ "| Bias | S20180423S0050-054 | 0 s, r-band |\n",
+ "| BPM | bpm_20220303_gmos-n_Ham_22_full_12amp.fits | |"
]
},
{
@@ -245,7 +247,8 @@
"http://archive.gemini.edu/file/S20180423S0051.fits\n",
"http://archive.gemini.edu/file/S20180423S0052.fits\n",
"http://archive.gemini.edu/file/S20180423S0053.fits\n",
- "http://archive.gemini.edu/file/S20180423S0054.fits\\\n",
+ "http://archive.gemini.edu/file/S20180423S0054.fits\n",
+ "https://archive.gemini.edu/file/bpm_20220303_gmos-n_Ham_22_full_12amp.fits\\\n",
"\" > gmos_im.list"
]
},
@@ -323,6 +326,30 @@
" dataselect.expr_parser('(observation_class==\"science\" and filter_name==\"g\")'))"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## **Bad pixel mask**\n",
+ "\n",
+ "Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as calibrations. They are downloadable from the archive instead of being packaged with the software. They are automatically associated like any other calibrations. This means that the user now must download the BPMs along with the other calibrations and add the BPMs to the local calibration manager.\n",
+ "\n",
+ "See [Getting Bad Pixel Masks from the archive](https://dragons.readthedocs.io/projects/gnirsimg-drtutorial/en/stable/05_tips_and_tricks.html#getbpm) in Tips and Tricks to learn about the various ways to get the BPMs from the archive.\n",
+ "\n",
+ "To add the BPM included in the data package to the local calibration database:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "for bpm in dataselect.select_data(all_files, ['BPM']):\n",
+ " caldb.add_cal(bpm)"
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
@@ -455,7 +482,7 @@
"metadata": {},
"outputs": [],
"source": [
- "plt.figure(figsize = (15,15))\n",
+ "plt.figure(figsize = (10,8))\n",
"plt.subplot(projection=wcs)\n",
"plt.imshow(image_data,cmap='gray',norm=LogNorm(vmin=3000, vmax=60000),origin='lower')\n",
"plt.xlabel('Right Ascension [hh:mm:ss]',fontsize=14,fontweight='bold')\n",
@@ -485,9 +512,9 @@
],
"metadata": {
"kernelspec": {
- "display_name": "DRAGONS-3.2.2 (DL,Py3.10.14)",
+ "display_name": "DRAGONS-4.0.0 (DL,Py3.12)",
"language": "python",
- "name": "dragons-3.2.2"
+ "name": "dragons-4.0.0"
},
"language_info": {
"codemirror_mode": {
@@ -499,7 +526,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.10.14"
+ "version": "3.12.10"
}
},
"nbformat": 4,
diff --git a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_Imaging_StarryField/GMOS_Imaging_StarryField.html b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_Imaging_StarryField/GMOS_Imaging_StarryField.html
index 38c142bf9..a969aff9a 100644
--- a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_Imaging_StarryField/GMOS_Imaging_StarryField.html
+++ b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_Imaging_StarryField/GMOS_Imaging_StarryField.html
@@ -7312,6 +7312,7 @@
processEnvironments: true
},
displayAlign: 'center',
+ messageStyle: 'none',
CommonHTML: {
linebreaks: {
automatic: true
@@ -7519,7 +7520,7 @@
__nbid__ = '0040'
__author__ = 'Brian Merino <brian.merino@noirlab.edu>, Vinicius Placco <vinicius.placco@noirlab.edu>'
-__version__ = '20241209' # yyyymmdd; version datestamp of this notebook
+__version__ = '20250912' # yyyymmdd; version datestamp of this notebook
__keywords__ = ['gmos','gemini','stars','dragons']
@@ -7533,7 +7534,7 @@
-Gemini GMOS starry field photometry reduction using DRAGONS Python API¶
adapted from https://dragons.readthedocs.io/projects/gmosimg-drtutorial/en/v3.2.1/ex1_gmosim_starfield_api.html¶
+Gemini GMOS starry field photometry reduction using DRAGONS Python API¶
adapted from https://dragons.readthedocs.io/projects/gmosimg-drtutorial/en/stable/ex1_gmosim_starfield_api.html¶
Showcase how to reduce GMOS imaging data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS (Py3.7)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing bias, flats, fringe, and science frames, and creating a single combined stacked image.
Showcase how to reduce GMOS imaging data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS-4.0.0 (DL,Py3.12)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing bias, flats, fringe, and science frames, and creating a single combined stacked image.
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
+DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines.
In this notebook, we present an example of a DRAGONS Jupyter notebook that works in the Data Lab environment to reduce example Gemini North GMOS I-band imaging data fully. This notebook will not present all of the details of the many options available to adjust or optimize the DRAGONS GMOS data reduction process; rather, it will just show one example of a standard reduction of a GMOS imaging dataset.
The data used in this notebook example is GMOS I band imaging from the Gemini archive of a starry field from the Gemini North Hamamatsu CCD commissioning (Program: GN-2017A-SV-151).
@@ -7621,23 +7623,23 @@import warnings
-import glob
-import os
-import shutil
+import warnings
+import glob
+import os
+import shutil
-from gempy.adlibrary import dataselect
-from gempy.utils import logutils
+from gempy.adlibrary import dataselect
+from gempy.utils import logutils
-from recipe_system import cal_service
-from recipe_system.reduction.coreReduce import Reduce
+from recipe_system import cal_service
+from recipe_system.reduction.coreReduce import Reduce
-from astropy.io import fits
-from astropy.wcs import WCS
-from astropy.utils.exceptions import AstropyWarning
+from astropy.io import fits
+from astropy.wcs import WCS
+from astropy.utils.exceptions import AstropyWarning
-import matplotlib.pyplot as plt
-from matplotlib.colors import LogNorm
+import matplotlib.pyplot as plt
+from matplotlib.colors import LogNorm
warnings.simplefilter('ignore', category=AstropyWarning)
@@ -7665,7 +7667,7 @@ Prepare the working directoryIn [ ]:
def clean_up(save_reduced=0):
+def clean_up(save_reduced=0):
#Does the calibrations directory already exist?
caldb_Exist = os.path.exists('./calibrations')
@@ -7750,6 +7752,11 @@ About the datasetN20170702S0178-182
40 to 16 s, i-band
+
+BPM
+bpm_20170306_gmos-n_Ham_22_full_12amp.fits
+
@@ -7799,7 +7806,8 @@ Downloading the datahttp://archive.gemini.edu/file/N20170702S0179.fits
http://archive.gemini.edu/file/N20170702S0180.fits
http://archive.gemini.edu/file/N20170702S0181.fits
-http://archive.gemini.edu/file/N20170702S0182.fits\
+http://archive.gemini.edu/file/N20170702S0182.fits
+http://archive.gemini.edu/file/bpm_20170306_gmos-n_Ham_22_full_12amp.fits\
" > gmos_im_star.list
Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as calibrations. They are downloadable from the archive instead of being packaged with the software. They are automatically associated like any other calibrations. This means that the user now must download the BPMs along with the other calibrations and add the BPMs to the local calibration manager.
+for bpm in dataselect.select_data(all_files, ['BPM']):
+ caldb.add_cal(bpm)
+We start the data reduction by creating a master bias for the science data. It can be created and added to the calibration database using the commands below. The master bias will have the name of the first bias with the suffix _bias.fits
__nbid__ = '0041'
__author__ = 'Brian Merino <brian.merino@noirlab.edu>, Vinicius Placco <vinicius.placco@noirlab.edu>'
-__version__ = '20241209' # yyyymmdd; version datestamp of this notebook
+__version__ = '20250912' # yyyymmdd; version datestamp of this notebook
__keywords__ = ['gmos','gemini','longslit','whitedwarf','dragons']
Showcase how to reduce GMOS longslit data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS (Py3.7)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.
Showcase how to reduce GMOS longslit data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS-4.0.0 (DL,Py3.12)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
+DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines.
In this notebook, we present an example of a DRAGONS Jupyter notebook that works in the Data Lab environment to reduce example Gemini South GMOS longslit data fully. This notebook will not present all of the details of the many options available to adjust or optimize the DRAGONS GMOS longslit data reduction process; rather, it will just show one example of a standard reduction of a GMOS longslit dataset.
The data used in this notebook example is GMOS longslit data from the Gemini archive of the candiate DB white dwarf J2145+0031 from the Gemini South program "Spectroscopy of massive DB white dwarf candidate stars", PI: Alejandra Romero, program ID GS-2017B-Q-72.
@@ -7626,27 +7627,27 @@import warnings
-import glob
-import os
-import shutil
+import warnings
+import glob
+import os
+import shutil
-import astrodata
+import astrodata
-from astropy.io import fits
-from astropy.utils.exceptions import AstropyWarning
+from astropy.io import fits
+from astropy.utils.exceptions import AstropyWarning
-import numpy as np
-import matplotlib.pyplot as plt
-from matplotlib.colors import LogNorm
+import numpy as np
+import matplotlib.pyplot as plt
+from matplotlib.colors import LogNorm
-import gemini_instruments
-from gempy.utils import logutils
-from gempy.adlibrary import dataselect
-from gempy.adlibrary import plotting
+import gemini_instruments
+from gempy.utils import logutils
+from gempy.adlibrary import dataselect
+from gempy.adlibrary import plotting
-from recipe_system import cal_service
-from recipe_system.reduction.coreReduce import Reduce
+from recipe_system import cal_service
+from recipe_system.reduction.coreReduce import Reduce
warnings.simplefilter('ignore', category=AstropyWarning)
warnings.filterwarnings("ignore")
@@ -7675,7 +7676,7 @@ Prepare the working directoryIn [ ]:
-def clean_up(save_reduced=0):
+def clean_up(save_reduced=0):
#Does the calibrations directory already exist?
caldb_Exist = os.path.exists('./calibrations')
diff --git a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_longslit_WhiteDwarf/GMOS_longslit_WhiteDwarf.ipynb b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_longslit_WhiteDwarf/GMOS_longslit_WhiteDwarf.ipynb
index 4e6bae196..4db2b225d 100644
--- a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_longslit_WhiteDwarf/GMOS_longslit_WhiteDwarf.ipynb
+++ b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GMOS_longslit_WhiteDwarf/GMOS_longslit_WhiteDwarf.ipynb
@@ -9,7 +9,7 @@
"source": [
"__nbid__ = '0041'\n",
"__author__ = 'Brian Merino , Vinicius Placco '\n",
- "__version__ = '20241209' # yyyymmdd; version datestamp of this notebook\n",
+ "__version__ = '20250912' # yyyymmdd; version datestamp of this notebook\n",
"__keywords__ = ['gmos','gemini','longslit','whitedwarf','dragons']"
]
},
@@ -21,7 +21,7 @@
"# Gemini GMOS longslit white dwarf reduction using DRAGONS Python API\n",
"***\n",
"## Public archival data from GS-2017B-Q-72 (J2145+0031)\n",
- "#### adapted from https://dragons.readthedocs.io/projects/gmosls-drtutorial/en/v3.2.1/ex1_gmosls_dithered_api.html\n",
+ "#### adapted from https://dragons.readthedocs.io/projects/gmosls-drtutorial/en/stable/ex1_gmosls_dithered_api.html\n",
"***"
]
},
@@ -54,11 +54,11 @@
"\n",
"\n",
"# Goals\n",
- "Showcase how to reduce GMOS longslit data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel `\"DRAGONS (Py3.7)\"`. The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.\n",
+ "Showcase how to reduce GMOS longslit data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel `\"DRAGONS-4.0.0 (DL,Py3.12)\"`. The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.\n",
"\n",
"\n",
"# Summary\n",
- "DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.\n",
+ "DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.\n",
"\n",
"\n",
"The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines. \n",
@@ -777,9 +777,9 @@
],
"metadata": {
"kernelspec": {
- "display_name": "DRAGONS-3.2.2 (DL,Py3.10.14)",
+ "display_name": "DRAGONS-4.0.0 (DL,Py3.12)",
"language": "python",
- "name": "dragons-3.2.2"
+ "name": "dragons-4.0.0"
},
"language_info": {
"codemirror_mode": {
@@ -791,7 +791,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.10.14"
+ "version": "3.12.10"
}
},
"nbformat": 4,
diff --git a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GNIRS_Imaging_GammaRayBurst/GNIRS_Imaging_GammaRayBurst.html b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GNIRS_Imaging_GammaRayBurst/GNIRS_Imaging_GammaRayBurst.html
index f9f3e0bf7..248ed52e1 100644
--- a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GNIRS_Imaging_GammaRayBurst/GNIRS_Imaging_GammaRayBurst.html
+++ b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GNIRS_Imaging_GammaRayBurst/GNIRS_Imaging_GammaRayBurst.html
@@ -7312,6 +7312,7 @@
processEnvironments: true
},
displayAlign: 'center',
+ messageStyle: 'none',
CommonHTML: {
linebreaks: {
automatic: true
@@ -7519,7 +7520,7 @@
__nbid__ = '0042'
__author__ = 'Brian Merino <brian.merino@noirlab.edu>, Vinicius Placco <vinicius.placco@noirlab.edu>'
-__version__ = '20241209' # yyyymmdd; version datestamp of this notebook
+__version__ = '20250912' # yyyymmdd; version datestamp of this notebook
__keywords__ = ['gnirs','gemini','GRB','dragons']
@@ -7534,7 +7535,7 @@
Gemini GNIRS gamma ray burst reduction using DRAGONS Python API¶
-Public archival data from gnirsimg_tutorial - GN-2011B-Q-34 (GRB120116A)¶
adapted from https://dragons.readthedocs.io/projects/gnirsimg-drtutorial/en/v3.2.1/ex1_gnirsim_twostars_api.html¶
+Public archival data from gnirsimg_tutorial - GN-2011B-Q-34 (GRB120116A)¶
adapted from https://dragons.readthedocs.io/projects/gnirsimg-drtutorial/en/stable/ex1_gnirsim_twostars_api.html¶
@@ -7556,15 +7557,16 @@ Table of contentsSet up the DRAGONS logger
Create File Lists
Create Master Dark
+Bad Pixel Mask
Create Master Flat Field
Reduce Science Images
Display stacked final image
Clean-up (optional)
-Goals¶
Showcase how to perform GNIRS keyhole imaging data reduction using the Gemini DRAGONS package on the Data Lab science platform. Uses a custom DRAGONS kernel "DRAGONS (Py3.7)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing of flats, darks, and science frames, and finally the creation of a single combined stacked image.
+Goals¶
Showcase how to perform GNIRS keyhole imaging data reduction using the Gemini DRAGONS package on the Data Lab science platform. Uses a custom DRAGONS kernel "DRAGONS-4.0.0 (DL,Py3.12)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing of flats, darks, and science frames, and finally the creation of a single combined stacked image.
-Summary¶
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
+Summary¶
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines.
In this notebook, we present an example of a DRAGONS Jupyter notebook that works in the Data Lab environment to reduce example Gemini North GNIRS J-band imaging data fully. This notebook will not present all of the details of the many options available to adjust or optimize the DRAGONS GNIRS data reduction process; rather, it will just show one example of a standard reduction of a GNIRS imaging dataset.
The data used in this notebook example is GNIRS J-band keyhole imaging from the Gemini archive of the gamma-ray burst GRB120116A from the Gemini North program "Investigating gamma-ray bursts and their use as cosmological probes", PI: Nial Tanvir, program ID GN-2011B-Q-34.
@@ -7622,22 +7624,22 @@ Importing Python librariesIn [ ]:
-import glob
-import os
-import shutil
+import glob
+import os
+import shutil
-from recipe_system import cal_service
-from recipe_system.reduction.coreReduce import Reduce
-from recipe_system.utils.reduce_utils import normalize_ucals
+from recipe_system import cal_service
+from recipe_system.reduction.coreReduce import Reduce
+from recipe_system.utils.reduce_utils import normalize_ucals
-from gempy.adlibrary import dataselect
-from gempy.utils import logutils
+from gempy.adlibrary import dataselect
+from gempy.utils import logutils
-from astropy.io import fits
-from astropy.wcs import WCS
+from astropy.io import fits
+from astropy.wcs import WCS
-import matplotlib.pyplot as plt
-from matplotlib.colors import Normalize
+import matplotlib.pyplot as plt
+from matplotlib.colors import Normalize
@@ -7663,7 +7665,7 @@ Prepare the working directoryIn [ ]:
-def clean_up(save_reduced=0):
+def clean_up(save_reduced=0):
#Does the calibrations directory already exist?
caldb_Exist = os.path.exists('./calibrations')
@@ -7752,6 +7754,11 @@ About the datasetN20120117S0042-49
lamps-off
+
+BPM
+bpm_20100716_gnirs_gnirsn_11_full_1amp.fits
+
A note about finding the darks in the GOA: GNIRS is not an imager and imaging through the keyhole is done only in extreme circumstances, so the archive does not have calibration association rules for the darks. One needs to search for the darks manually. Here is the search that was done to find the darks for this observation sequence:
@@ -7835,7 +7842,8 @@ Downloading the datahttp://archive.gemini.edu/file/N20120117S0046.fits
http://archive.gemini.edu/file/N20120117S0047.fits
http://archive.gemini.edu/file/N20120117S0048.fits
-http://archive.gemini.edu/file/N20120117S0049.fits\
+http://archive.gemini.edu/file/N20120117S0049.fits
+http://archive.gemini.edu/file/bpm_20100716_gnirs_gnirsn_11_full_1amp.fits\
" > gnirs.list
@@ -7997,6 +8005,35 @@ Create master dark
+
+Bad pixel mask¶
Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as calibrations. They are downloadable from the archive instead of being packaged with the software. They are automatically associated like any other calibrations. This means that the user now must download the BPMs along with the other calibrations and add the BPMs to the local calibration manager.
+See Getting Bad Pixel Masks from the archive in Tips and Tricks to learn about the various ways to get the BPMs from the archive.
+To add the BPM included in the data package to the local calibration database:
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+for bpm in dataselect.select_data(all_files, ['BPM']):
+ caldb.add_cal(bpm)
+
+
+
+
+
+
+
+
+
+
+
+
Create master flat field¶
A GNIRS master flat is created from a series of lamp-on and lamp-off exposures. Each flavor is stacked, and then the lamp-off stack is subtracted from the lamp-on stack.
The master flat will be saved with the suffix _flat.fits.
diff --git a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GNIRS_Imaging_GammaRayBurst/GNIRS_Imaging_GammaRayBurst.ipynb b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GNIRS_Imaging_GammaRayBurst/GNIRS_Imaging_GammaRayBurst.ipynb
index b8d6bc509..b9fb6d0b3 100644
--- a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GNIRS_Imaging_GammaRayBurst/GNIRS_Imaging_GammaRayBurst.ipynb
+++ b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GNIRS_Imaging_GammaRayBurst/GNIRS_Imaging_GammaRayBurst.ipynb
@@ -8,7 +8,7 @@
"source": [
"__nbid__ = '0042'\n",
"__author__ = 'Brian Merino , Vinicius Placco '\n",
- "__version__ = '20241209' # yyyymmdd; version datestamp of this notebook\n",
+ "__version__ = '20250912' # yyyymmdd; version datestamp of this notebook\n",
"__keywords__ = ['gnirs','gemini','GRB','dragons']"
]
},
@@ -21,7 +21,7 @@
"# Gemini GNIRS gamma ray burst reduction using DRAGONS Python API\n",
"***\n",
"## Public archival data from gnirsimg_tutorial - GN-2011B-Q-34 (GRB120116A)\n",
- "#### adapted from https://dragons.readthedocs.io/projects/gnirsimg-drtutorial/en/v3.2.1/ex1_gnirsim_twostars_api.html\n",
+ "#### adapted from https://dragons.readthedocs.io/projects/gnirsimg-drtutorial/en/stable/ex1_gnirsim_twostars_api.html\n",
"***"
]
},
@@ -40,6 +40,7 @@
"* [Set up the DRAGONS logger](#DRAGONS_logger)\n",
"* [Create File Lists](#File_Lists)\n",
"* [Create Master Dark](#Master_dark)\n",
+ "* [Bad Pixel Mask](#BPM)\n",
"* [Create Master Flat Field](#Master_Flat)\n",
"* [Reduce Science Images](#Reduce_Science)\n",
"* [Display stacked final image](#Display_Image)\n",
@@ -47,11 +48,11 @@
"\n",
"\n",
"# Goals\n",
- "Showcase how to perform GNIRS keyhole imaging data reduction using the Gemini DRAGONS package on the Data Lab science platform. Uses a custom DRAGONS kernel `\"DRAGONS (Py3.7)\"`. The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing of flats, darks, and science frames, and finally the creation of a single combined stacked image.\n",
+ "Showcase how to perform GNIRS keyhole imaging data reduction using the Gemini DRAGONS package on the Data Lab science platform. Uses a custom DRAGONS kernel `\"DRAGONS-4.0.0 (DL,Py3.12)\"`. The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing of flats, darks, and science frames, and finally the creation of a single combined stacked image.\n",
"\n",
"\n",
"# Summary\n",
- "DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.\n",
+ "DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.\n",
"\n",
"The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines. \n",
"\n",
@@ -207,6 +208,7 @@
"| Science darks | N20120102S0538-547 | 60 s, like Science |\n",
"| Flats | N20120117S0034-41 | lamps-on |\n",
"| Flats | N20120117S0042-49 | lamps-off |\n",
+ "| BPM | bpm_20100716_gnirs_gnirsn_11_full_1amp.fits | |\n",
"\n",
"A note about finding the darks in the GOA: GNIRS is not an imager and imaging through the keyhole is done only in extreme circumstances, so the archive does not have calibration association rules for the darks. One needs to search for the darks manually. Here is the search that was done to find the darks for this observation sequence:\n",
"\n",
@@ -284,7 +286,8 @@
"http://archive.gemini.edu/file/N20120117S0046.fits\n",
"http://archive.gemini.edu/file/N20120117S0047.fits\n",
"http://archive.gemini.edu/file/N20120117S0048.fits\n",
- "http://archive.gemini.edu/file/N20120117S0049.fits\\\n",
+ "http://archive.gemini.edu/file/N20120117S0049.fits\n",
+ "http://archive.gemini.edu/file/bpm_20100716_gnirs_gnirsn_11_full_1amp.fits\\\n",
"\" > gnirs.list"
]
},
@@ -406,6 +409,30 @@
"reduce_darks.runr()"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## **Bad pixel mask**\n",
+ "\n",
+ "Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as calibrations. They are downloadable from the archive instead of being packaged with the software. They are automatically associated like any other calibrations. This means that the user now must download the BPMs along with the other calibrations and add the BPMs to the local calibration manager.\n",
+ "\n",
+ "See [Getting Bad Pixel Masks from the archive](https://dragons.readthedocs.io/projects/gnirsimg-drtutorial/en/stable/05_tips_and_tricks.html#getbpm) in Tips and Tricks to learn about the various ways to get the BPMs from the archive.\n",
+ "\n",
+ "To add the BPM included in the data package to the local calibration database:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "for bpm in dataselect.select_data(all_files, ['BPM']):\n",
+ " caldb.add_cal(bpm)"
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
@@ -452,7 +479,9 @@
{
"cell_type": "code",
"execution_count": null,
- "metadata": {},
+ "metadata": {
+ "scrolled": true
+ },
"outputs": [],
"source": [
"reduce_target = Reduce()\n",
@@ -534,9 +563,9 @@
],
"metadata": {
"kernelspec": {
- "display_name": "DRAGONS-3.2.2 (DL,Py3.10.14)",
+ "display_name": "DRAGONS-4.0.0 (DL,Py3.12)",
"language": "python",
- "name": "dragons-3.2.2"
+ "name": "dragons-4.0.0"
},
"language_info": {
"codemirror_mode": {
@@ -548,7 +577,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.10.14"
+ "version": "3.12.10"
}
},
"nbformat": 4,
diff --git a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GSAOI_Imaging_EllipticalGalaxy/GSAOI_Imaging_EllipticalGalaxy.html b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GSAOI_Imaging_EllipticalGalaxy/GSAOI_Imaging_EllipticalGalaxy.html
index f5af7ba00..64893aa5d 100644
--- a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GSAOI_Imaging_EllipticalGalaxy/GSAOI_Imaging_EllipticalGalaxy.html
+++ b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GSAOI_Imaging_EllipticalGalaxy/GSAOI_Imaging_EllipticalGalaxy.html
@@ -7520,7 +7520,7 @@
__nbid__ = '0043'
__author__ = 'Brian Merino <brian.merino@noirlab.edu>, Vinicius Placco <vinicius.placco@noirlab.edu>'
-__version__ = '20250709' # yyyymmdd; version datestamp of this notebook
+__version__ = '20250912' # yyyymmdd; version datestamp of this notebook
__keywords__ = ['gsaoi','gemini','galaxy','dragons']
@@ -7535,7 +7535,7 @@
Gemini GSAOI elliptical galaxy reduction using DRAGONS Python API¶
-Public archival data from gsaoiimg_tutorial - GS-2017A-Q-29 (NGC5128)¶
adapted from https://dragons.readthedocs.io/projects/gsaoiimg-drtutorial/en/v3.2.1/index.html¶
Disclaimer: Due to the large size of GSAOI images, this notebook will likely take 20+ minutes to finish running.¶
+Public archival data from gsaoiimg_tutorial - GS-2017A-Q-29 (NGC5128)¶
adapted from https://dragons.readthedocs.io/projects/gsaoiimg-drtutorial/en/stable/¶
Disclaimer: Due to the large size of GSAOI images, this notebook will likely take 20+ minutes to finish running.¶
@@ -7563,9 +7563,9 @@ Table of contentsClean-up (optional)
-Goals¶
Showcase how to reduce GSAOI imaging data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS (Py3.7)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.
+Goals¶
Showcase how to reduce GSAOI imaging data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel "DRAGONS-4.0.0 (DL,Py3.12)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.
-Summary¶
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
+Summary¶
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines.
In this notebook, we present an example of a DRAGONS Jupyter notebook that works in the Data Lab environment to reduce example Gemini South NIRI Kshort-band imaging data fully. This notebook will not present all of the details of the many options available to adjust or optimize the DRAGONS GSAOI data reduction process; rather, it will just show one example of a standard reduction of a GSAOI imaging dataset.
The data used in this notebook example is GSAOI KshortH-band imaging from the Gemini archive of the giant elliptical galaxy NGC 5128 from the Gemini South program "Stellar Population GeMology: Long Period Variables at High Metallicity in the Nearest Elliptical Galaxy", PI: John Blakeslee, program ID GS-2017A-Q-29.
@@ -7932,7 +7932,8 @@ Setting up the DRAGONS logger
-Add the Bad Pixel Masks to the calibration database¶
+
+Create file lists¶
The first step is to create input file lists. The tool "dataselect" helps with that. It uses Astrodata tags and "descriptors" to select the files and send the filenames to a text file that can then be fed to "reduce". (See the Astrodata User Manual for information about Astrodata.)
@@ -7944,7 +7945,17 @@ Add the Bad Pixel M
In [ ]:
-caldb.add_cal(glob.glob('bpm*.fits')[0])
+list_of_flats_Ks = dataselect.select_data(
+ all_files,['FLAT'],[],
+ dataselect.expr_parser('filter_name=="Kshort"'))
+
+list_of_std_stars = dataselect.select_data(
+ all_files,[],[],
+ dataselect.expr_parser('observation_class=="partnerCal"'))
+
+list_of_science_images = dataselect.select_data(
+ all_files,[],[],
+ dataselect.expr_parser('(observation_class=="science" and exposure_time==60.)'))
@@ -7957,8 +7968,9 @@ Add the Bad Pixel M
-
-Create file lists¶
The first step is to create input file lists. The tool "dataselect" helps with that. It uses Astrodata tags and "descriptors" to select the files and send the filenames to a text file that can then be fed to "reduce". (See the Astrodata User Manual for information about Astrodata.)
+Add the Bad Pixel Masks to the calibration database¶
Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as calibrations. They are downloadable from the archive instead of being packaged with the software. They are automatically associated like any other calibrations. This means that the user now must download the BPMs along with the other calibrations and add the BPMs to the local calibration manager.
+See Getting Bad Pixel Masks from the archive in Tips and Tricks to learn about the various ways to get the BPMs from the archive.
+To add the BPM included in the data package to the local calibration database:
@@ -7970,17 +7982,7 @@ Create file listsIn [ ]:
-list_of_flats_Ks = dataselect.select_data(
- all_files,['FLAT'],[],
- dataselect.expr_parser('filter_name=="Kshort"'))
-
-list_of_std_stars = dataselect.select_data(
- all_files,[],[],
- dataselect.expr_parser('observation_class=="partnerCal"'))
-
-list_of_science_images = dataselect.select_data(
- all_files,[],[],
- dataselect.expr_parser('(observation_class=="science" and exposure_time==60.)'))
+caldb.add_cal(glob.glob('bpm*.fits')[0])
@@ -8132,7 +8134,7 @@ Display the stacked imageIn [ ]:
-plt.figure(figsize = (15,15))
+plt.figure(figsize = (10,8))
plt.subplot(projection=wcs)
plt.imshow(image_data,cmap='bone',norm=Normalize(vmin=1, vmax=1000),origin='lower')
plt.xlabel('Right Ascension [hh:mm:ss]',fontsize=14,fontweight='bold')
diff --git a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GSAOI_Imaging_EllipticalGalaxy/GSAOI_Imaging_EllipticalGalaxy.ipynb b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GSAOI_Imaging_EllipticalGalaxy/GSAOI_Imaging_EllipticalGalaxy.ipynb
index c5a531dab..61ea2e24f 100644
--- a/04_HowTos/DataReduction/DRAGONS_reduction_examples/GSAOI_Imaging_EllipticalGalaxy/GSAOI_Imaging_EllipticalGalaxy.ipynb
+++ b/04_HowTos/DataReduction/DRAGONS_reduction_examples/GSAOI_Imaging_EllipticalGalaxy/GSAOI_Imaging_EllipticalGalaxy.ipynb
@@ -8,7 +8,7 @@
"source": [
"__nbid__ = '0043'\n",
"__author__ = 'Brian Merino , Vinicius Placco '\n",
- "__version__ = '20250709' # yyyymmdd; version datestamp of this notebook\n",
+ "__version__ = '20250912' # yyyymmdd; version datestamp of this notebook\n",
"__keywords__ = ['gsaoi','gemini','galaxy','dragons']"
]
},
@@ -19,7 +19,7 @@
"# Gemini GSAOI elliptical galaxy reduction using DRAGONS Python API\n",
"***\n",
"## Public archival data from gsaoiimg_tutorial - GS-2017A-Q-29 (NGC5128)\n",
- "#### adapted from https://dragons.readthedocs.io/projects/gsaoiimg-drtutorial/en/v3.2.1/index.html\n",
+ "#### adapted from https://dragons.readthedocs.io/projects/gsaoiimg-drtutorial/en/stable/\n",
"\n",
"#### **Disclaimer: Due to the large size of GSAOI images, this notebook will likely take 20+ minutes to finish running.**\n",
"***"
@@ -48,11 +48,11 @@
"\n",
"\n",
"# Goals\n",
- "Showcase how to reduce GSAOI imaging data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel `\"DRAGONS (Py3.7)\"`. The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.\n",
+ "Showcase how to reduce GSAOI imaging data using the Gemini DRAGONS package on the Data Lab science platform using a custom DRAGONS kernel `\"DRAGONS-4.0.0 (DL,Py3.12)\"`. The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.\n",
"\n",
"\n",
"# Summary\n",
- "DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.\n",
+ "DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.\n",
"\n",
"The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines. \n",
"\n",
@@ -349,22 +349,6 @@
"all_files.sort()"
]
},
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Add the Bad Pixel Masks to the calibration database"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "caldb.add_cal(glob.glob('bpm*.fits')[0])"
- ]
- },
{
"cell_type": "markdown",
"metadata": {},
@@ -394,6 +378,28 @@
" dataselect.expr_parser('(observation_class==\"science\" and exposure_time==60.)'))"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Add the Bad Pixel Masks to the calibration database\n",
+ "\n",
+ "Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as calibrations. They are downloadable from the archive instead of being packaged with the software. They are automatically associated like any other calibrations. This means that the user now must download the BPMs along with the other calibrations and add the BPMs to the local calibration manager.\n",
+ "\n",
+ "See [Getting Bad Pixel Masks from the archive](https://dragons.readthedocs.io/projects/gnirsimg-drtutorial/en/stable/05_tips_and_tricks.html#getbpm) in Tips and Tricks to learn about the various ways to get the BPMs from the archive.\n",
+ "\n",
+ "To add the BPM included in the data package to the local calibration database:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "caldb.add_cal(glob.glob('bpm*.fits')[0])"
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
@@ -522,7 +528,7 @@
"metadata": {},
"outputs": [],
"source": [
- "plt.figure(figsize = (15,15))\n",
+ "plt.figure(figsize = (10,8))\n",
"plt.subplot(projection=wcs)\n",
"plt.imshow(image_data,cmap='bone',norm=Normalize(vmin=1, vmax=1000),origin='lower')\n",
"plt.xlabel('Right Ascension [hh:mm:ss]',fontsize=14,fontweight='bold')\n",
diff --git a/04_HowTos/DataReduction/DRAGONS_reduction_examples/NIRI_Imaging_Supernova/NIRI_Imaging_Supernova.html b/04_HowTos/DataReduction/DRAGONS_reduction_examples/NIRI_Imaging_Supernova/NIRI_Imaging_Supernova.html
index dbde3b88e..8b2976955 100644
--- a/04_HowTos/DataReduction/DRAGONS_reduction_examples/NIRI_Imaging_Supernova/NIRI_Imaging_Supernova.html
+++ b/04_HowTos/DataReduction/DRAGONS_reduction_examples/NIRI_Imaging_Supernova/NIRI_Imaging_Supernova.html
@@ -7312,6 +7312,7 @@
processEnvironments: true
},
displayAlign: 'center',
+ messageStyle: 'none',
CommonHTML: {
linebreaks: {
automatic: true
@@ -7519,7 +7520,7 @@
__nbid__ = '0044'
__author__ = 'Brian Merino <brian.merino@noirlab.edu>, Vinicius Placco <vinicius.placco@noirlab.edu>'
-__version__ = '20241209' # yyyymmdd; version datestamp of this notebook
+__version__ = '20250912' # yyyymmdd; version datestamp of this notebook
__keywords__ = ['niri','gemini','supernova','dragons']
@@ -7534,7 +7535,7 @@
Gemini NIRI supernova reduction using DRAGONS Python API¶
-Public archival data from niriimg_tutorial - GN-2015B-Q-31 (SN2014J)¶
adapted from https://dragons.readthedocs.io/projects/niriimg-drtutorial/en/v3.2.1/ex1_niriim_extended_api.html¶
+Public archival data from niriimg_tutorial - GN-2015B-Q-31 (SN2014J)¶
adapted from https://dragons.readthedocs.io/projects/niriimg-drtutorial/en/stable/ex1_niriim_extended_api.html¶
@@ -7564,9 +7565,9 @@ Table of contentsClean-up (optional)
-Goals¶
Showcase how to reduce NIRI imaging data using the Gemini DRAGONS package on the Data Lab science platform, using a custom DRAGONS kernel "DRAGONS (Py3.7)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.
+Goals¶
Showcase how to reduce NIRI imaging data using the Gemini DRAGONS package on the Data Lab science platform, using a custom DRAGONS kernel "DRAGONS-4.0.0 (DL,Py3.12)". The steps include downloading data from the Gemini archive, setting up a DRAGONS calibration service, processing flats, darks, a bad pixel mask, and science frames, and creating a single combined stacked image.
-Summary¶
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GHOST and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
+Summary¶
DRAGONS is a Python-based astronomical data reduction platform written by the Gemini Science User Support Department. It can currently be used to reduce imaging data from Gemini instruments GMOS, NIRI, Flamingos 2, GSAOI, and GNIRS, as well as spectroscopic data taken with GNIRS, GHOST, and GMOS in longslit mode. Linked here is a general list of guides, manuals, and tutorials about the use of DRAGONS.
The DRAGONS kernel has been made available in the Data Lab environment, allowing users to access the routines without being dependent on installing the software on their local machines.
In this notebook, we present an example of a DRAGONS Jupyter notebook that works in the Data Lab environment to reduce example Gemini North NIRI H-band imaging data fully. This notebook will not present all of the details of the many options available to adjust or optimize the DRAGONS NIRI data reduction process; instead, it will just show one example of a standard reduction of a NIRI imaging dataset.
The data used in this notebook example is NIRI H-band imaging from the Gemini archive of the Supernova SN2014J from the Gemini North program "SN2014J at very late phases", PI: Marten van Kerkwijk, program ID GN-2015B-Q-31.
@@ -7624,23 +7625,23 @@ Importing Python librariesIn [ ]:
-import warnings
-import glob
-import os
-import shutil
+import warnings
+import glob
+import os
+import shutil
-from gempy.adlibrary import dataselect
-from gempy.utils import logutils
+from gempy.adlibrary import dataselect
+from gempy.utils import logutils
-from recipe_system import cal_service
-from recipe_system.reduction.coreReduce import Reduce
+from recipe_system import cal_service
+from recipe_system.reduction.coreReduce import Reduce
-from astropy.io import fits
-from astropy.wcs import WCS
-from astropy.utils.exceptions import AstropyWarning
+from astropy.io import fits
+from astropy.wcs import WCS
+from astropy.utils.exceptions import AstropyWarning
-import matplotlib.pyplot as plt
-from matplotlib.colors import Normalize
+import matplotlib.pyplot as plt
+from matplotlib.colors import Normalize
warnings.simplefilter('ignore', category=AstropyWarning)
@@ -7668,7 +7669,7 @@ Prepare the working directoryIn [ ]:
-def clean_up(save_reduced=0):
+def clean_up(save_reduced=0):
#Does the calibrations directory already exist?
caldb_Exist = os.path.exists('./calibrations')
@@ -7772,6 +7773,11 @@ About the datasetN20160102S0295-299
+BPM
+bpm_20010317_niri_niri_11_full_1amp.fits
+
@@ -7856,7 +7862,8 @@ Downloading the datahttp://archive.gemini.edu/file/N20160102S0296.fits
http://archive.gemini.edu/file/N20160102S0297.fits
http://archive.gemini.edu/file/N20160102S0298.fits
-http://archive.gemini.edu/file/N20160102S0299.fits\
+http://archive.gemini.edu/file/N20160102S0299.fits
+http://archive.gemini.edu/file/bpm_20010317_niri_niri_11_full_1amp.fits\
" > niri.list
@@ -8031,9 +8038,37 @@ Create master dark
-Bad pixel mask¶
The DRAGONS Gemini data reduction package, geminidr, comes with a static NIRI bad pixel mask (BPM) that gets automatically added to all the NIRI data as they gets processed. The user can also create a supplemental, fresher BPM from the flats and recent short darks. That new BPM is later fed to "reduce" as a user BPM to be combined with the static BPM. Using the static and fresh BPM from recent data leads to a better representation of the bad pixels. It is an optional but recommended step.
+Bad pixel mask¶
Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as calibrations. They are downloadable from the archive instead of being packaged with the software. They are automatically associated like any other calibrations. This means that the user now must download the BPMs along with the other calibrations and add the BPMs to the local calibration manager.
+See Getting Bad Pixel Masks from the archive in Tips and Tricks to learn about the various ways to get the BPMs from the archive.
+To add the BPM included in the data package to the local calibration database:
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+for bpm in dataselect.select_data(all_files, ['BPM']):
+ caldb.add_cal(bpm)
+
+
+
+
+
+
+
+
+
+
+
+
+You can also create a supplemental, fresher BPM from the flats and recent short darks. That new BPM is later fed to "reduce" as a user BPM to be combined with the static BPM. Using the static and fresh BPM from recent data leads to a better representation of the bad pixels. It is an optional but recommended step.
The flats and the short darks are the inputs.
-The flats must be passed to the input list first to ensure that the recipe library associated with NIRI flats is selected. We will not use the default recipe, but rather the special recipe from that library called makeProcessedBPM.
+The flats must be passed to the input list first to ensure that the recipe library associated with NIRI flats is selected. We will use the special recipe from that library called makeProcessedBPM.
The BPM produced is named N20160102S0373_bpm.fits.
@@ -8232,7 +8267,7 @@ Display the stacked imageIn [ ]:
-plt.figure(figsize = (10,10))
+plt.figure(figsize = (10,8))
plt.subplot(projection=wcs)
plt.imshow(image_data,cmap='bone',norm=Normalize(vmin=1, vmax=50000),origin='lower')
plt.xlabel('Right Ascension [hh:mm:ss]',fontsize=14,fontweight='bold')
@@ -8269,6 +8304,20 @@ Optional: re
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+
+
+