RCAL-959 COSMOS and GAIA Sources in Catalogs

[PaulHuwe]

Adding methods to select random COSMOS galaxies and/or GAIA stars to make more realistic catalogs.

[codecov]

Codecov Report

Attention: Patch coverage is 86.11111% with 15 lines in your changes missing coverage. Please review.

Project coverage is 89.63%. Comparing base (0dd191e) to head (aff3e51).
Report is 23 commits behind head on main.

Files with missing lines	Patch %	Lines
romanisim/catalog.py	86.11%	15 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #199      +/-   ##
==========================================
+ Coverage   80.71%   89.63%   +8.91%     
==========================================
  Files          17       17              
  Lines        2287     2277      -10     
==========================================
+ Hits         1846     2041     +195     
+ Misses        441      236     -205     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[PaulHuwe]

Sample Level 3 Mosaic with radius = 0.1 degree (can see the area search):

[PaulHuwe]

Histogram of stellar and galaxy densities (wrt. all objects):

Same histogram of stellar and galaxy densities, but wrt. their individual classes:

[PaulHuwe]

Histogram of rsim's selected cosmos sources vs. the full cosmos catalog (densities wrt. each dataset):

[PaulHuwe]

Setting this ready for review, as the failing tests are in main, not this PR.

[schlafly]

This looks good. Can you add a section to the documentation with maybe your image and a little bit of information? i.e.:

• density of sources
• method for simulating Roman fluxes (linear interpolation among ultra vista wavelengths)
• depth of simulation (incomplete beyond mag ~ 25 IIRC?---not super deep)
• basic simulation parameters (random angles, flat distribution of sersic indices, sizes from COSMOS HST imaging)
• ... something else I'm forgetting?
• real Gaia stars with approximate fluxes (no real colors)

Re the density, can you remind me if you think it is right? Right now you compute the area as (ramax - ramin) * (decmax - decmin), and then get the density as the number divided by the area. I worry that the ultra vista area is only a portion of the whole field, and so (ramax - ramin) * (decmax - decmin) isn't a good approximation of the area that you end up keeping after the cuts. Did you check that?

Can you also say what ends up in the "streamlined" catalog? The size is reasonable so it's okay from that perspective, but we should probably ask permission before providing it as an alternative source of the COSMOS catalog unless it's sufficiently stripped down as to have limited scientific value beyond our use case. We should also cite the COSMOS papers extensively in your new docs.

[PaulHuwe]

The streamlined catalog has had the following cuts from the full COSMOS catalog:

• Trimmed to only the columns we need: 'FLUX_RADIUS', 'DELTA_J2000', 'UVISTA_H_FLUX_AUTO', 'HSC_r_FLUX_AUTO', 'HSC_z_FLUX_AUTO', 'lp_type','UVISTA_Y_FLUX_AUTO', 'ID', 'ALPHA_J2000', 'UVISTA_J_FLUX_AUTO', 'UVISTA_Ks_FLUX_AUTO', 'ACS_A_WORLD', 'FLAG_UDEEP', 'ACS_B_WORLD'
• Only keep 'lp_type" galaxies and Xray sources
• Trim to only those that contain the ultra-deep regions of UltraVISTA data
• Keep those with positive, nonzero flux radius, shape measurements (ACS_A_WORLD, ACS_B_WORLD )

These cuts trim the total number of objects from 1720700 to 345994. The filters in particular are very Roman-specific.

[schlafly]

This looks good. Can I ask you to make one 3-color image using your new catalog? I'd simulate an image in each of three bands, read each of the images into memory, and use this function
https://github.com/spacetelescope/romanisim/blob/main/romanisim/util.py#L74
to try to get a good scaling. Mostly it would be good to see that the colors look kind of realistic. It would be good to replace the current black/white image with this one.

[schlafly]

Image looks good, thanks. Can you help me with
#199 (comment)
?

[schlafly]

Merge when ready, thanks!

[PaulHuwe]

I have removed the duplicate code in romanisim-gaia-catalog and confirmed the output remained effectively unchanged.