Power spectrum responses for SSC #1134

RyoTerasawa · 2023-11-06T00:23:10Z

In pyccl/pkresponse.py, I implemented the functions to calculate power spectrum responses to the super-survey modes which are responsible for super-sample covariance (SSC), based on arXiv:2310.13330.
We approximate the power spectrum growth response to the super-survey modes by its growth response to the Hubble parameter.

For the galaxy-matter and galaxy-auto power spectra, the halo statistics (halo-matter/halo-auto power spectrum, mass function) are calculated by DarkEmulator.

coveralls · 2023-12-04T08:34:58Z

Pull Request Test Coverage Report for Build 12132306089

Details

304 of 346 (87.86%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.5%) to 96.946%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pyccl/pkresponse.py	303	345	87.83%

Totals
Change from base Build 11958349869:	-0.5%
Covered Lines:	6824
Relevant Lines:	7039

💛 - Coveralls

damonge · 2024-02-21T08:35:24Z

@RyoTerasawa can I check what the status of this PR is?

RyoTerasawa · 2024-02-21T11:17:50Z

I am waiting for someone to review the code. I haven't asked for a specific person yet, but maybe I should ask a member of the MCPCov group to review it. I am happy to explain the details if needed. I am also working on writing the unit test and benchmark test to be included in this PR.

carlosggarcia · 2024-04-08T14:47:11Z

@RyoTerasawa @YueNan-c, maybe you want to also implement the DarkEmulator here: https://github.com/LSSTDESC/CCL/blob/master/pyccl/emulators/cosmicemu_pk.py.

Are the unit tests and benchmark ready?

damonge · 2024-06-18T08:51:18Z

@RyoTerasawa @carlosggarcia can I check what the status of this is?

RyoTerasawa · 2024-06-22T12:57:20Z

@carlosggarcia @damonge
Sorry for the late reply. I just added the unit tests and benchmark, and this pull request is ready to be reviewed.

carlosggarcia

Ok. I first round of comments. Some suggestions:

Add unit tests per function/method declared to be sure they are doing what they should
I would unify the .npy files in a single .npz file. Also, it seems that you are only using the z=0 case, I'd be great if you can compare all redshift, since you have it.
Add References to the equations in your paper for later reference (and to make my life easier when reviewing the PR)
@damonge, there are integrals and biases computed that I think they might be obtained from the current CCL implementation. Can you have a look? The are in pkresponse.py, where I pinged you, too.

In general, it looks good, though! Good work!

carlosggarcia · 2024-07-03T10:54:29Z

pyccl/tests/test_pkresponse.py

+def test_Pmm_resp():
+    response = Pmm_resp(cosmo, deltah=deltah, lk_arr=lk_arr, a_arr=a_arr)
+
+    assert np.all(np.isfinite(response)), "Pmm_resp produced infinity values."


What is this "Pmm_resp produced infinity values." doing?

I realized the error message is not needed, so I simply deleted the message.

carlosggarcia · 2024-07-03T10:58:35Z

pyccl/pkresponse.py

+def Pmm_resp(
+    cosmo,
+    deltah=0.02,
+    extra_parameters={


These aren't used anywhere.

I added dependency to extra_parameters.

carlosggarcia · 2024-07-03T11:01:25Z

pyccl/pkresponse.py

+
+
+def set_hmodified_cosmology(cosmo, deltah):
+    Omega_c = cosmo["Omega_c"]


hmm, and what if there are extra parameters given, eg. massive neutrinos? Wouldn't that be relevant? I guess you should make sure the cosmology object is exactly the same, except for the h

In the massive neutrinos' case, m_nu should be the same. I edited the function set_hmodified_cosmology to explicitly make sure the cosmology object is exactly same except for the h, by starting with the copy of the fiducial cosmology object through its keys.

carlosggarcia · 2024-07-03T11:13:09Z

pyccl/pkresponse.py

+    return b2
+
+
+def darkemu_set_cosmology(emu, cosmo):


Add and underscore if these methods are meant to be private (so not used by other libraries); e.g. _darkemu_set_cosmology. Also, add documentation. Same with the other "utility functions".

I made all the utility functions private. Also added documentation.

carlosggarcia · 2024-07-03T11:17:02Z

pyccl/tests/test_pkresponse.py

In general, you should add a unit test for each function that you have defined in pkresponse.py to check that they are working properly. For instance, for set_hmodified_cosmology you would like to check that the output cosmologies are exactly the same as the input one, except for the h's.

I added a unit test for all functions defined in pkresponse.py. For set_hmodified_cosmology the unit test checks that the output cosmologies are exactly the same as the input one, except for the h's.

carlosggarcia · 2024-07-03T14:14:57Z

pyccl/pkresponse.py

+
+    kmin = 1e-2
+    for ia, aa in enumerate(a_arr):
+        pk = pk2d.__call__(k_use, aa, cosmo)


You don't need to use __call__, you can just call it, pk2d(k_use, aa, cosmo)

Replaced the all __call__ with pk2d() in pkresponse.py.

carlosggarcia · 2024-07-03T14:18:29Z

pyccl/pkresponse.py

+    hbf = halos.HaloBiasTinker10(mass_def=mass_def)
+
+    # dark emulator is valid for 0 =< z <= 1.48
+    if np.any(1.0 / a_arr - 1) > 1.5:


if the maximum is 1.48, check for 1.48, not 1.5

Modified the pkresponse.py to check for 1.48.

carlosggarcia · 2024-07-03T14:19:31Z

pyccl/pkresponse.py

+
+    # dark emulator is valid for 0 =< z <= 1.48
+    if np.any(1.0 / a_arr - 1) > 1.5:
+        print("dark emulator is valid for z={:.2f}<1.48")


hmm, what does {:.2f} do here? I think you have to remove that.

The {:.2f} was unnecessary. I removed that.

carlosggarcia · 2024-07-03T14:21:11Z

pyccl/pkresponse.py

+            / ng
+        )
+
+        bgE2 = (


@damonge , can this be done with the current methods of the halo model?

Same with the other integrals

I think so. @RyoTerasawa can you check:

CCL/pyccl/halos/halo_model.py

Line 108 in 5273ca6

def _integrate_over_mf(self, array_2):

and

CCL/pyccl/halos/halo_model.py

Line 113 in 5273ca6

def _integrate_over_mbf(self, array_2):

and

CCL/pyccl/halos/halo_model.py

Line 118 in 5273ca6

def integrate_over_massfunc(self, func, cosmo, a):

?

I changed the pkresponse.py, and now all the integrals of the mass function are using _integrate_over_mf or _integrate_over_mbf.

carlosggarcia · 2024-07-03T14:26:55Z

benchmarks/test_pkresponse.py

+# Construct the full path for each data file (z=0)
+k_data_path = os.path.join(data_directory_path, "k_h.npy")
+k_data_mm_path = os.path.join(data_directory_path, "k_h_mm.npy")
+Pmm_resp_data_path = os.path.join(data_directory_path, "Pmm_resp_z0.npy")


You have so much more data generated, at different redshifts. Why don't you use it?

I added the tests for higher redshifts, z ~ [0.5, 1.0, 1.5].

damonge · 2024-11-13T10:06:32Z

@carlosggarcia @RyoTerasawa can I check what the status of this PR is?

RyoTerasawa · 2024-11-15T06:03:31Z

@carlosggarcia @RyoTerasawa can I check what the status of this PR is?

I am currently working on addressing Carlos's comments.

RyoTerasawa · 2024-12-03T03:31:06Z

@carlosggarcia Thank you for the comments and suggestions.
I complete addressing these comments except for the integrals.

Added unit tests per function/method declared to be sure they are doing what they should
Unified the .npy files in a single .npz file and now the benchmark test compares all redshift.
Added References to the equations in the paper.
Have not addressed to the integrals in pkresponse.py

@damonge, there are integrals and biases computed that I think they might be obtained from the current CCL implementation. Can you have a look? The are in pkresponse.py, where I pinged you, too.

damonge · 2024-12-04T09:04:40Z

@RyoTerasawa I responded to Carlos' comment about the integrals. If possible, it'd be good if CCL does all of these consistently.

RyoTerasawa · 2024-12-08T07:35:34Z

@damonge Thank you for the suggestion. I switched to the integral method of pyccl/halos/halo_model.py and it helped to make the code bit simpler.

Merge branch 'master' into SSC

RyoTerasawa · 2024-12-08T08:17:36Z

@carlosggarcia, I complete addressing these comments. Although the build failed at the unit test.

RyoTerasawa added 4 commits November 6, 2023 07:19

add darkemulator.py

867e33c

add darkemulator mass function

0df32e9

Linting

a37dcca

delete darkemulator.py

efcdc7f

RyoTerasawa requested a review from carlosggarcia November 6, 2023 00:23

add dependency on dark_emulator

2ee5ba5

This was referenced Dec 4, 2023

Ssc terasawa #991

Closed

SSC for g-g lensing and galaxy clustering using DarkEmulator #1079

Closed

Merge remote-tracking branch 'origin/master' into SSC

9b7450d

RyoTerasawa force-pushed the SSC branch from 43d5eb5 to 9b7450d Compare June 4, 2024 05:27

damonge and others added 6 commits June 18, 2024 09:51

Merge branch 'master' into SSC

f76b69f

try bumping python

4b5f3a6

try george in conda

a382551

Merge branch 'master' into SSC

5f8d35d

add unit test and benchmark test

ca3dcd9

lint

0341150

carlosggarcia requested changes Jul 3, 2024

View reviewed changes

RyoTerasawa and others added 5 commits December 2, 2024 01:58

Merge branch 'master' into SSC

12fe1d1

Add References to the equations

a25862e

switch to MassFuncNishimichi19

08af485

add dependency to extra_parameters

46e92de

make sure the cosmology object is exactly the same

d4a21ca

RyoTerasawa added 6 commits December 2, 2024 07:33

create pkresponse.npz

f7347fe

update benchmark

896f812

add documentation

59ea6ea

make utility functions private and add unit tests for them

a205e5e

update test_pkresponse.py

de64ba3

lint

1928d7c

switched to halo model integral

2925dac

Merge pull request #1217 from LSSTDESC/master

2f5417a

Merge branch 'master' into SSC



		def set_hmodified_cosmology(cosmo, deltah):
		Omega_c = cosmo["Omega_c"]

Power spectrum responses for SSC #1134

Are you sure you want to change the base?

Power spectrum responses for SSC #1134

Conversation

RyoTerasawa commented Nov 6, 2023

coveralls commented Dec 4, 2023 • edited Loading

Pull Request Test Coverage Report for Build 12132306089

Details

💛 - Coveralls

damonge commented Feb 21, 2024

RyoTerasawa commented Feb 21, 2024

carlosggarcia commented Apr 8, 2024

damonge commented Jun 18, 2024

RyoTerasawa commented Jun 22, 2024

carlosggarcia left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damonge commented Nov 13, 2024

RyoTerasawa commented Nov 15, 2024

RyoTerasawa commented Dec 3, 2024

damonge commented Dec 4, 2024

RyoTerasawa commented Dec 8, 2024

RyoTerasawa commented Dec 8, 2024

coveralls commented Dec 4, 2023 •

edited

Loading