Use real GPU snapshot for nvhpc regression test

[peverwhee]

Issue Type

Code Clean-up

Issue Description

As referenced in #451, the RRTMGP GPU test is running against a CPU snapshot because:

1. I (Courtney) could not get a GPU snapshot to run with no answer changes
   Answer differences for the first timestep using the ne30 GPU snapshot (possible configuration issues present):

********** Physics Check Data Results **********

 TIMESTEP:             1

 Variable                                          # Diffs  Max Diff   Max Diff loc (rank, col, lev)
 --------                                          -------  --------   -----------------------------
 air_temperature                                   2818800  0.83E-02   (43,131,58)
diffuse_solar_radiative_flux_at_surface_ge_700nm_to_coupler
                                                     24300  0.25E+03   (44,66)
diffuse_solar_radiative_flux_at_surface_lt_700nm_to_coupler
                                                     24300  0.32E+03   (47,37)
direct_solar_radiative_flux_at_surface_ge_700nm_to_coupler
                                                     24288  0.54E+03   (42,339)
direct_solar_radiative_flux_at_surface_lt_700nm_to_coupler
                                                     24288  0.51E+03   (47,333)
 geopotential_height_wrt_surface                   2818800  0.83E-02   (43,131,58)
 geopotential_height_wrt_surface_at_interface      2818800  0.83E-02   (43,131,58)
 longwave_net_outgoing_flux_at_model_top             48600  0.35E+03   (23,149)
 longwave_net_upward_flux_at_surface                 48600  0.25E+03   (23,149)
 shortwave_net_outgoing_flux_at_model_top            24300  0.13E+04   (47,312)
 shortwave_net_upward_flux_at_surface                24300  0.11E+04   (46,181)
 tendency_of_air_temperature_due_to_model_physics  2818800  0.78E+02   (24,53,58)
tendency_of_dry_air_enthalpy_at_constant_pressure_due_to_longwave_radiation_adjusted_by_air_pressure_thickness
                                                   2818800  0.11E+04   (68,342,34)
tendency_of_dry_air_enthalpy_at_constant_pressure_due_to_shortwave_radiation_adjusted_by_air_pressure_thickness
                                                   1409400  0.70E+03   (43,139,27)

 ********** End Physics Check Data Results **********

ALSO the second timestep failed with:

cam_ccpp_physics_run: gas_optics(): array play has values outside range

2. The only GPU snapshot I was able to make was with ne30 and was thus huge and I felt like it took up too much space on glade without actually working.
3. I'd expect answer differences with the CPU snapshot, but not the massive answer differences that we're seeing:

 ********** Physics Check Data Results **********

 TIMESTEP:             1

 Variable                                          # Diffs  Max Diff   Max Diff loc (rank, col, lev)
 --------                                          -------  --------   -----------------------------
 air_temperature                                     28188  0.59E-02   (37,1,58)
diffuse_solar_radiative_flux_at_surface_ge_700nm_to_coupler
                                                       242  0.20E+03   (42,2)
diffuse_solar_radiative_flux_at_surface_lt_700nm_to_coupler
                                                       242  0.26E+03   (42,2)
direct_solar_radiative_flux_at_surface_ge_700nm_to_coupler
                                                       242  0.49E+03   (50,2)
direct_solar_radiative_flux_at_surface_lt_700nm_to_coupler
                                                       242  0.50E+03   (50,2)
 geopotential_height_wrt_surface                     28188  0.59E-02   (37,1,58)
 geopotential_height_wrt_surface_at_interface        28188  0.59E-02   (37,1,58)
 longwave_net_outgoing_flux_at_model_top               486  0.21E+06   (30,2)
 longwave_net_upward_flux_at_surface                   486  0.82E+07   (30,2)
 shortwave_net_outgoing_flux_at_model_top              242  0.13E+04   (48,3)
 shortwave_net_upward_flux_at_surface                  242  0.10E+04   (50,2)
 tendency_of_air_temperature_due_to_model_physics    28188  0.15E+02   (57,2,1)
tendency_of_dry_air_enthalpy_at_constant_pressure_due_to_longwave_radiation_adjusted_by_air_pressure_thickness
                                                     28188  0.54E+07   (25,3,39)
tendency_of_dry_air_enthalpy_at_constant_pressure_due_to_shortwave_radiation_adjusted_by_air_pressure_thickness
                                                     14036  0.48E+03   (42,1,33)

TLDR; something is wrong with the way the GPU test is running for it to end up with these massive differences (or errors from RRTMGP's internal checks)

This fix may change answers for NVHPC RRTMGP.

Will this change answers?

Yes

Will you be implementing this yourself?

Any Software Engineer can do this