Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/openqxd #1414

Open
wants to merge 185 commits into
base: develop
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
185 commits
Select commit Hold shift + click to select a range
ed966db
initial structure (completely untested and still a lot of things WIP)
mathiaswagner Sep 26, 2022
71c1897
compilation fixes and plaquette dummy
mathiaswagner Sep 27, 2022
efc24b8
Updated rank formula
fernandezdlg Sep 27, 2022
61b8a02
Typo corrected
fernandezdlg Sep 27, 2022
a40b032
Typo corrected
fernandezdlg Sep 27, 2022
0a69472
fix rankFromCoords to consider BLK order
nyholme Sep 28, 2022
c29aa66
some fixes for openQCD testing
mathiaswagner Sep 28, 2022
d3d4022
Still not working,
fernandezdlg Jan 27, 2023
9e1b7ad
Now working, todo: add ipt directly in quda
fernandezdlg Jan 30, 2023
aec93e3
TODO: comments & planning
fernandezdlg Feb 3, 2023
1f0e573
hardwire quda rankfromcoords
fernandezdlg Feb 20, 2023
6271fac
hardwire test 2
fernandezdlg Feb 25, 2023
ef1e5af
Merge branch 'feature/openqxd' of github.com:fernandezdlg/quda into f…
fernandezdlg Feb 25, 2023
8550ed0
ignore build folder
fernandezdlg Mar 3, 2023
c9d0f6c
rankFromCoords for OpenQxD gauge fields working.
fernandezdlg Mar 8, 2023
301378b
OpenQCD spinor interface from TIFR (need adjusting
fernandezdlg Mar 14, 2023
240d614
commented out openqcd spinors sections for compile
fernandezdlg Mar 20, 2023
6202ac4
spinors feature OK (still needs correct indexing)
fernandezdlg Mar 20, 2023
cf3a04a
openqcd dslash correction + load color spinor
fernandezdlg Mar 21, 2023
16b7df0
load-save in spinors+gauge + format
fernandezdlg Mar 22, 2023
12f22f4
small correction on load fct
fernandezdlg Mar 23, 2023
d535512
ordering done, lexicographical io w openqxd
fernandezdlg Mar 23, 2023
f538e53
added openqcd dirac order
fernandezdlg Mar 23, 2023
568d1cd
dslash quda := NO dslash operator applied
fernandezdlg Mar 23, 2023
dc8c99f
strange bad indexing behaviour (check3 log)
fernandezdlg Mar 23, 2023
ecc2177
small corrections + comments
fernandezdlg Mar 24, 2023
8b1830d
cleanup and test oddeven
fernandezdlg Mar 24, 2023
f738510
latest test
fernandezdlg Mar 24, 2023
ad8f689
formatting
fernandezdlg Mar 25, 2023
189896e
corrected over-include enum + cmmnts
fernandezdlg Mar 26, 2023
37e6023
return to previous formatting
fernandezdlg Mar 26, 2023
b0076de
small corrs + formatting
fernandezdlg Mar 26, 2023
2525a5b
gauge save load
fernandezdlg Mar 26, 2023
9dce279
quda openqcd gauge loadsave, load, and save
fernandezdlg Mar 26, 2023
4ff5cce
removed comments
fernandezdlg Mar 27, 2023
b2d7446
rearrangement
fernandezdlg Mar 27, 2023
fdf641c
plaquette only function
fernandezdlg Mar 27, 2023
ca66195
small correction
fernandezdlg Mar 27, 2023
bc95079
gauge working, color using spacespinorcolor totest
fernandezdlg Apr 2, 2023
e1bed36
final
fernandezdlg Apr 13, 2023
5e60bc5
stated to clean up mess a little
chaoos Aug 14, 2023
402cab6
cleaned up
chaoos Aug 15, 2023
ed234bd
refactored plaquette
chaoos Aug 21, 2023
bfe766f
Only have generic lexicographical ordering in OpenQCDOrder
chaoos Aug 23, 2023
b837622
norms on both sides coincide
chaoos Aug 24, 2023
23b1e0b
all reordering of spinor fields done in quda now
chaoos Aug 25, 2023
b577392
added functions gamma{0,2,3}() that don't work
chaoos Aug 25, 2023
26918e6
Add QUDA_OPENQCD_GAMMA_BASIS
Aug 31, 2023
d3fdc43
fixed openQCD_qudaGamma
chaoos Sep 11, 2023
fe01653
removed implicit fallthrough in switch case of gammai()
chaoos Sep 11, 2023
34d388d
global minus for openqcd gamma matrix transformation
chaoos Sep 11, 2023
b6157c5
dirac operator seems to work now
chaoos Sep 12, 2023
fdd1b62
clover term works
chaoos Sep 13, 2023
be90f9d
GCR inverter works
chaoos Sep 14, 2023
736d2cd
cleaned up a little
chaoos Sep 14, 2023
3f131b0
Add fvtx for profiler
donion74 Sep 19, 2023
92a8b50
Optimize malloc
donion74 Sep 19, 2023
088678e
Add parameters for multigrid (incomplete)
Sep 20, 2023
271f828
Undo minus sign in back transformation
Sep 21, 2023
00136bf
Add global minus in Dirac
donion74 Sep 21, 2023
076cf0e
Add back and forth test
donion74 Sep 21, 2023
d9c4fff
Remove *= -1
donion74 Sep 21, 2023
a37c286
Revert "Undo minus sign in back transformation"
Sep 21, 2023
7c89362
Fix back_and_forth test
donion74 Sep 21, 2023
5c21522
Merge branch 'feature/openqxd-thesis-release' of https://github.com/c…
Sep 21, 2023
30ae3bd
Fix mistake in transformation and remove Romans' minus sign in transf…
donion74 Sep 21, 2023
7985910
Fix mistake in Udagger
donion74 Sep 21, 2023
5f57224
Modify gamma5
Sep 21, 2023
fd9975e
Changes for MG support.
timrupertharris Sep 22, 2023
78465cf
Merge branch 'feature/openqxd-thesis-release' of https://github.com/c…
timrupertharris Sep 22, 2023
9f52f30
Add debugging information
Sep 22, 2023
61e4985
Polish code
donion74 Sep 25, 2023
ef78041
Change back and forth test
donion74 Sep 25, 2023
18518f5
Merge branch 'multgrid-solver' into feature/openqxd-thesis-release
donion74 Sep 25, 2023
29da468
Modify back and forth test
donion74 Sep 25, 2023
8ababe5
Modify printfQuda
donion74 Sep 25, 2023
53d9a8c
Merge branch 'multgrid-solver' into feature/openqxd-thesis-release
donion74 Sep 25, 2023
4a9ba76
Return true_res and disable half precision in multigrid
donion74 Sep 25, 2023
1cac7c0
Remove qudaInvert
donion74 Sep 25, 2023
96bb7ab
Merge branch 'feature/openqxd-thesis-release' of https://github.com/c…
Sep 25, 2023
ed254d5
Add comment about Precision for multigrid
donion74 Sep 25, 2023
febf4fb
Moved init of cuda_prec_sloppy into openQCD_qudaMultigrid.
timrupertharris Sep 26, 2023
83ce801
added clover field load function to load cloer field from openQCD
chaoos Sep 27, 2023
0c9bcf0
we need csw and coeff to determine kappa in clover_field_order.h Open…
chaoos Sep 27, 2023
2a91f8b
Merge remote-tracking branch 'refs/remotes/origin/feature/openqxd-the…
chaoos Sep 27, 2023
3a0012c
first attempt for no loads
chaoos Sep 29, 2023
d929534
Merge branch 'feature/openqxd-thesis-release' of https://github.com/c…
Oct 4, 2023
14baf0f
Debug cstar error
donion74 Sep 22, 2023
1f7fb9a
Debug cstar error
donion74 Sep 22, 2023
ac3d0d0
Use layout.cstar for setting cstar
donion74 Sep 25, 2023
e7eaaa9
Fix }
donion74 Sep 25, 2023
abdcbcc
Use global_ipr for ranksFromCoords
Oct 4, 2023
732d760
Print debug
Sep 29, 2023
25ed056
Remove debug info
Oct 4, 2023
b69b67a
Fix bracket, clean up
Oct 4, 2023
748693a
Comment debugging
Oct 4, 2023
f663ec2
Remove comment
Oct 4, 2023
129c578
Restore info about communicator
Oct 6, 2023
512e3e9
Rename variables
Oct 6, 2023
4d9b28a
Remove debug statements
Oct 6, 2023
236f869
added relevant parameters to Dirac struct
chaoos Oct 6, 2023
cdd360e
Use BUILD_INTERFACE_OPENQCD to set cstar
Oct 9, 2023
025b0e3
Merge branch 'feature/openqxd-thesis-release' of https://github.com/c…
Oct 9, 2023
dd0e4a5
Remove abs(dims)
Oct 9, 2023
86134fb
towards a general solver interface
chaoos Oct 17, 2023
3ad9b06
works with QCD+QED (clover term is transfered always)
chaoos Oct 18, 2023
36251b5
finally transfer clover only when needed (ie QCD+QED)
chaoos Oct 18, 2023
a0d8622
added remaining settings
chaoos Oct 19, 2023
ec3e911
solved a bug that prevented reordering on CPU to work
chaoos Nov 1, 2023
86ff1d5
in case of openQCD gauge fields, allocate and transfer 4*VOLUME +
chaoos Nov 1, 2023
b8bc138
reordering of gauge field done purely within QUDA (only load, not store)
chaoos Nov 1, 2023
1ce6b19
changed gauge loading and added eigensolver interface (not working yet)
chaoos Nov 1, 2023
9e81a4c
solved the mystical local lattice direction = 6 bug
chaoos Nov 1, 2023
9778011
changed comments to C89 style (such that we can link C89 programs
chaoos Nov 2, 2023
8f4c7dc
setup verbosity
chaoos Nov 3, 2023
608dec2
Merge pull request #1 from lattice/develop
chaoos Nov 6, 2023
1c85393
fixed eigensolverDestroy
chaoos Nov 6, 2023
3af70d1
Merge remote-tracking branch 'origin/develop' into feature/openqxd
chaoos Nov 6, 2023
0842ed0
cleaned up tabs/spaces, comment, unnecessary additions
chaoos Nov 6, 2023
ef57751
fixed gitignore
chaoos Nov 6, 2023
0020993
removed comment
chaoos Nov 6, 2023
e622ce8
cleaned up openqcd interface
chaoos Nov 6, 2023
7085cc5
outsourced all openqcd index functions
chaoos Nov 6, 2023
ad6787a
fixed reordering on GPU
chaoos Nov 6, 2023
df1bd16
macro war to enable "-std=c89 -pedantic -Werror" in openqxds checks
chaoos Nov 7, 2023
93c3f6a
added comment for loadGaugeQuda
chaoos Nov 8, 2023
8301dbf
added MatQuda profiling
chaoos Nov 8, 2023
200295d
removed check functions and unnecessary including
chaoos Nov 14, 2023
59578cc
added requested comments
chaoos Nov 14, 2023
bab885c
check for nullptr in comm_create_topology
chaoos Dec 5, 2023
dd29926
ipt and iup function exposed, parsing infile section "QUDA"
chaoos Dec 5, 2023
43a1b4f
the Wilson and Wilson-clover Dirac operator are gamma5 Hermtitian
chaoos Dec 6, 2023
23fefb1
added functionality to compute both (left and right) singular vectors…
chaoos Dec 6, 2023
848e6d9
added comments for eigensolver interface
chaoos Dec 6, 2023
02b8481
some convenience mappings for the enums
chaoos Dec 8, 2023
ae170fe
in TRLM: n_kr must be >= n_conv+12, else iter_keep may become negatice
chaoos Dec 8, 2023
d56921e
verbose error message if section is not a quda section
chaoos Dec 12, 2023
886c90f
more verbose error msg when solver section is not a QUDA one
chaoos Dec 15, 2023
e6ce728
added function comm_dim_cstar to query if a dimension is C*
chaoos Dec 15, 2023
ba9ace1
solved P2P communication bug when C* boundaries are active
chaoos Dec 15, 2023
1b1a55c
added packed formats
chaoos Feb 5, 2024
c2c8ef6
Merge branch 'feature/openqxd' of https://github.com/chaoos/quda into…
chaoos Feb 5, 2024
dd92262
dynamic t_boundary
chaoos Feb 7, 2024
20e59ba
enable RECONSTRUCT_8/9/12/13/NO for openqxd interface
chaoos Feb 7, 2024
d40d6e5
fixed eigensolver, when cuda_prec_eigensolver is not set in the infile
chaoos Feb 13, 2024
6c7bb9d
only transfer/recalc gauge/clover field according to openqxd's flag DB
chaoos Feb 14, 2024
005aed9
added void pointer to QudaInvertParam struct for arbitrary additinal
chaoos Feb 19, 2024
ad1ab82
changed solver interface to only deal with solver section IDs
chaoos Feb 19, 2024
c1fec52
refactored interface
chaoos Feb 21, 2024
d356fa0
removed ptr in getHandle
chaoos Feb 22, 2024
a80dd55
removed ptr in getHandle (2)
chaoos Feb 23, 2024
466d702
fixed clang complaining "error: suggest braces around initialization of
chaoos Feb 23, 2024
c79cb4a
in openQCD sometimes we test with gauge field all set to unity, then the
chaoos Feb 23, 2024
6edc863
remove trailing whitespaces
chaoos Feb 23, 2024
568dd37
applied clang-format as described in https://github.com/lattice/quda/…
chaoos Feb 23, 2024
3150761
eigensolver iterface rewritten to use identifiers too (as inverter
chaoos Feb 28, 2024
0752a23
fixed compiler error in CI/CD
chaoos Feb 28, 2024
ca54f7e
clang-format
chaoos Feb 28, 2024
9d9ace7
fixed eigensolver deallocation
chaoos Mar 12, 2024
f63a507
added openQCD_qudaDw_NoLoads (wrapper around Dirac operator that does
chaoos Jul 3, 2024
bddb46e
Merge remote-tracking branch 'upstream/develop' into feature/openqxd
chaoos Jan 14, 2025
7d06520
Merge remote-tracking branch 'upstream/develop' into feature/openqxd
chaoos Jan 14, 2025
f77a907
remove minimize_memory
chaoos Jan 16, 2025
fe998e4
solves https://github.com/lattice/quda/issues/1532
chaoos Jan 16, 2025
db2fc52
remove C89 comments
chaoos Jan 16, 2025
0c2ea4e
revert all the c89 changes
chaoos Jan 16, 2025
21fd2bc
remove default geo_block_size of 4x4x4x4 for level 0
Jan 21, 2025
63feebe
added back newline at EOF
chaoos Jan 22, 2025
49f3fd3
Merge remote-tracking branch 'upstream/develop' into feature/openqxd
chaoos Jan 23, 2025
4f91376
fix when compile with QUDA_DIRAC_DEFAULT_OFF=OFF
chaoos Feb 6, 2025
1f538ec
fix problem, when qhat changes when U(3) gauge group -> need transfer…
chaoos Feb 12, 2025
e7a96b5
apply clang-format
chaoos Feb 14, 2025
91819f1
reverted changes in interface_quda.cpp
chaoos Feb 14, 2025
66c60dc
added asych solver call
chaoos Feb 28, 2025
e177aec
moved structs, added async interface functions
chaoos Mar 3, 2025
d712f5b
fixed async with MPICH
chaoos Mar 3, 2025
2f26208
nullptr to MPI_COMM_NULL
chaoos Mar 3, 2025
1fca8a3
fixed async solver call for cray MPI
chaoos Mar 4, 2025
4ea946f
first implementation of two grids
chaoos Mar 13, 2025
444b9f8
added bcasts for status and residual in invert
chaoos Mar 14, 2025
8eb4461
remove manual reordering from quda to openqcd
chaoos Mar 20, 2025
8949c22
cleaned up mess
chaoos Mar 20, 2025
ce3a31d
WITH_COMM() and in_comm()
chaoos Mar 20, 2025
06adc2d
Merge pull request #5 from chaoos/feature/openqxd-two-grids
chaoos Mar 20, 2025
632e92e
connect to multi RHS interface functions
chaoos Mar 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixed openQCD_qudaGamma
chaoos committed Sep 11, 2023
commit d3fdc43eeb13dccc5a31e64dbd8a3a63852518cb
28 changes: 20 additions & 8 deletions include/kernels/copy_color_spinor.cuh
Original file line number Diff line number Diff line change
@@ -137,8 +137,14 @@ namespace quda {
__device__ __host__ inline void operator()(complex<FloatOut> out[Ns*Nc], const complex<FloatIn> in[Ns*Nc]) const {
int s1[4] = {0, 1, 0, 1};
int s2[4] = {2, 3, 2, 3};
FloatOut K1[4] = {static_cast<FloatOut>(kP), static_cast<FloatOut>(kP), static_cast<FloatOut>(-kP), static_cast<FloatOut>(-kP)};
FloatOut K2[4] = {static_cast<FloatOut>(kP), static_cast<FloatOut>(kP), static_cast<FloatOut>(kP), static_cast<FloatOut>(kP)};
FloatOut K1[4] = {static_cast<FloatOut>(kP),
static_cast<FloatOut>(kP),
static_cast<FloatOut>(-kP),
static_cast<FloatOut>(-kP)};
FloatOut K2[4] = {static_cast<FloatOut>(kP),
static_cast<FloatOut>(kP),
static_cast<FloatOut>(kP),
static_cast<FloatOut>(kP)};
/* U = [1 0 1 0]
[0 1 0 1]
[-1 0 1 0]
@@ -158,12 +164,18 @@ namespace quda {
__device__ __host__ inline void operator()(complex<FloatOut> out[Ns*Nc], const complex<FloatIn> in[Ns*Nc]) const {
int s1[4] = {0, 1, 0, 1};
int s2[4] = {2, 3, 2, 3};
FloatOut K1[4] = {static_cast<FloatOut>(kU), static_cast<FloatOut>(kU), static_cast<FloatOut>(kU), static_cast<FloatOut>(kU)};
FloatOut K2[4] = {static_cast<FloatOut>(-kU), static_cast<FloatOut>(-kU), static_cast<FloatOut>(kU), static_cast<FloatOut>(kU)};
/* U = [1 0 -1 0]
[0 1 0 -1]
[1 0 1 0]
[0 1 0 1] / sqrt(2) */
FloatOut K1[4] = {static_cast<FloatOut>(-kU),
static_cast<FloatOut>(-kU),
static_cast<FloatOut>(-kU),
static_cast<FloatOut>(-kU)};
FloatOut K2[4] = {static_cast<FloatOut>(kU),
static_cast<FloatOut>(kU),
static_cast<FloatOut>(-kU),
static_cast<FloatOut>(-kU)};
/* U = [-1 0 1 0]
[ 0 -1 0 1]
[-1 0 -1 0]
[ 0 -1 0 -1] / sqrt(2) */
for (int s=0; s<Ns; s++) {
for (int c=0; c<Nc; c++) {
out[s*Nc+c] = K1[s]*static_cast<complex<FloatOut> >(in[s1[s]*Nc+c]) + K2[s]*static_cast<complex<FloatOut> >(in[s2[s]*Nc+c]);
38 changes: 23 additions & 15 deletions lib/openqcd_interface.cpp
Original file line number Diff line number Diff line change
@@ -461,6 +461,13 @@ void openQCD_qudaSetDslashOptions(double kappa, double mu)
}


/**
* @brief Calculates the norm of a spinor.
*
* @param[in] h_in input spinor of type spinor_dble[NSPIN]
*
* @return norm
*/
double openQCD_qudaNorm(void *h_in)
{
lat_dim_t X;
@@ -486,15 +493,16 @@ double openQCD_qudaNorm(void *h_in)


/**
* @brief Applies Dirac matrix to spinor.
*
* openQCD_out = gamma[dir] * openQCD_in
*
* @param dir Dirac index, 0 <= dir <= 5
* @param openQCD_in of type spinor_dble[NSPIN]
* @param openQCD_out of type spinor_dble[NSPIN]
* @brief Applies Dirac matrix to spinor.
*
* openQCD_out = gamma[dir] * openQCD_in
*
* @param[in] dir Dirac index, 0 <= dir <= 5, notice that dir is in
* openQCD convention, ie. (0: t, 1: x, 2: y, 3: z, 4: 5, 5: 5)
* @param[in] openQCD_in of type spinor_dble[NSPIN]
* @param[out] openQCD_out of type spinor_dble[NSPIN]
*/
void openQCD_qudaGamma(int dir, void *openQCD_in, void *openQCD_out)
void openQCD_qudaGamma(const int dir, void *openQCD_in, void *openQCD_out)
{
lat_dim_t X;

@@ -528,14 +536,14 @@ void openQCD_qudaGamma(int dir, void *openQCD_in, void *openQCD_out)
// gamma5 runs within QUDA using QUDA fields
if (dir == 5 || dir == 4) {
gamma5(out, in);
} else if (dir == 0) {
} else if (dir == 0) { // t direction
gamma3(out, in);
} else if (dir == 1) { // x direction
gamma0(out, in);
} else if (dir == 1) {
} else if (dir == 2) { // y direction
gamma1(out, in);
} else if (dir == 2) {
} else if (dir == 3) { // z direction
gamma2(out, in);
} else if (dir == 3) {
gamma3(out, in);
} else {
errorQuda("Unknown gamma: %d\n", dir);
}
@@ -713,7 +721,7 @@ void openQCD_qudaInvert(int external_precision, int quda_precision, double mass,
// void setInvertParam(QudaInvertParam &invertParam, openQCD_QudaInvertArgs_t &inv_args, int external_precision,
// int quda_precision, double kappa, double reliable_delta);

void setInvertParam(QudaInvertParam &invertParam, openQCD_QudaInvertArgs_t &inv_args, int external_precision,
/*void setInvertParam(QudaInvertParam &invertParam, openQCD_QudaInvertArgs_t &inv_args, int external_precision,
int quda_precision, double kappa, double reliable_delta)
{

@@ -751,6 +759,6 @@ void setInvertParam(QudaInvertParam &invertParam, openQCD_QudaInvertArgs_t &inv_
invertParam.clover_order = QUDA_PACKED_CLOVER_ORDER;

invertParam.compute_action = 0;
}
}*/

// TODO: OpenQCDMultigridPack functions a la MILC (cf. milc_interface.cpp)