Feature/openqxd

[chaoos]

• Adds interface openQxD <--> QUDA
• Adds capability for C* boundaries in 1,2 or 3 spatial directions
• Fixed CPU reordering
• Adds openQCD gamma matrix basis (and basis transformations)
• Adds order class for spinors
• Adds order class for clover field (U(1) clover field is transfered from openQxD)
• Adds order class for gauge field
• Changed all the comments in quda.h to C89 style comments
• Rebased on your current develop branch

[chaoos]

I was able to commit to the wiki, thanks for giving me access!
The CI/CD also seems to be fixed.

[havogt]

cscs-ci run

[maddyscientist]

I've had a review of the latest code. This is a great contribution, but a few things left outstanding to be resolved. I have left some comments, but beyond those:

• The C89 changes are of concern, since we will be embracing a C99 interface, e.g., use of bool.
• The new openQCD interface is rather opaque to me. Can this be better documented?
• Thanks for the wiki additions. Is there a test input file or equivalent to allow us to run a test with this?

[maddyscientist]

This addition here looks bogus: the Wilson operator is not Hermitian. Is this addition intentional?

[chaoos]

This is in the class DiracG5M (i.e. gamma5 times the Dirac operator), which is Hermitian. I had to change this, since the eigensolver didn't accept a non-Hermitian operator else. So this should be correct (only for gamma5*D and D*gamma5 though).

[chaoos]

The C89 changes are of concern, since we will be embracing a C99 interface, e.g., use of bool.

Dear Kate,

• The C99 thing is ok from our side. Should I just revert all changes I made in quda.h regarding this?
• We are currently in the phase of documenting, so give me some days for this. Do you want more comments in the code (I think every function/method has a javadoc-decoration), or conceptual things in the wiki?
• Regarding testing of openqxd functionality; we have some testing on the side of openqxd. If you want to test this in this repo, then you need to clone and compile the openqxd repo and run the tests from there. We are working on making this repo public, but there is still some politics going on: that's not in my power.

[maddyscientist]

@chaoos apologies for the slow reply at my end. GTC and other travel got in the way.....

If you can revert the C99 -> C89 changes that would be great. It would also narrow the scope of the PR which would also be helpful.

Regarding documentation, the basic rule is that all new functions should have doxygen markup next to their declarations, together with overall conceptual documentation in the wiki. If you can write up the documentation to include cloning the openq*d and compiling the code to work with QUDA that would be great. Understood that the repo isn't public yet, and that's fine for now. Does that sound ok to you?

[chaoos]

Hello

I'm sorry for the long wait! Finally I was able to finish this up.

I've reverted all the C89 changes, appled clang-format again, implemented CI/CD testing for all this on our side, and fixed the issues we still had.

So nothing really stops this anymore from being merged. Please review again. Let me know if there is something I can do to settle this.

Thanks and best wishes,
Roman

PS: I've created a spack package for QUDA (pull-request awaits approval, see spack/spack#48939). This became very useful for our CSCS CI/CD workflow. Maybe it's also useful for others, and can be put in the wiki or so?