Add par_reduce_inner functions

[bprather]

All credit for these to @pdmullen! I will fix bugs if they come up though.

At some point @pdmullen sent me some implementations of inner loops for performing reductions, i.e. par_for_inner but for par_reduce. This is useful if reducing to a vector, replacing values with an average, etc.

I am using the 1D variant in production happily, but haven't needed the 2D and 3D versions yet -- it seems difficult to believe they're incorrect though. I'm poking them upstream as a part of reducing the changeset between KHARMA's parthenon and develop, which is now quite small!

It might be useful to have a unit test for these, which I can add soon.

PR Checklist

• Code passes cpplint
• New features are documented.
• Adds a test for any bugs fixed. Adds tests for new features.
• Code is formatted
• Changes are summarized in CHANGELOG.md
• Change is breaking (API, behavior, ...)
  • Change is additionally added to CHANGELOG.md in the breaking section
  • PR is marked as breaking
  • Short summary API changes at the top of the PR (plus optionally with an automated update/fix script)
• CI has been triggered on Darwin for performance regression tests.
• Docs build
• (@lanl.gov employees) Update copyright on changed files

[lroberts36]

Does #1142 provide this functionality @acreyes?

[acreyes]

Does #1142 provide this functionality @acreyes?

It doesn't at the moment, but it should be straightforward to add

[lroberts36]

Approved once comment is at least discussed.

[lroberts36]

Should these be

par_reduce_inner(InnerLoopPatternTTR, team_mbr_t team_member, const int kl, const int ku, const int jl,
                 const int ju, const int il, const int iu, const Function &function,
                 T reduction) {

[bprather]

I'm not sure. I didn't really think about it when dropping these in, and I don't think I fully understand the way loop pattern resolution is done anyway...
The way I understand it, since I use TVR_INNER_LOOP for ThreadVectorRange, these wouldn't resolve under that, right? So, we'd need a TVR and SIMD pattern for these too, if we wanted to make them selectable. Since these should be used pretty rarely, I don't know if the performance benefit of allowing customization is worth three implementations we're just going to stamp over with all the better general stuff you guys are writing for the next release.

[lroberts36]

I could see it both ways. With my suggestion, you would have to call parthenon::par_reduce_inner(inner_loop_pattern_ttr_tag, ...) instead of parthenon::par_reduce_inner(...), which would be more explicit that you aren't necessarily using the default inner loop pattern and conform to how the rest of the par_*_inner functions look. But this doesn't change anything about how this behaves.

[bprather]

Ah, right, you can explicitly tag loops! It seems better to be explicit, then, let me give it a go today.

[pgrete]

As discussed during the sync last week, we'll merge this now (despite also being added in #1142 as the latter requires some more detailed discussion and this functionality should make it into the immediately upcoming release).