Loose/Flexible GMRES

[f0uriest]

I'm interesting in adding some new GMRES variants to Lineax that allow reusing some of the krylov vectors between restarts as well as variable and potentially nonlinear preconditioners. Would it make more sense to add them as options to the existing GMRES, or to make new solvers for each?

[patrick-kidger]

I think probably easiest to do these as options, as long as the branching isn't too complicated!
(The main bottleneck is my ability to review/maintain whatever is added.)
FWIW whilst we're on this topic, I haven't forgotten about #86, which I'd be happy to return to. (This fell off my plate entirely.)

[f0uriest]

Cool, I'll take a look at how difficult it would be. Looking at the scipy version here I think the main thing would be adding an additional arg to GMRES._gmres_compute to allow passing in additional vectors to orthogonalize against. Scipy also has a separate arg for cs used in GCROT which might also be useful in the future? Other than that it looks like mostly plumbing.

In the meantime, I've updated #86

[johannahaffner]

The implementation of preconditioning in lx.CG might also be a useful reference here.