Wrap gradient_free_optimizers (local)

[gauravmanmode]

PR Description

This PR adds optimizers from gradient_free_optimizers

The following optimizers are now available

• Hill Climbing
• Stochastic Hill Climbing
• Simulated Annealing
• Repulsing Hill Climbing
• Downhill Simplex Optimization
• Simulated Annealing
• Powell's Method

Other Changes

• Add a new field experimental to AlgoInfo for optimizers that need to skip tests.
• Add a new example problem SphereExampleInternalOptimizationProblemWithConverter with converter in internal_optimization_problem.py.
• Refactor test_many_algorithms.py, add a PRECISION_LOOKUP dict for algorithm specific precision.

Helper Functions

[codecov]

Codecov Report

❌ Patch coverage is 99.49239% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...agic/optimization/internal_optimization_problem.py	50.00%	1 Missing ⚠️

Files with missing lines	Coverage Δ
src/optimagic/algorithms.py	88.51% <100.00%> (+0.33%)	⬆️
src/optimagic/mark.py	90.00% <ø> (ø)
src/optimagic/optimization/algorithm.py	94.97% <100.00%> (+0.02%)	⬆️
src/optimagic/optimizers/gfo_optimizers.py	100.00% <100.00%> (ø)
...agic/optimization/internal_optimization_problem.py	92.85% <50.00%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[gauravmanmode]

Hi @janosg
There are a few points

1. I am failing tests in test_many_algorithms for stochastic algorithms as in the case of all gfo optimizers needs_bounds = True but the tests do not have them.
2. Writing test for helper functions is a bit difficult as I have exposed the converter. Should I pass arrays only after converting?

[janosg]

Hi @gauravmanmode

1. Is this because we currently use the is_global flag to decide which tests get called wtih bounds? If so, we can simply use the new flags in test_many_algorithms (either adding a new test case for non global optimizers that require bounds or restructuring what weh have).
2. I don't understand what the problem is and what you mean by "Should I pass arrays only after converting?". Is the problem that SphereExampleInternalOptimizationProblem does not have a converter?

[gauravmanmode]

Hi @janosg,
Could you please review the changes?

Changes

add a new example problem with converter in internal_optimization_problem

Added functions and converter dealing with dict input.

refactor test_many_algorithms.

( this is minimal and is just for passing tests on this one)

1. For now, the new tests in test_many_algorithms_new cover all of the tests existing in the original test_many_algorithms_new .
2. The only failing test is scipy_trust_constr with binding bounds which fails barely . For the purpose of passing this one test, I have set the accuracy of local algorithms to 3 for now.
3. Why test_with _binding_bounds test does not test with finite bounds ?
   Should this be in a separate PR?

gfo

1. Many algos in GFO do not pass tests with their default values. How should I go about that?
2. Some algorithms in GFO particularly population based ones do not advance ( n_iterations_performed is low ) if convergence_iter_noimprove is set to something low like < 100. Not sure but some optimizers jump randomly to new positions to explore the search space might be the reason.

[janosg]

Should we set this to None or a really high value instead? Is there another convercence criterion we could set to a non-None value instead? We don't want all optimizers just to run until max_iter but we also don't want pre-mature stopping of course.

[gauravmanmode]

I am a bit confused about this.
Most of the time convergence_iter_noimprove behaves as stopping_maxiter. The other convergence criteria dont seem to be respected. Even after a > 100000 iterations the algorithm does not converge to a good solution. I might be missing something but hope to get a solution here.

[janosg]

Thanks for creating the issue over there.

[gauravmanmode]

Since convergence_iter_noimprove is set to a large value, it will look for a change in a large no. of iterartions and convergence_ftol_abs and convergence_ftol_rel will not have effect.

[gauravmanmode]

Hi @janosg, The tests are passing, could you review this as well?

[janosg]

@gauravmanmode can you fix the merge conflicts?

[gauravmanmode]

@janosg , I've fixed the merge conflicts.

[janosg]

Very nice. Just minor comments!

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[janosg]

Thanks for the changes and sorry for the delay. This is ready to merge from my side. Maybe have a quick look again if it is ready from your side and then click on merge.

[gauravmanmode]

sure. no worries. i need to look once again and will let you know.