Find Mode TrustVerify

[cderb]

Additional MIOPEN_FIND_MODE = 6 (TrustVerify)
This mode extends DynamicHybrid

Running with TrustVerify will first attempt to load tuning results from system resources
If no solution is returned tuning will be triggered
If a solution is retuned the user find db will be checked for the solution
If solution is from the user find db it will be used
If a solution is from the system (db or model), the solution will be evaluated and the new time will be compared to the time reported by the solution
If the evaluated / reported time is less than the tolerance threshold then the system result is added to the user db and the solution is returned
If the evaluated / reported time exceeds the tolerance, then tuning will be triggered

This find mode will ensure that all configurations are tuned for the deployment system. Solutions from system resources are verified once and tuned if markedly different from expectation. Results from user dbs are considered reliable and used without further verification.

[BrianHarrisonAMD]

Needs docs like the others.
The PR description is probably pretty close to what is needed!

[cderb]

added doc

[cderb]

The performance of find mode TrustVerify is between the performance of find mode DynamicHybrid and find enforce SearchDbUpdate.
TrustVerify will have the performance of SearchDbUpdate when

• There is no system db or model solution
• The solution returned from system is a degree slower than reported by the solution itself based on a 1 time evaluation
  TrustVerify will have the performance of DynamicHybrid when
• The solution returned from system resource performs as expected based on 1 time evaluation
• Solution is present in user db

First run with TrustVerify is generally slower than DynamicHybrid.
After the first run, the user find db will be fully populated and user perf db populated with regenerated entries. All following runs should exhibit ideal runtimes.

[cderb]

Found that setting find enforce disables find mode inputs. So removing these unnecessary lines.

[amd-jnovotny]

Looks good. I've made a couple of minor editorial suggestions.