Add enhancement flags to funcbench

[geekodour]

for #431

Behavior:

$ git checkout master
# both master and e208afcc957f056728b4e44af9103aedfc15a76f are evaluated
$ ./funcbench -n "lazydaytest" e208afcc957f056728b4e44af9103aedfc15a76f "BenchmarkIsolation*" ./tsdb

# cached data is fetched for both
$ ./funcbench -n "lazydaytest" e208afcc957f056728b4e44af9103aedfc15a76f "BenchmarkIsolation*" ./tsdb

# cached data is fetched only for master
$ ./funcbench -n "lazydaytest" 6f28c46f4c8812b124ce9bc5ebfd803974f7ec2d "BenchmarkIsolation*" ./tsdb

$ git checkout e208afcc957f056728b4e44af9103aedfc15a76f
# cached data is fetched for both
$ ./funcbench -n "lazydaytest" 6f28c46f4c8812b124ce9bc5ebfd803974f7ec2d "BenchmarkIsolation*" ./tsdb

$ pwd
/home/geekodour/OpenSource/prometheus/funcbench-results
$ tree
.
└── lazydaytest
    ├── QmVuY2htYXJrSXNvbGF0aW9uKg==-6f28c46f4c8812b124ce9bc5ebfd803974f7ec2d.out
    ├── QmVuY2htYXJrSXNvbGF0aW9uKg==-db13003721bf185f695ed63d6cff5c156042a4f3.out
    └── QmVuY2htYXJrSXNvbGF0aW9uKg==-e208afcc957f056728b4e44af9103aedfc15a76f.out

Changes:

• Update default value of result cache
• Add --user-test-name flag to help identify tests
• Add --perflock flag for perflock support
• Checkout git worktree of target in a temporary directory

Questions:

Then we do C, D, E and we usually compare with A and don't want to redo A all the time. Funbench however by design always do A and Target commit.

funcbench currently does not always do A and target commit, if they are cached, the cached versions will be used. (It does not however store the output of the comparison though). Did you mean something else, or I might be misunderstanding here?

To work on the workflow you mentioned, I can simply have the commits A,B,C,D,E and then be on one of branch (in our case A and just call funcbench for the other commits. won't this suffice?

git checkout A
for commit in B C D E
do
  ./funcbench -n "lazydaytest" $commit "BenchmarkIsolation*" ./tsdb
done

Notes

• It'd be nice to make funcbench go gettable, but it would need funcbench to have its own go.mod file no? (I am unfamilar with publishing go packages) :(
• funcbench-results should be gitignored.

cc: @bwplotka @krasi-georgiev

Signed-off-by: Hrishikesh Barman [email protected]

[krasi-georgiev]

I will let you and Bartek decide how to process with this PR

[bwplotka]

Looks like a nice addition. It's not used anywhere in Prometheus at the moment (it's not prombench)