hsbench: add unordered listing mode

[mattbenjamin]

Add mode 'u' to do unordered bucket listing, using the
ListObjectsRequest mechanism.

Signed-off-by: Matt Benjamin [email protected]

[markhpc]

Tried this against my current test cluster but it didn't appear to be working vs the regular list:

2019/08/29 08:55:09 Running Loop 0 OBJECT PUT TEST
2019/08/29 08:55:10 Loop: 0, Int: 0, Dur(s): 1.0, Mode: PUT, Ops: 6439, MB/s: 25.15, IO/s: 6439, Lat(ms): [ min: 2.9, avg: 19.5, 99%: 50.2, max: 74.0 ], Slowdowns: 0
2019/08/29 08:55:11 Loop: 0, Int: 1, Dur(s): 1.0, Mode: PUT, Ops: 6384, MB/s: 24.94, IO/s: 6384, Lat(ms): [ min: 2.3, avg: 20.0, 99%: 61.8, max: 88.8 ], Slowdowns: 0
2019/08/29 08:55:12 Loop: 0, Int: 2, Dur(s): 1.0, Mode: PUT, Ops: 6888, MB/s: 26.91, IO/s: 6888, Lat(ms): [ min: 1.4, avg: 18.6, 99%: 52.8, max: 73.4 ], Slowdowns: 0
2019/08/29 08:55:13 Loop: 0, Int: 3, Dur(s): 1.0, Mode: PUT, Ops: 6716, MB/s: 26.23, IO/s: 6716, Lat(ms): [ min: 2.2, avg: 19.0, 99%: 54.1, max: 82.9 ], Slowdowns: 0
2019/08/29 08:55:14 Loop: 0, Int: 4, Dur(s): 1.0, Mode: PUT, Ops: 6646, MB/s: 25.96, IO/s: 6646, Lat(ms): [ min: 2.2, avg: 19.3, 99%: 56.5, max: 85.0 ], Slowdowns: 0
2019/08/29 08:55:15 Loop: 0, Int: 5, Dur(s): 1.0, Mode: PUT, Ops: 6622, MB/s: 25.87, IO/s: 6622, Lat(ms): [ min: 2.7, avg: 19.2, 99%: 55.7, max: 86.8 ], Slowdowns: 0
2019/08/29 08:55:16 Loop: 0, Int: 6, Dur(s): 1.0, Mode: PUT, Ops: 5784, MB/s: 22.59, IO/s: 5784, Lat(ms): [ min: 1.9, avg: 22.1, 99%: 119.9, max: 157.2 ], Slowdowns: 0
2019/08/29 08:55:17 Loop: 0, Int: 7, Dur(s): 1.0, Mode: PUT, Ops: 6594, MB/s: 25.76, IO/s: 6594, Lat(ms): [ min: 1.9, avg: 19.5, 99%: 66.8, max: 93.7 ], Slowdowns: 0
2019/08/29 08:55:18 Loop: 0, Int: 8, Dur(s): 1.0, Mode: PUT, Ops: 7050, MB/s: 27.54, IO/s: 7050, Lat(ms): [ min: 2.9, avg: 17.9, 99%: 122.5, max: 158.1 ], Slowdowns: 0
2019/08/29 08:55:19 Loop: 0, Int: 9, Dur(s): 1.0, Mode: PUT, Ops: 8782, MB/s: 34.30, IO/s: 8782, Lat(ms): [ min: 3.6, avg: 14.9, 99%: 29.9, max: 42.3 ], Slowdowns: 0
2019/08/29 08:55:20 Loop: 0, Int: 10, Dur(s): 1.0, Mode: PUT, Ops: 8563, MB/s: 33.45, IO/s: 8563, Lat(ms): [ min: 4.3, avg: 15.0, 99%: 25.4, max: 34.4 ], Slowdowns: 0
2019/08/29 08:55:21 Loop: 0, Int: 11, Dur(s): 1.0, Mode: PUT, Ops: 7862, MB/s: 30.71, IO/s: 7862, Lat(ms): [ min: 4.9, avg: 16.2, 99%: 78.7, max: 88.8 ], Slowdowns: 0
2019/08/29 08:55:22 Loop: 0, Int: 12, Dur(s): 1.0, Mode: PUT, Ops: 8206, MB/s: 32.05, IO/s: 8206, Lat(ms): [ min: 3.0, avg: 15.6, 99%: 54.1, max: 85.6 ], Slowdowns: 0
2019/08/29 08:55:23 Loop: 0, Int: TOTAL, Dur(s): 13.9, Mode: PUT, Ops: 100000, MB/s: 28.17, IO/s: 7211, Lat(ms): [ min: 1.4, avg: 17.7, 99%: 57.2, max: 158.1 ], Slowdowns: 0
2019/08/29 08:55:23 Running Loop 0 BUCKET LIST TEST
2019/08/29 08:55:29 Loop: 0, Int: TOTAL, Dur(s): 5.5, Mode: LIST, Ops: 100, MB/s: 0.00, IO/s: 18, Lat(ms): [ min: 41.5, avg: 55.0, 99%: 65.4, max: 65.9 ], Slowdowns: 0
2019/08/29 08:55:29 Running Loop 0 BUCKET UNORDERED LIST TEST
2019/08/29 08:55:29 Loop: 0, Int: TOTAL, Dur(s): 0.1, Mode: UNORDERED LIST, Ops: 1, MB/s: 0.00, IO/s: 20, Lat(ms): [ min: 50.3, avg: 50.3, 99%: 50.3, max: 50.3 ], Slowdowns: 0

[markhpc]

nit: trying to keep these short since the log lines are so big. :) Can we change it from "UNORDERED LIST" to "ULIST"?

[markhpc]

I remember there was some issue with using marker with the V1 ListObjects which is ultimately why I switched to ListObjectsPages. Not sure if this is related to the issue I hit above?