I tried running some benchmarks on Java 8 and got an error "CICompilerCount of 1 is invalid; must be at least 2"
As far as I know, TieredCompilation was introduced in Java 7 but not enabled by default. It seems to be enabled by default for Java 8, so both the client and server compilers need their own threads.
Simply disabling it with -XX:-TieredCompliation does the trick, but I was wondering if this could be addressed in the caliper codebase.