Realtime Coverage

[iiTzHyper]

Description

Realtime Coverage

Type of change

• New feature (non-breaking change which adds functionality)

[b-scholz]

This is the error log:

21:22:44  stochastic/coverage/counters_test.go:33:14: Error return value of `binary.Write` is not checked (errcheck)
21:22:44  	binary.Write(buf, binary.LittleEndian, counterFileHeader{
21:22:44  	            ^
21:22:44  stochastic/coverage/counters_test.go:42:14: Error return value of `binary.Write` is not checked (errcheck)
21:22:44  	binary.Write(buf, binary.LittleEndian, counterSegmentHeader{
21:22:44  	            ^
21:22:44  stochastic/coverage/counters_test.go:49:14: Error return value of `binary.Write` is not checked (errcheck)
21:22:44  	binary.Write(buf, binary.LittleEndian, uint32(3))  // numCounters
21:22:44  	            ^
21:22:44  stochastic/coverage/counters_test.go:308:17: Error return value of `reader.ReadByte` is not checked (errcheck)
21:22:44  	reader.ReadByte()
21:22:44  	               ^
21:22:44  stochastic/coverage/counters.go:75:3: QF1006: could lift into loop condition (staticcheck)
21:22:44  		if reader.Len() == counterFileFooterSize {
21:22:44  		^
21:22:44  5 issues:
21:22:44  * errcheck: 4
21:22:44  * staticcheck: 1
21:22:44  make: *** [Makefile:144: check] Error 1

[codecov]

Codecov Report

❌ Patch coverage is 91.80000% with 41 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
stochastic/coverage/meta.go	84.46%	8 Missing and 8 partials ⚠️
stochastic/coverage/counters.go	88.63%	5 Missing and 5 partials ⚠️
stochastic/replayer/replay.go	82.75%	4 Missing and 6 partials ⚠️
stochastic/coverage/filter.go	93.02%	2 Missing and 1 partial ⚠️
stochastic/coverage/tracker.go	97.22%	1 Missing and 1 partial ⚠️

Files with missing lines	Coverage Δ
cmd/aida-stochastic-sdb/stochastic/replay.go	62.79% <ø> (+13.95%)	⬆️
stochastic/coverage/reader.go	100.00% <100.00%> (ø)
stochastic/replayer/coverage_bias.go	100.00% <100.00%> (ø)
stochastic/statistics/markov/markov.go	95.18% <100.00%> (+1.43%)	⬆️
utils/config.go	87.59% <ø> (ø)
utils/default_config.go	98.00% <100.00%> (+0.02%)	⬆️
stochastic/coverage/tracker.go	97.22% <97.22%> (ø)
stochastic/coverage/filter.go	93.02% <93.02%> (ø)
stochastic/coverage/counters.go	88.63% <88.63%> (ø)
stochastic/replayer/replay.go	92.41% <82.75%> (-1.56%)	⬇️
... and 1 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[matejmode]

Please request review again once you have rebased the branch.

[wsodsong]

Can you please add a short description to this PR explaining how coverage works? It would help greatly.

[iiTzHyper]

Can you please add a short description to this PR explaining how coverage works? It would help greatly.

It snapshots runtime coverage every N operations. We read Go’s runtime coverage counters in-process, filter to Carmen packages, and track new coverage lines discovered since the previous snapshot. This is then used to boost the Markov chain weights of states so more weights is given towards unexplored code.

This feedback loop makes the stochastic runner a coverage-guided fuzzer which is faster at surfacing untested paths instead of blindly following recorded frequencies. The previous replay sampled transitions purely from the Markov model. It could loop in frequent paths, miss rarer branches, and provided no coverage metrics or progress signal.

[matejmode]

Looking good. Thanks