petri: lower severity of com3 emitted logs for all vmm_tests #2003
Conversation
petri/src/vm/mod.rs
Outdated
| Some(client_ref) => { | ||
| tracing::info!("Agent is online now"); | ||
| let sh = client_ref.unix_shell(); | ||
| pipette_client::cmd!(sh, "dmesg -n 3").run().await?; |
There was a problem hiding this comment.
We shouldn't need to do any system level filtering like this, instead we can alter the value for OPENVMM_LOG that we pass to the command line during vm construction.
There was a problem hiding this comment.
That said, we probably shouldn't be doing this in tests anyways. If a test is too noisy we should lower the levels of individual events from debug to trace.
There was a problem hiding this comment.
i disagree - i think we want com3 output, but we want to lower the output we spit out once we have better channels available (like kmsg). com3 is really really slow, so we shouldn't rely on verbose output there of any kind.
There was a problem hiding this comment.
note that this is because early output (kernel/bootshim) we have no other mechanism. we could also perhaps look at virtio serial for both as well in a future change.
There was a problem hiding this comment.
In that case we'd need to handle reboots here too.
petri/src/vm/mod.rs
Outdated
| self.wait_for_agent().await | ||
| let client = self.wait_for_agent().await?; | ||
|
|
||
| match self.openhcl_diag() { |
There was a problem hiding this comment.
Will refactor the duplicated code, but I wanted to get two things done first:
- List of test failures
- Feedback on the nature of these changes, to see if I'm on the right track.
damanm24
changed the title
damanm24
changed the title
tjones60
added
the
backport_2505
#2003 Introduced a change to potentially fix a flaky vmm_test related to validating mnf usage in guests. However, it looks like the test flakiness has not been resolved. I am disabling the test for now to unblock contributors while I have a proper chance to investigate the issue. --------- Co-authored-by: Daman Mulye <[email protected]>
…icrosoft#2003)" This reverts commit a1ac181.
chris-oo
removed
the
backport_2505
5 participants
#1937 introduced new vmm_tests which seem to be flaky as they will occasionally timeout. Upon further investigation, the following message can be observed in the petri logs:
hv_vmbus: probe failed for device 766e96f8-2ceb-437e-afe3-a93169e48a7b. It's likely that openhcl is busy with other tasks (such as writing to com3) so it is unable to handle the vmbus probe request from vtl0. This PR attempts to fix that by only emitting high severity logs once kmsg is available.