[QE]monitor cpu usage in e2e test case #4977
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
WalkthroughReplaces per-step CPU checks in E2E feature files with timestamp-recording steps and Changes
Sequence Diagram(s)sequenceDiagram
participant Scenario
participant Monitor
participant FS as "test-results/*"
rect rgb(230,245,255)
Note over Scenario: Scenario tagged @performance
Scenario->>Monitor: NewMonitor(interval)
Scenario->>Monitor: Start()
Monitor-->>Monitor: spawn collectLoop (goroutine)
end
rect rgb(245,255,230)
Note over Scenario: Test execution
Scenario->>FS: Record timestamp ("start"/"deployment"/"stop"/"start again")
Monitor->>FS: Append periodic CPU sample to cpu-consume.txt
Scenario->>FS: Record memory data points (when present)
end
rect rgb(255,235,235)
Note over Scenario: Teardown
Scenario->>Monitor: Stop()
Monitor-->>Monitor: cancel context + wait for goroutine to finish
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (4)
test/e2e/testsuite/performace.go (3)
15-15: Minor: Fix typos in comments.Multiple comments contain typos:
- Line 15: "cancle" should be "cancel"
- Line 41: "stary" should be "start"
- Line 53: "cancle" should be "cancel"
- Line 70: "cancled" should be "cancelled"
Apply these corrections:
- // cancle flag of monitoring + // cancel flag of monitoring cancelFunc context.CancelFunc - // stary goroutine + // start goroutine go m.collectLoop(ctx) - // call cancle function + // call cancel function if m.cancelFunc != nil { - // 1. check Context whether be cancled + // 1. check Context whether be cancelled select {Also applies to: 41-41, 53-53, 70-70
34-34: Minor: Missing newline characters in fmt.Printf calls.Lines 34 and 84 use
fmt.Printfwithout a trailing newline, which can cause output formatting issues.Add newline characters:
- fmt.Printf("Attempt to start CPU collector, interval: %s", m.interval) + fmt.Printf("Attempt to start CPU collector, interval: %s\n", m.interval) - fmt.Printf("Error: fail to collect CPU data: %v", err) + fmt.Printf("Error: fail to collect CPU data: %v\n", err)Also applies to: 84-84
6-12: Minor: Fix import formatting and comment spacing.Static analysis indicates the file is not properly formatted. Run
goimportsandgofmtto fix import ordering and the missing space in the comment on line 11.Based on static analysis hints, run:
#!/bin/bash goimports -w test/e2e/testsuite/performace.go gofmt -w test/e2e/testsuite/performace.gotest/e2e/testsuite/testsuite.go (1)
1344-1345: Minor: Remove unnecessary blank lines.Lines 1344-1345 contain unnecessary blank lines that violate Go formatting conventions.
Remove the extra blank lines:
return util.WriteToFile(data, file) } - - - func getTimestamp(content string) error {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
test/e2e/features/story_microshift.feature(3 hunks)test/e2e/features/story_openshift.feature(2 hunks)test/e2e/testsuite/performace.go(1 hunks)test/e2e/testsuite/testsuite.go(5 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
test/e2e/testsuite/testsuite.go (2)
test/e2e/testsuite/performace.go (1)
NewMonitor(23-27)test/extended/util/fileops.go (1)
WriteToFile(93-108)
test/e2e/testsuite/performace.go (1)
test/extended/util/fileops.go (1)
WriteToFile(93-108)
🪛 GitHub Check: build (ubuntu-22.04, 1.24)
test/e2e/testsuite/testsuite.go
[failure] 269-269:
printf: fmt.Println call has possible Printf formatting directive %v (govet)
[failure] 400-400:
printf: fmt.Println call has possible Printf formatting directive %v (govet)
[failure] 1352-1352:
printf: fmt.Println call has possible Printf formatting directive %v (govet)
[failure] 1345-1345:
File is not properly formatted (gofmt)
[failure] 1355-1355:
Error return value of util.WriteToFile is not checked (errcheck)
test/e2e/testsuite/performace.go
[failure] 8-8:
File is not properly formatted (goimports)
[failure] 6-6:
File is not properly formatted (gofmt)
[failure] 11-11:
commentFormatting: put a space between // and comment text (gocritic)
[failure] 94-94:
Error return value of util.WriteToFile is not checked (errcheck)
🪛 GitHub Check: build (ubuntu-latest, 1.24)
test/e2e/testsuite/testsuite.go
[failure] 269-269:
printf: fmt.Println call has possible Printf formatting directive %v (govet)
[failure] 400-400:
printf: fmt.Println call has possible Printf formatting directive %v (govet)
[failure] 1352-1352:
printf: fmt.Println call has possible Printf formatting directive %v (govet)
[failure] 1345-1345:
File is not properly formatted (gofmt)
[failure] 1355-1355:
Error return value of util.WriteToFile is not checked (errcheck)
test/e2e/testsuite/performace.go
[failure] 8-8:
File is not properly formatted (goimports)
[failure] 6-6:
File is not properly formatted (gofmt)
[failure] 11-11:
commentFormatting: put a space between // and comment text (gocritic)
[failure] 94-94:
Error return value of util.WriteToFile is not checked (errcheck)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (14)
- GitHub Check: build (macOS-14, 1.24)
- GitHub Check: build (macOS-13, 1.24)
- GitHub Check: build-qe (windows, amd64)
- GitHub Check: build-qe (linux, arm64)
- GitHub Check: build-installer (windows-2022, 1.24)
- GitHub Check: build (windows-2022, 1.24)
- GitHub Check: Run OKD bundle with crc (1.24)
- GitHub Check: build-qe (darwin, arm64)
- GitHub Check: build-qe (linux, amd64)
- GitHub Check: build-qe (darwin, amd64)
- GitHub Check: build (ubuntu-latest, 1.24)
- GitHub Check: verify-devcontainer
- GitHub Check: build (macOS-13, 1.24)
- GitHub Check: Konflux kflux-prd-rh02 / crc-binary-on-pull-request
🔇 Additional comments (1)
test/e2e/testsuite/testsuite.go (1)
171-171: The monitor lifecycle concerns are addressed—code is correct.Verification confirms:
- All scenarios with
@performancetag properly manage Start/Stop (only 1 scenario uses it, both in conditional hooks)- Monitor cleanup happens even on scenario failure—Godog's After hook executes regardless of outcome
- The 5-second wait is intentional, documented in the code itself: "Wait for 5 seconds to confirm that the collection task will no longer output data"
Note: There is a minor bug at lines 268 and 399 using
fmt.Printlnwith format strings instead offmt.Printf, which means error messages won't display correctly. This is separate from the lifecycle concerns.Likely an incorrect or invalid review comment.
| if len(totalPercent) > 0 { | ||
| data := fmt.Sprintf("[%s], cpu percent: %.2f%%\n", | ||
| time.Now().Format("15:04:05"), totalPercent[0]) | ||
| wd, _ := os.Getwd() | ||
| file := filepath.Join(wd, "../test-results/cpu-consume.txt") | ||
| util.WriteToFile(data, file) | ||
| } |
There was a problem hiding this comment.
Major: Unhandled errors in data collection.
The error from util.WriteToFile is not checked (line 94), and the error from os.Getwd is silently ignored (line 92). This could lead to silent data loss.
Apply this diff to handle errors properly:
if len(totalPercent) > 0 {
data := fmt.Sprintf("[%s], cpu percent: %.2f%%\n",
time.Now().Format("15:04:05"), totalPercent[0])
- wd, _ := os.Getwd()
+ wd, err := os.Getwd()
+ if err != nil {
+ fmt.Printf("Error: failed to get working directory: %v\n", err)
+ continue
+ }
file := filepath.Join(wd, "../test-results/cpu-consume.txt")
- util.WriteToFile(data, file)
+ if err := util.WriteToFile(data, file); err != nil {
+ fmt.Printf("Error: failed to write CPU data: %v\n", err)
+ }
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if len(totalPercent) > 0 { | |
| data := fmt.Sprintf("[%s], cpu percent: %.2f%%\n", | |
| time.Now().Format("15:04:05"), totalPercent[0]) | |
| wd, _ := os.Getwd() | |
| file := filepath.Join(wd, "../test-results/cpu-consume.txt") | |
| util.WriteToFile(data, file) | |
| } | |
| if len(totalPercent) > 0 { | |
| data := fmt.Sprintf("[%s], cpu percent: %.2f%%\n", | |
| time.Now().Format("15:04:05"), totalPercent[0]) | |
| wd, err := os.Getwd() | |
| if err != nil { | |
| fmt.Printf("Error: failed to get working directory: %v\n", err) | |
| continue | |
| } | |
| file := filepath.Join(wd, "../test-results/cpu-consume.txt") | |
| if err := util.WriteToFile(data, file); err != nil { | |
| fmt.Printf("Error: failed to write CPU data: %v\n", err) | |
| } | |
| } |
🧰 Tools
🪛 GitHub Check: build (ubuntu-22.04, 1.24)
[failure] 94-94:
Error return value of util.WriteToFile is not checked (errcheck)
🪛 GitHub Check: build (ubuntu-latest, 1.24)
[failure] 94-94:
Error return value of util.WriteToFile is not checked (errcheck)
🤖 Prompt for AI Agents
In test/e2e/testsuite/performace.go around lines 89 to 95, the call to os.Getwd
and util.WriteToFile ignore returned errors which can cause silent failures;
update the code to capture and handle both errors: check the error from os.Getwd
and return or log it (with context) if non-nil, build the filepath only on
success, then call util.WriteToFile and check its error — if it fails, return or
log that error (with context) so data collection failures are not silent.
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
test/e2e/features/story_microshift.feature (1)
22-41: Remove orphan step: “get cpu data” no longer has a step definition.CPU collection moved to background Monitor; this step will fail at runtime.
Apply:
- When with up to "20" retries with wait period of "5s" http response from "http://httpd-example-testproj.apps.crc.testing" has status code "200" - And get cpu data "After deployment" + When with up to "20" retries with wait period of "5s" http response from "http://httpd-example-testproj.apps.crc.testing" has status code "200" + # CPU data is collected by the background monitor; explicit CPU step removed
♻️ Duplicate comments (2)
test/e2e/testsuite/performance.go (1)
98-104: Handle filesystem errors when writing samples (Getwd/WriteToFile).Errors are ignored; causes silent data loss and CI failures. Same concern was raised earlier.
Apply:
- wd, _ := os.Getwd() - file := filepath.Join(wd, "../test-results/cpu-consume.txt") - util.WriteToFile(data, file) + wd, err := os.Getwd() + if err != nil { + fmt.Printf("Error: failed to get working directory: %v\n", err) + continue + } + file := filepath.Join(wd, "../test-results/cpu-consume.txt") + if err := os.MkdirAll(filepath.Dir(file), 0o755); err != nil { + fmt.Printf("Error: failed to create results dir: %v\n", err) + continue + } + if err := util.WriteToFile(data, file); err != nil { + fmt.Printf("Error: failed to write CPU data: %v\n", err) + }test/e2e/testsuite/testsuite.go (1)
1346-1355: Return error from getTimestamp; fix printf and check WriteToFile (matches prior review).Currently errors are dropped and fmt.Println uses a formatting verb.
Apply:
-func getTimestamp(content string) { +func getTimestamp(content string) error { data := fmt.Sprintf("[%s], %s\n", time.Now().Format("15:04:05"), content) wd, err := os.Getwd() if err != nil { - fmt.Println("failed to get working directory: %v", err) + return fmt.Errorf("failed to get working directory: %v", err) } file := filepath.Join(wd, "../test-results/time-stamp.txt") - util.WriteToFile(data, file) + if err := os.MkdirAll(filepath.Dir(file), 0o755); err != nil { + return fmt.Errorf("failed to create results dir: %v", err) + } + return util.WriteToFile(data, file) }Also update callers to handle the returned error (see Before hook change above).
🧹 Nitpick comments (2)
test/e2e/testsuite/performance.go (1)
38-51: Minor logging/comment polish.Add newline to Printf; fix typos in comments (“start goroutine”, “cancel”, “canceled”). No behavior change.
- fmt.Printf("Attempt to start CPU collector, interval: %s", m.interval) + fmt.Printf("Attempt to start CPU collector, interval: %s\n", m.interval) - // create a context.WithCancel + // create a context.WithCancel - // stary goroutine + // start goroutinetest/e2e/testsuite/testsuite.go (1)
399-405: Nice fix on Printf; consider removing the extra 5s sleep.Stop waits for the collector goroutine (wg.Wait), so an additional fixed 5s delay slows tests without benefit. If you want a guard, sleep for at most one collection interval.
- fmt.Printf("Collection has stopped. Wait for 5 seconds to confirm that the collection task will no longer output data\n") - time.Sleep(5 * time.Second) + // Optional: allow up to one interval for any buffered I/O to flush + // time.Sleep(1 * time.Second)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
test/e2e/features/story_microshift.feature(3 hunks)test/e2e/features/story_openshift.feature(2 hunks)test/e2e/testsuite/performance.go(1 hunks)test/e2e/testsuite/testsuite.go(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- test/e2e/features/story_openshift.feature
🧰 Additional context used
🧬 Code graph analysis (2)
test/e2e/testsuite/performance.go (1)
test/extended/util/fileops.go (1)
WriteToFile(93-108)
test/e2e/testsuite/testsuite.go (2)
test/e2e/testsuite/performance.go (1)
NewMonitor(24-28)test/extended/util/fileops.go (1)
WriteToFile(93-108)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (16)
- GitHub Check: build (windows-2022, 1.24)
- GitHub Check: Run OKD bundle with crc (1.24)
- GitHub Check: build (ubuntu-latest, 1.24)
- GitHub Check: Konflux kflux-prd-rh02 / crc-binary-on-pull-request
- GitHub Check: build-qe (windows, amd64)
- GitHub Check: build-qe (darwin, amd64)
- GitHub Check: build (ubuntu-22.04, 1.24)
- GitHub Check: build-qe (darwin, arm64)
- GitHub Check: build-qe (linux, arm64)
- GitHub Check: build-qe (linux, amd64)
- GitHub Check: build (ubuntu-latest, 1.24)
- GitHub Check: build (macOS-14, 1.24)
- GitHub Check: build (macOS-13, 1.24)
- GitHub Check: build (macOS-14, 1.24)
- GitHub Check: build-installer (windows-2022, 1.24)
- GitHub Check: verify-devcontainer
🔇 Additional comments (3)
test/e2e/features/story_microshift.feature (1)
11-11: Timestamp step addition looks good.test/e2e/testsuite/testsuite.go (2)
171-171: Monitor instance scope: confirm no parallel scenarios.Single monitor captured by closures; parallel execution would interleave Start/Stop. If parallelism is possible, create one per scenario (in Before) or guard with a package-level singleton and a refcount.
587-589: Step should propagate failures from timestamp writer.After updating getTimestamp to return error (see below), keeping this binding will surface I/O issues to Godog.
lilyLuLiu
force-pushed
the
main
branch
2 times, most recently
from
776af4f to
7bcaf7c
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (2)
test/e2e/testsuite/testsuite.go (1)
1350-1359: Return error immediately when os.Getwd() fails.When
os.Getwd()fails (line 1354), the code prints an error but continues to use the invalid working directory to construct the file path. This will causeutil.WriteToFileto fail. Instead, return the error immediately, consistent with howgetMemoryDatahandles the same error (lines 1342-1345).Apply this diff:
func getTimestamp(content string) error { data := fmt.Sprintf("[%s], %s\n", time.Now().Format("15:04:05"), content) wd, err := os.Getwd() if err != nil { - fmt.Printf("failed to get working directory: %v\n", err) + return fmt.Errorf("failed to get working directory: %v", err) } file := filepath.Join(wd, "../test-results/time-stamp.txt") return util.WriteToFile(data, file) }test/e2e/testsuite/performance.go (1)
95-104: Handle os.Getwd() error properly.When
os.Getwd()fails (line 98), the code continues with an invalid path, which will causeutil.WriteToFileto fail. Instead of ignoring this error, handle it and continue to the next iteration.Apply this diff:
if len(totalPercent) > 0 { data := fmt.Sprintf("[%s], cpu percent: %.2f%%\n", time.Now().Format("15:04:05"), totalPercent[0]) - wd, _ := os.Getwd() + wd, err := os.Getwd() + if err != nil { + fmt.Printf("Error: failed to get working directory: %v\n", err) + continue + } file := filepath.Join(wd, "../test-results/cpu-consume.txt") err := util.WriteToFile(data, file) if err != nil { fmt.Printf("Error: fail to write to %s: %v", file, err) } }
🧹 Nitpick comments (7)
test/e2e/testsuite/performance.go (4)
37-37: Add trailing newline to log message.The log message on line 37 is missing a trailing newline, which will cause the next log output to appear on the same line.
Apply this diff:
- fmt.Printf("Attempt to start CPU collector, interval: %s", m.interval) + fmt.Printf("Attempt to start CPU collector, interval: %s\n", m.interval)
52-67: Release mutex before calling cancelFunc and wg.Wait().While the current implementation is safe, it's better practice to release the mutex before calling
cancelFunc()andwg.Wait(). Holding the mutex during these blocking operations is unnecessary and could cause issues if the cancellation triggers any callbacks that need the mutex.Apply this diff:
func (m *Monitor) Stop() error { m.mu.Lock() - defer m.mu.Unlock() if !m.isRunning { + m.mu.Unlock() return fmt.Errorf("The collector is not running") } + cancel := m.cancelFunc + m.isRunning = false + m.mu.Unlock() + if m.cancelFunc != nil { - m.cancelFunc() + cancel() } - m.isRunning = false m.wg.Wait() fmt.Println("CPU collector has sent a stop signal") - // may need wait a while to stop return nil }
76-76: Fix typo in comment."cancled" should be "cancelled".
Apply this diff:
- // 1. check Context whether be cancled + // 1. check Context whether be cancelled
105-117: Remove or document commented code.The large block of commented-out memory collection code should either be removed if it's no longer needed, or documented with a TODO/NOTE comment explaining why it's kept for future use.
test/e2e/testsuite/testsuite.go (3)
267-272: Consider recording a start timestamp.Based on past review comments, it was suggested to record a "start" timestamp when entering the
@performancescenario. Currently, only a "finish" timestamp is recorded in the After hook (line 400). Recording both start and finish timestamps would provide complete timing data for performance analysis.Apply this diff:
if tag.Name == "@performance" { - + if err := getTimestamp("start"); err != nil { + fmt.Printf("Failed to record start timestamp: %v\n", err) + } if err := monitor.Start(); err != nil { fmt.Printf("Failed to start monitor: %v\n", err) } }
400-403: Improve error message wording.The error message "Failed to get finish getTimestamp" is awkwardly phrased. Consider rephrasing to "Failed to record finish timestamp" for clarity.
Apply this diff:
err := getTimestamp("finish") if err != nil { - fmt.Printf("Failed to get finish getTimestamp: %v\n", err) + fmt.Printf("Failed to record finish timestamp: %v\n", err) }
407-407: Use fmt.Println instead of fmt.Printf for plain strings.When printing a plain string without format specifiers, use
fmt.Printlninstead offmt.Printffor consistency and clarity.Apply this diff:
- fmt.Printf("Collection has stopped. Wait for 5 seconds to confirm that the collection task will no longer output data\n") + fmt.Println("Collection has stopped. Wait for 5 seconds to confirm that the collection task will no longer output data")
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
test/e2e/features/story_microshift.feature(3 hunks)test/e2e/features/story_openshift.feature(2 hunks)test/e2e/testsuite/performance.go(1 hunks)test/e2e/testsuite/testsuite.go(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- test/e2e/features/story_microshift.feature
- test/e2e/features/story_openshift.feature
🧰 Additional context used
🧬 Code graph analysis (2)
test/e2e/testsuite/performance.go (1)
test/extended/util/fileops.go (1)
WriteToFile(93-108)
test/e2e/testsuite/testsuite.go (2)
test/e2e/testsuite/performance.go (1)
NewMonitor(23-27)test/extended/util/fileops.go (1)
WriteToFile(93-108)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (18)
- GitHub Check: verify-devcontainer
- GitHub Check: build (ubuntu-latest, 1.24)
- GitHub Check: build-qe (linux, amd64)
- GitHub Check: build-qe (linux, arm64)
- GitHub Check: build-qe (windows, amd64)
- GitHub Check: build-qe (darwin, arm64)
- GitHub Check: build-qe (darwin, amd64)
- GitHub Check: build-installer (windows-2022, 1.24)
- GitHub Check: Run OKD bundle with crc (1.24)
- GitHub Check: build (macOS-14, 1.24)
- GitHub Check: build (macOS-13, 1.24)
- GitHub Check: build (windows-2022, 1.24)
- GitHub Check: build (windows-2022, 1.24)
- GitHub Check: build (macOS-14, 1.24)
- GitHub Check: build (ubuntu-22.04, 1.24)
- GitHub Check: build (ubuntu-latest, 1.24)
- GitHub Check: build (macOS-13, 1.24)
- GitHub Check: Konflux kflux-prd-rh02 / crc-binary-on-pull-request
lilyLuLiu
force-pushed
the
main
branch
3 times, most recently
from
e725ad3 to
f9dd334
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
test/e2e/testsuite/testsuite.go (1)
1354-1363: Critical: Return immediately when os.Getwd() fails.If
os.Getwd()fails at line 1358, the error is logged but the function continues to construct a file path with a potentially emptywdvariable. This leads to an invalid path being passed toutil.WriteToFile, and the root cause error is lost, making debugging difficult.Apply this diff to return the error immediately:
func getTimestamp(content string) error { data := fmt.Sprintf("[%s], %s\n", time.Now().Format("15:04:05"), content) wd, err := os.Getwd() if err != nil { - fmt.Printf("failed to get working directory: %v\n", err) + return fmt.Errorf("failed to get working directory: %v", err) } file := filepath.Join(wd, "../test-results/time-stamp.txt") return util.WriteToFile(data, file) }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
test/e2e/features/story_microshift.feature(2 hunks)test/e2e/features/story_openshift.feature(2 hunks)test/e2e/testsuite/performance.go(1 hunks)test/e2e/testsuite/testsuite.go(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- test/e2e/testsuite/performance.go
- test/e2e/features/story_openshift.feature
🧰 Additional context used
🧬 Code graph analysis (1)
test/e2e/testsuite/testsuite.go (2)
test/e2e/testsuite/performance.go (1)
NewMonitor(23-27)test/extended/util/fileops.go (1)
WriteToFile(93-108)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (17)
- GitHub Check: verify-devcontainer
- GitHub Check: build (ubuntu-latest, 1.24)
- GitHub Check: build (windows-2022, 1.24)
- GitHub Check: build (macOS-14, 1.24)
- GitHub Check: build (macOS-13, 1.24)
- GitHub Check: build (macOS-13, 1.24)
- GitHub Check: build-qe (linux, arm64)
- GitHub Check: build (ubuntu-22.04, 1.24)
- GitHub Check: build (macOS-14, 1.24)
- GitHub Check: build (ubuntu-latest, 1.24)
- GitHub Check: Run OKD bundle with crc (1.24)
- GitHub Check: build-qe (linux, amd64)
- GitHub Check: build-qe (windows, amd64)
- GitHub Check: build-qe (darwin, arm64)
- GitHub Check: build-qe (darwin, amd64)
- GitHub Check: build-installer (windows-2022, 1.24)
- GitHub Check: Konflux kflux-prd-rh02 / crc-binary-on-pull-request
🔇 Additional comments (4)
test/e2e/features/story_microshift.feature (1)
20-43: LGTM! Timestamp recording steps are well-placed.The addition of the
@performancetag and timestamp recording steps aligns with the PR's objective to implement timestamp-based performance monitoring. The placement of timestamps around deployment, stop, and restart operations is logical and will enable performance analysis of these critical operations.test/e2e/testsuite/testsuite.go (3)
171-171: LGTM! Monitor initialization is appropriate.The monitor is correctly initialized with a 1-second sampling interval and scoped to the scenario lifecycle.
403-413: LGTM! Performance monitoring teardown is correctly implemented.The cleanup sequence properly records the finish timestamp, stops the monitor with appropriate error handling, and includes a verification delay.
595-596: LGTM! Step binding is correctly defined.The regex pattern and function binding are appropriate for capturing timestamp labels from the feature files.
| if tag.Name == "@performance" { | ||
| if err := monitor.Start(); err != nil { | ||
| fmt.Printf("Failed to start monitor: %v\n", err) | ||
| } | ||
| if err := getMemoryData("Before start"); err != nil { | ||
| fmt.Printf("Failed to collect memory data: %v\n", err) | ||
| err := getTimestamp("start") | ||
| if err != nil { | ||
| fmt.Printf("Failed to get finish getTimestamp: %v\n", err) | ||
| } | ||
| getMemoryData("Before start") | ||
| } |
There was a problem hiding this comment.
Fix misleading error message and handle getMemoryData error.
Two issues in the performance monitoring setup:
- Line 273: The error message says "Failed to get finish getTimestamp" but this is recording the "start" timestamp, making debugging confusing.
- Line 275: The error returned by
getMemoryDatais not checked, which is inconsistent with the error handling forgetTimestampand could hide failures in memory data collection.
Apply this diff:
if tag.Name == "@performance" {
if err := monitor.Start(); err != nil {
fmt.Printf("Failed to start monitor: %v\n", err)
}
err := getTimestamp("start")
if err != nil {
- fmt.Printf("Failed to get finish getTimestamp: %v\n", err)
+ fmt.Printf("Failed to record start timestamp: %v\n", err)
}
- getMemoryData("Before start")
+ if err := getMemoryData("Before start"); err != nil {
+ fmt.Printf("Failed to collect memory data: %v\n", err)
+ }
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if tag.Name == "@performance" { | |
| if err := monitor.Start(); err != nil { | |
| fmt.Printf("Failed to start monitor: %v\n", err) | |
| } | |
| if err := getMemoryData("Before start"); err != nil { | |
| fmt.Printf("Failed to collect memory data: %v\n", err) | |
| err := getTimestamp("start") | |
| if err != nil { | |
| fmt.Printf("Failed to get finish getTimestamp: %v\n", err) | |
| } | |
| getMemoryData("Before start") | |
| } | |
| if tag.Name == "@performance" { | |
| if err := monitor.Start(); err != nil { | |
| fmt.Printf("Failed to start monitor: %v\n", err) | |
| } | |
| err := getTimestamp("start") | |
| if err != nil { | |
| fmt.Printf("Failed to record start timestamp: %v\n", err) | |
| } | |
| if err := getMemoryData("Before start"); err != nil { | |
| fmt.Printf("Failed to collect memory data: %v\n", err) | |
| } | |
| } |
🤖 Prompt for AI Agents
In test/e2e/testsuite/testsuite.go around lines 267 to 276, the error message
for getTimestamp incorrectly references "finish" while recording the "start"
timestamp and getMemoryData's returned error is ignored; change the fmt.Printf
call to correctly state "Failed to get start timestamp" (or similar) when
getTimestamp("start") fails, and capture and handle the error returned by
getMemoryData (e.g., if err := getMemoryData("Before start"); err != nil {
fmt.Printf("Failed to collect memory data: %v\n", err) }) so both operations
have consistent, accurate error handling and messages.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
test/e2e/features/story_microshift.feature (1)
44-48: Consider adding memory data collection after restart for consistency.Memory data is collected at key checkpoints: "After start" (line 11), "After deployment" (line 37), and "After stop" (line 42). However, there's no memory data collection after the restart on line 44. For comprehensive performance monitoring and consistency, consider adding a memory checkpoint after the CRC restart.
Apply this diff to add a memory checkpoint after restart:
And record timestamp "start again" And starting CRC with default bundle succeeds + And get memory data "After restart" And checking that CRC is running And with up to "4" retries with wait period of "1m" http response from "http://httpd-example-testproj.apps.crc.testing" has status code "200"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
test/e2e/features/story_microshift.feature(2 hunks)test/e2e/features/story_openshift.feature(2 hunks)test/e2e/testsuite/performance.go(1 hunks)test/e2e/testsuite/testsuite.go(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- test/e2e/testsuite/performance.go
- test/e2e/testsuite/testsuite.go
- test/e2e/features/story_openshift.feature
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (17)
- GitHub Check: build-qe (linux, arm64)
- GitHub Check: build-qe (darwin, arm64)
- GitHub Check: build-qe (linux, amd64)
- GitHub Check: build-qe (windows, amd64)
- GitHub Check: build-qe (darwin, amd64)
- GitHub Check: build-installer (windows-2022, 1.24)
- GitHub Check: Run OKD bundle with crc (1.24)
- GitHub Check: build (ubuntu-22.04, 1.24)
- GitHub Check: build (ubuntu-latest, 1.24)
- GitHub Check: build (macOS-14, 1.24)
- GitHub Check: build (ubuntu-latest, 1.24)
- GitHub Check: verify-devcontainer
- GitHub Check: build (windows-2022, 1.24)
- GitHub Check: build (windows-2022, 1.24)
- GitHub Check: build (macOS-14, 1.24)
- GitHub Check: build (macOS-13, 1.24)
- GitHub Check: Konflux kflux-prd-rh02 / crc-binary-on-pull-request
🔇 Additional comments (2)
test/e2e/features/story_microshift.feature (2)
20-20: LGTM:@performancetag added for CPU monitoring.The addition of the
@performancetag aligns with the PR objective to monitor CPU usage in E2E tests. This tag triggers the Monitor lifecycle to sample CPU data periodically during scenario execution.
22-22: No issues found. The timestamp labels are correctly handled.The
getTimestampfunction implementation (testsuite.go:1357-1366) shows that the timestamp label parameter is only used for string formatting when writing to a fixed log file (../test-results/time-stamp.txt). Spaces in labels like"start again"are handled correctly—they appear as plain text in the timestamp log without any adverse effects. The label is not used in file naming, identifier derivation, or path operations where spaces might cause issues.Likely an incorrect or invalid review comment.
|
@lilyLuLiu: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
1 participant
Description
Fixes: #N
Relates to: #N, PR #N, ...
Type of change
test, version modification, documentation, etc.)
Proposed changes
Testing
Contribution Checklist
Summary by CodeRabbit
Tests
New Features