test(e2e): standardize smoke vs full distinction and default to real-…#132
test(e2e): standardize smoke vs full distinction and default to real-…#132
Conversation
Summary of ChangesHello @teng-lin, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request refines the end-to-end (E2E) testing strategy by changing the default test profile to 'real-full' for more comprehensive testing against real CLI binaries. It also ensures that the full test suite is only executed when the 'real-full' profile is explicitly selected and all necessary prerequisites are met, providing clearer distinction and control over test execution. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request standardizes the logic for enabling smoke and full end-to-end tests and changes the default test profile to real-full. The changes are consistent across the test suite and correctly implement the intended logic. I've added one suggestion to refactor the now-duplicated test condition logic into a shared helper function to improve maintainability, in line with our best practices for avoiding code duplication and extracting shared helper functions to common utility files.
| const profile = getE2EProfile(); | ||
| const prereqs = getAgentSdkPrereqState(); | ||
| const canBindLocalhost = canBindLocalhostSync(); | ||
| const runSmoke = prereqs.ok && canBindLocalhost; | ||
| const runFull = runSmoke && prereqs.canRunPromptTests; | ||
| const runFull = runSmoke && prereqs.canRunPromptTests && profile === "real-full"; |
There was a problem hiding this comment.
This block of logic for determining runSmoke and runFull is now duplicated across several e2e test files (agent-sdk, claude, codex, gemini, opencode). To improve maintainability and reduce code duplication, consider extracting this logic into a shared helper function.
For example, you could add a function in a shared module like prereqs.ts or a new helper file:
// In a shared helper module
import { canBindLocalhostSync } from "./helpers.js";
import { getE2EProfile } from "./e2e-profile.js";
import type { BackendPrereqState } from "./prereqs.js";
export function getTestRunConditions(prereqs: BackendPrereqState) {
const profile = getE2EProfile();
const canBindLocalhost = canBindLocalhostSync();
const runSmoke = prereqs.ok && canBindLocalhost;
const runFull = runSmoke && prereqs.canRunPromptTests && profile === "real-full";
return { runSmoke, runFull };
}Then, each test file could simply do:
import { getTestRunConditions } from "./path/to/helpers.js";
import { getAgentSdkPrereqState } from "./prereqs.js";
const { runSmoke, runFull } = getTestRunConditions(getAgentSdkPrereqState());
describe(...)This would make the setup in each test file cleaner and ensure consistency if this logic needs to be changed in the future.
References
- To improve maintainability and avoid code duplication, extract repeated logic blocks into a private helper method.
- Extract shared helper functions (e.g., store access helpers) to a common utility or store file to prevent code duplication across multiple components.
…e integration tests from e2e profiles
1 participant
agent-sdk, claude) to consistently use E2E_PROFILE=real-full as a guard for prompt-based
tests.
real-full: Changed the default E2E profile from mock to real-full insrc/e2e/e2e-profile.ts, aligning with the project's move toward real-binary testing.
backends to prevent failures when credentials are missing.