Update dry run for atmos vendor pull to support ssh + detailed SCP urls alignment #1076
Conversation
Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]>
Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]>
📝 WalkthroughWalkthroughThis pull request revises the URL detection and handling in the Go execution utilities. It renames and upgrades the Git detector to support SCP-style URLs and multiple Git hosting services with improved token injection. Logging is added to vendor-related functions for better debugging during dry-run operations, and a function for masking basic authentication in URLs is introduced. Additionally, new test cases and configuration files are added to support different vendor pull scenarios. Documentation is updated with new environment variables for Bitbucket and GitLab authentication. Changes
Possibly related PRs
Suggested labels
📜 Recent review detailsConfiguration used: .coderabbit.yaml 📒 Files selected for processing (1)
🧰 Additional context used🧠 Learnings (1)internal/exec/go_getter_utils.go (1)⏰ Context from checks skipped due to timeout of 90000ms (2)
🔇 Additional comments (11)
✨ Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
|
|
Listener430
changed the title
Listener430
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (10)
tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml (3)
1-1: Excess Blank Line at File Start
There's an extra blank line at the beginning that triggers the YAML lint warning. Consider removing it for cleaner formatting.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
13-13: Trailing Spaces Detected
Line 13 contains trailing whitespace. Please remove the extra spaces to resolve the linting error.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 13-13: trailing spaces
(trailing-spaces)
21-21: Extra Blank Line at End of Section
An unnecessary blank line is present at line 21. Removing it will clear the YAML lint warning.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 21-21: too many blank lines
(1 > 0) (empty-lines)
tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml (3)
1-1: Remove Leading Blank Line
A leading blank line is present at the top of the file. Consider removing it to satisfy YAML lint requirements.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
13-13: Eliminate Trailing Spaces
Trailing spaces are detected at line 13. Removing these spaces will improve the file’s consistency.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 13-13: trailing spaces
(trailing-spaces)
22-22: Extra Blank Line Detected
An extra blank line at line 22 is causing a lint warning. Please remove it to tidy up the file.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 22-22: too many blank lines
(2 > 0) (empty-lines)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
18-18: Trailing Spaces in Metadata Section
Trailing whitespace was found on line 18. Removing these spaces will resolve the linting issue.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 18-18: trailing spaces
(trailing-spaces)
tests/test-cases/demo-vendoring.yaml (2)
96-96: Remove Trailing Spaces in Test Block
Trailing spaces were detected on line 96. Please remove them to tidy up the YAML file.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 96-96: trailing spaces
(trailing-spaces)
98-98: Eliminate Trailing Whitespace
Line 98 has extra whitespace at the end. Cleaning this up will resolve the linting warning.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 98-98: trailing spaces
(trailing-spaces)
website/docs/cli/configuration/configuration.mdx (1)
681-683: Clear and Informative Addition of Authentication Environment Variables
The new entries forATMOS_BITBUCKET_TOKEN,ATMOS_BITBUCKET_USERNAME, andATMOS_GITLAB_TOKENare well-documented and align with the rest of the environment variable table. They clearly state the purpose and rate-limit considerations for each token, which will help users manage Bitbucket and GitLab authentication effectively.A minor suggestion: consider adding links to the official Bitbucket and GitLab documentation for users seeking more detailed guidance on token generation and usage.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (13)
internal/exec/go_getter_utils.go(3 hunks)internal/exec/vendor_model.go(2 hunks)internal/exec/vendor_model_component.go(2 hunks)pkg/utils/url_utils.go(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml(1 hunks)tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendoring-dry-run/components/terraform/ipinfo/component.yaml(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh_component.stderr.golden(1 hunks)tests/test-cases/demo-vendoring.yaml(1 hunks)website/docs/cli/configuration/configuration.mdx(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh_component.stderr.golden (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-02-14T23:12:38.030Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
internal/exec/go_getter_utils.go (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: internal/exec/go_getter_utils.go:74-75
Timestamp: 2025-02-13T07:30:28.946Z
Learning: In the `CustomGitDetector.Detect` method of `internal/exec/go_getter_utils.go`, verbose debug logging of raw URLs is intentionally kept for debugging purposes, despite potential credential exposure risks.
🪛 YAMLlint (1.35.1)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml
[error] 18-18: trailing spaces
(trailing-spaces)
tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
[warning] 22-22: too many blank lines
(2 > 0) (empty-lines)
tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
[warning] 21-21: too many blank lines
(1 > 0) (empty-lines)
tests/test-cases/demo-vendoring.yaml
[error] 96-96: trailing spaces
(trailing-spaces)
[error] 98-98: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (19)
internal/exec/go_getter_utils.go (7)
61-61: Good to enable composite SSH scheme.Including
"git::ssh"in the list of valid schemes is correct for go-getter to handle composite SSH requests seamlessly.
66-71: Struct name and field additions look appropriate.Renaming it to
CustomGitDetectorclarifies that it handles multiple services, not just GitHub. Thesourcefield supports subdir logic, aligning well with the existing workflow.
74-117: SCP-style URL rewrite logic looks robust.The regex-based approach for detecting and transforming SCP-style URLs into standard SSH URLs is clean and adequately logged. The fallback to prepending "https://" is a practical default. The debug logging is well-structured, and sensitive info is masked via
MaskBasicAuth.
120-142: Thoughtful error and environment checks.Parsing the URL early and warning about
SSH_AUTH_SOCKis helpful for diagnosing SSH agent issues upfront. This aligns with a more user-friendly debugging experience.
144-209: Token injection mechanism is clear and modular.Switching on the host for different tokens (e.g., GitHub, GitLab) is well-organized. Defaulting to a safe username and masking credentials in logs ensures better security while maintaining clarity.
211-222: Subdirectory logic is handled gracefully.Automatically appending
//.to top-level repos when a subdir isn't specified is a handy usability enhancement, though do watch for edge cases involving custom paths.Would you like a brief test script to search for references to confirm no conflicting path usage?
224-242: Setting a shallow clone by default is sensible.Enforcing
depth=1by default can save clone time. Since you allow it to be overridden, this is a solid choice for performance-conscious workflows.pkg/utils/url_utils.go (1)
8-20: Straightforward masking utility is well-implemented.Replacing credentials with
"xxx"ensures logs don’t reveal sensitive data. It’d be beneficial to confirm handling of edge cases (like a missing password) through tests, but overall this is a clean approach.tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh_component.stderr.golden (1)
1-17: Snapshot logs are consistent with the new logic.The debug messages match the SCP rewrite, agent checks, and token injection. Credentials are masked as expected. This confirms the correctness of recent changes in dry-run mode.
tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden (1)
1-35: LGTM! The test snapshot provides comprehensive logging coverage.The snapshot effectively captures the expected log output for the vendoring process, including:
- Debug logs for configuration and package processing
- Token masking for security
- Clear indication of dry-run mode
- Proper handling of SCP-style SSH URLs
tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden (1)
1-206: LGTM! The test snapshot demonstrates robust error handling and security.The snapshot effectively captures:
- Detailed file inclusion logs
- Proper masking of sensitive credentials
- Clear error messages with stack traces
- Accurate summary of successful and failed operations
internal/exec/vendor_model_component.go (2)
13-13: LGTM! Added logging dependency.The charmbracelet/log package is a good choice for structured logging.
70-81: LGTM! Enhanced dry-run flow with improved detection.The changes improve the dry-run functionality by:
- Adding detailed logging
- Using CustomGitDetector for URL validation
- Providing clear error messages
internal/exec/vendor_model.go (2)
15-15: LGTM! Added logging dependency.The charmbracelet/log package is a good choice for structured logging.
244-256: LGTM! Enhanced dry-run flow with improved logging and detection.The changes improve the vendoring process by:
- Adding debug logging for package operations
- Using CustomGitDetector for URL validation
- Providing clear error messages
tests/fixtures/scenarios/vendoring-dry-run/components/terraform/ipinfo/component.yaml (1)
1-17: Component Vendor Configuration is Well-Structured
This new configuration file for the ipinfo component is neatly organized and follows the expected schema. The SCP-style URI is properly formatted for detection in the new logic.tests/test-cases/demo-vendoring.yaml (3)
43-60: SSH Vendor Pull Test Case Looks Solid
This test case aptly verifies the dry-run vendoring with an SSH-style URL. It correctly configures logging, dry-run mode, and token injection. Good job!
61-79: Component-Specific Vendoring Test is Clear
The dry-run test for the ipinfo component is well-defined. It ensures that component vendoring parameters are correctly processed in dry-run mode.
80-100: Credentials Leakage Prevention Test is Effective
This test case ensures that sensitive tokens are properly masked in the logs and not leaked. The expectations for both stderr and stdout are clearly stated.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 96-96: trailing spaces
(trailing-spaces)
[error] 98-98: trailing spaces
(trailing-spaces)
… only in dry mode
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (2)
9-18: Basic HTTPS Source Configuration Review
The first source entry clearly documents a basic HTTPS URL (with token injection expected), and the structure is correct. Note that line 18 has trailing spaces that should be removed to satisfy YAML lint rules.- # Basic HTTPS default (token injection expected)␣␣ + # Basic HTTPS default (token injection expected)🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 18-18: trailing spaces
(trailing-spaces)
19-27: Direct Credentials URL Configuration Caution
The second source entry uses direct credentials in the URL. While this may be intentional for testing or specific scenarios, please verify that exposing credentials in configuration is acceptable for your environment.tests/test-cases/demo-vendoring.yaml (2)
61-79: Environment Variable Formatting Consistency
The "atmos vendor pull ssh component" test case is sound in its intent. However, notice that theATMOS_GITHUB_TOKENvalue is provided without quotes here while quoted in the SSH test case. For maintainability and consistency, consider using the same quoting style for environment variables across test cases.
96-96: Remove Trailing Whitespace
Static analysis has detected trailing spaces on these lines. Removing these extraneous spaces helps maintain clean YAML formatting.- - "Injecting token" + - "Injecting token" - - "!supersecret" + - "!supersecret"Also applies to: 98-98
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 96-96: trailing spaces
(trailing-spaces)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
internal/exec/vendor_utils.go(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden(1 hunks)tests/test-cases/demo-vendoring.yaml(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- internal/exec/vendor_utils.go
🚧 Files skipped from review as they are similar to previous changes (2)
- tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden
- tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden
🧰 Additional context used
🪛 YAMLlint (1.35.1)
tests/test-cases/demo-vendoring.yaml
[error] 96-96: trailing spaces
(trailing-spaces)
[error] 98-98: trailing spaces
(trailing-spaces)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml
[error] 18-18: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: [localstack] demo-localstack
- GitHub Check: Acceptance Tests (macos-latest, macos)
- GitHub Check: Acceptance Tests (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (4)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (2)
1-8: File Header Configuration Looks Good
The top-level keys such asapiVersion,kind, andmetadata(withnameanddescription) are clearly defined and follow the expected schema for an Atmos vendoring manifest.
28-36: SSH Implicit Source Configuration Validated
The third source entry correctly represents an SSH implicit method, aligning with the enhanced SCP and SSH URL support highlighted in the PR objectives. The structure is consistent with the other components, ensuring that the custom detectors in the codebase can correctly process these URLs.tests/test-cases/demo-vendoring.yaml (2)
43-60: SSH Dry-Run Test Case Validation
This test case correctly exercises the dry-run mode for vendor pulls using an SSH-style URL. The use of detailed logging (via--logs-level=Debug) and the dry-run flag ensures that no actual file downloads occur. The work directory and environment variable setup align with the PR objectives.
80-101: Credential Masking Verification
This test case does an excellent job ensuring that injected credentials do not appear in log outputs by checking for a masked output (e.g., "!supersecret") and an indicative log message ("Injecting token"). The expected stdout and stderr values properly cover the security requirement regarding credential leakage.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 96-96: trailing spaces
(trailing-spaces)
[error] 98-98: trailing spaces
(trailing-spaces)
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
tests/test-cases/demo-vendoring.yaml (2)
61-79: Review of SSH Component Test Case:
This component-specific test case is clear and concise. One small nitpick—consider using consistent quoting for the environment variable value. In the previous test case,"supersecret"is quoted, whereas here it’s unquoted. Ensuring consistency in YAML can avoid potential ambiguity.
80-100: Review of Custom Detector Credentials Leakage Test Case:
This test case is thoughtfully designed to ensure that injected credentials are masked properly in logs. A couple of points to note:
- The test is currently disabled (
enabled: false). If credential leakage validation is a priority, please remember to enable it once the underlying functionality is fully ready.- Static analysis has flagged trailing spaces on lines 96 and 98. Removing these extra spaces will satisfy YAMLlint and keep the file tidy.
Below is a suggested diff to remove the trailing spaces:
- - "Injecting token" + - "Injecting token" - - "!supersecret" + - "!supersecret"🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 96-96: trailing spaces
(trailing-spaces)
[error] 98-98: trailing spaces
(trailing-spaces)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/test-cases/demo-vendoring.yaml(1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
tests/test-cases/demo-vendoring.yaml
[error] 96-96: trailing spaces
(trailing-spaces)
[error] 98-98: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: website-deploy-preview
- GitHub Check: Analyze (go)
- GitHub Check: Summary
🔇 Additional comments (1)
tests/test-cases/demo-vendoring.yaml (1)
43-60: Review of SSH Dry-Run Test Case:
The new test case for SSH-style URL dry-run is well configured. The arguments correctly include the--logs-level=Debugand--dry-runflags, and the environment variableATMOS_GITHUB_TOKENis properly defined. Just ensure that theworkdirpath ("fixtures/scenarios/vendor-pulls-ssh") accurately reflects the intended test scenario.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (8)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
17-17: Add newline at end of file.Add a trailing newline to the end of the file to follow YAML best practices.
tags: - demo +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 17-17: no new line character at the end of file
(new-line-at-end-of-file)
tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml (3)
1-1: Remove extra blank line at start of file.Remove the extra blank line at the beginning of the file.
- base_path: "./"🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
13-13: Remove trailing whitespace.Remove the trailing whitespace at the end of this line.
- auto_generate_backend_file: false - + auto_generate_backend_file: false🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 13-13: trailing spaces
(trailing-spaces)
21-22: Fix file ending format.Clean up the end of the file by removing excessive blank lines and ensuring there's exactly one newline at the end.
stacks: base_path: "stacks" included_paths: - "deploy/**/*" excluded_paths: - "**/_defaults.yaml" name_pattern: "{stage}" - -22 +🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 21-21: too many blank lines
(1 > 0) (empty-lines)
tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml (2)
18-18: Remove trailing whitespace.Remove trailing spaces at the end of line 18.
- +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 18-18: trailing spaces
(trailing-spaces)
35-35: Add newline at end of file.Add a trailing newline to the end of the file to follow YAML best practices.
- demo +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
tests/test-cases/demo-vendoring.yaml (2)
96-96: Remove trailing whitespace.Remove trailing spaces at the end of line 96.
- - "!supersecret" + - "!supersecret"🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 96-96: trailing spaces
(trailing-spaces)
98-98: Add newline at end of file.Add a trailing newline to the end of the file to follow YAML best practices.
exit_code: 0 +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 98-98: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
internal/exec/go_getter_utils.go(2 hunks)tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_with_custom_detector_and_handling_credentials_leakage.stderr.golden(1 hunks)tests/test-cases/demo-vendoring.yaml(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- tests/snapshots/TestCLICommands_atmos_vendor_pull_with_custom_detector_and_handling_credentials_leakage.stderr.golden
🚧 Files skipped from review as they are similar to previous changes (2)
- tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden
- tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden
🧰 Additional context used
🧠 Learnings (1)
internal/exec/go_getter_utils.go (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: internal/exec/go_getter_utils.go:74-75
Timestamp: 2025-02-13T07:30:28.946Z
Learning: In the `CustomGitDetector.Detect` method of `internal/exec/go_getter_utils.go`, verbose debug logging of raw URLs is intentionally kept for debugging purposes, despite potential credential exposure risks.
🪛 YAMLlint (1.35.1)
tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml
[error] 18-18: trailing spaces
(trailing-spaces)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
tests/test-cases/demo-vendoring.yaml
[error] 96-96: trailing spaces
(trailing-spaces)
[error] 98-98: no new line character at the end of file
(new-line-at-end-of-file)
tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
[warning] 21-21: too many blank lines
(1 > 0) (empty-lines)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml
[error] 17-17: no new line character at the end of file
(new-line-at-end-of-file)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (14)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
11-17: SSH URL format is appropriate.The source URL correctly uses the SCP-style SSH format (
[email protected]:cloudposse/terraform-null-label.git) which is a legitimate SSH URL format for Git repositories.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 17-17: no new line character at the end of file
(new-line-at-end-of-file)
tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml (3)
10-17: Good configuration for token injection testing.This component setup properly tests the basic HTTPS case where token injection should occur.
19-27: Good configuration for handling embedded credentials.This case properly tests direct credential handling, where a URL with embedded credentials should not have tokens injected.
28-35: Good configuration for pre-existing auth testing.This setup correctly tests the case where a URL has pre-existing auth info that should be preserved.
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
tests/test-cases/demo-vendoring.yaml (2)
43-60: Test case looks good for SSH vendor pull.This test case correctly verifies the dry-run functionality for SSH-style URLs.
61-79: Test case looks good for component-specific SSH vendor pull.This test case properly verifies component-specific vendoring with SSH URLs.
internal/exec/go_getter_utils.go (8)
66-68: Good refactoring of detector type.Renaming from
CustomGitHubDetectortoCustomGitDetectorbetter reflects its expanded functionality supporting multiple Git hosting services.
74-116: Enhanced detection logic handles SCP-style URLs well.The refactored
Detectmethod now properly handles SCP-style URLs, normalizes paths, and manages token injection in a more robust way. The code is well-structured and has good logging.
142-158: Good implementation of ensureScheme.This method correctly handles SSH URL transformation and adds default HTTPS scheme when needed. It also properly masks credentials in logs.
160-180: Well-implemented SCP URL rewriting.The
rewriteSCPURLfunction correctly transforms SCP-style URLs (user@host:path) to proper SSH URLs (ssh://user@host/path), making them compatible with standard URL parsing.
192-203: Secure token injection.This method handles token injection securely and includes appropriate debug logging with masked credentials.
205-237: Good token resolution for multiple hosting services.The token resolution logic now supports GitHub, Bitbucket, and GitLab, with fallbacks between different environment variable names.
239-259: Comprehensive username handling for different services.The function correctly provides appropriate default usernames for different Git hosting services, including special handling for Bitbucket.
261-272: Good implementation of subdirectory adjustment.The method correctly identifies when a subdirectory needs to be appended to the repository path.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
17-17: Fix typos in the comment.There are two typos in this comment line.
- ## Explicit ssh vednoring (the schema is explicitly spcified along with a username, no custom detector is invoked) + ## Explicit ssh vendoring (the schema is explicitly specified along with a username, no custom detector is invoked)tests/test-cases/vendoring-ssh-dryrun.yaml (1)
41-59: Good security test for credential leakage.This test case crucially verifies that sensitive credentials are never leaked in logs, which is vital for security.
There are minor formatting issues to fix:
- - "!supersecret" + - "!supersecret" - "!ATMOS_GITHUB_TOKEN" +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
internal/exec/go_getter_utils.go (1)
208-240: Extract string constants for Git hosting services.The host names are repeated multiple times in the codebase and should be extracted as named constants.
+// Constants for Git hosting services +const ( + GitHostGitHub = "github.com" + GitHostGitLab = "gitlab.com" + GitHostBitbucket = "bitbucket.org" +) func (d *CustomGitDetector) resolveToken(host string) (string, string) { var token, tokenSource string switch host { - case "github.com": + case GitHostGitHub: if d.AtmosConfig.Settings.InjectGithubToken { tokenSource = "ATMOS_GITHUB_TOKEN" token = os.Getenv(tokenSource) if token == "" { tokenSource = "GITHUB_TOKEN" token = os.Getenv(tokenSource) } } else { tokenSource = "GITHUB_TOKEN" token = os.Getenv(tokenSource) } - case "bitbucket.org": + case GitHostBitbucket: tokenSource = "BITBUCKET_TOKEN" token = os.Getenv(tokenSource) if token == "" { tokenSource = "ATMOS_BITBUCKET_TOKEN" token = os.Getenv(tokenSource) } - case "gitlab.com": + case GitHostGitLab: tokenSource = "GITLAB_TOKEN" token = os.Getenv(tokenSource) if token == "" { tokenSource = "ATMOS_GITLAB_TOKEN" token = os.Getenv(tokenSource) } } return token, tokenSource }Update the other occurrences of these strings throughout the file as well.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
internal/exec/go_getter_utils.go(2 hunks)pkg/utils/url_utils.go(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden(1 hunks)tests/test-cases/demo-vendoring.yaml(1 hunks)tests/test-cases/vendoring-ssh-dryrun.yaml(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- tests/test-cases/demo-vendoring.yaml
🚧 Files skipped from review as they are similar to previous changes (3)
- pkg/utils/url_utils.go
- tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden
- tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden
🧰 Additional context used
🧠 Learnings (1)
internal/exec/go_getter_utils.go (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: internal/exec/go_getter_utils.go:74-75
Timestamp: 2025-02-13T07:30:28.946Z
Learning: In the `CustomGitDetector.Detect` method of `internal/exec/go_getter_utils.go`, verbose debug logging of raw URLs is intentionally kept for debugging purposes, despite potential credential exposure risks.
🪛 YAMLlint (1.35.1)
tests/test-cases/vendoring-ssh-dryrun.yaml
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
🪛 GitHub Check: golangci
internal/exec/go_getter_utils.go
[failure] 245-245: [golangci] internal/exec/go_getter_utils.go#L245
add-constant: string literal "github.com" appears, at least, 4 times, create a named constant for it (revive)
[failure] 247-247: [golangci] internal/exec/go_getter_utils.go#L247
add-constant: string literal "gitlab.com" appears, at least, 4 times, create a named constant for it (revive)
[failure] 249-249: [golangci] internal/exec/go_getter_utils.go#L249
add-constant: string literal "bitbucket.org" appears, at least, 4 times, create a named constant for it (revive)
⏰ Context from checks skipped due to timeout of 90000ms (5)
- GitHub Check: Acceptance Tests (macos-latest, macos)
- GitHub Check: Acceptance Tests (windows-latest, windows)
- GitHub Check: Acceptance Tests (ubuntu-latest, linux)
- GitHub Check: [localstack] demo-localstack
- GitHub Check: Summary
🔇 Additional comments (11)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (2)
9-16: Confirms the use of proper SCP-style SSH URL for vendoring.The component correctly implements an SCP-style SSH URL ([email protected]:path/repo.git) which is a standard SSH format. This aligns with the PR objective of supporting SSH-style URLs in dry run mode.
18-24: Properly implements explicit SSH URL format.The second component correctly uses the explicit SSH protocol format (ssh://[email protected]/path/repo.git), providing good coverage of SSH URL patterns.
tests/test-cases/vendoring-ssh-dryrun.yaml (2)
3-20: Well-structured test case for SSH-style URL vendoring.This test case properly verifies the dry-run functionality with SSH URLs, ensuring token injection works correctly while maintaining security.
22-39: Component-specific vendoring test is appropriate.Test case validates component-specific vendoring alongside SSH support, which ensures both features work together correctly.
internal/exec/go_getter_utils.go (7)
66-71: Good refactoring to support multiple Git hosting services.Renamed from
CustomGitHubDetectortoCustomGitDetectorto reflect expanded functionality supporting multiple Git hosting services (GitHub, GitLab, Bitbucket).
74-124: Enhanced detection logic with improved URL handling.The refactored
Detectmethod now properly handles different URL formats including SCP-style URLs and adds detailed logging to aid in debugging during dry runs.
126-139: Good use of named constants for regex match indices.Using named constants improves code readability and maintainability.
141-158: Well-implemented scheme detection and SCP URL rewriting.The
ensureSchememethod correctly handles different URL formats and ensures proper scheme detection, with detailed logging of the transformations.
160-183: Comprehensive SCP URL parsing with regex.The regex pattern effectively captures all components of SCP-style URLs and the function properly reconstructs them into standard SSH URLs.
242-262: Good implementation of host-specific default usernames.The function correctly handles different authentication requirements for each Git hosting service, with special handling for Bitbucket.
🧰 Tools
🪛 GitHub Check: golangci
[failure] 245-245: [golangci] internal/exec/go_getter_utils.go#L245
add-constant: string literal "github.com" appears, at least, 4 times, create a named constant for it (revive)
[failure] 247-247: [golangci] internal/exec/go_getter_utils.go#L247
add-constant: string literal "gitlab.com" appears, at least, 4 times, create a named constant for it (revive)
[failure] 249-249: [golangci] internal/exec/go_getter_utils.go#L249
add-constant: string literal "bitbucket.org" appears, at least, 4 times, create a named constant for it (revive)
264-275: Path adjustment logic correctly handles subdirectories.The function ensures proper path handling for repository subdirectories, with helpful debug logging for path modifications.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
tests/test-cases/vendoring-ssh-dryrun.yaml (2)
57-57: Remove trailing whitespace.There's a trailing space at the end of line 57.
- - !not 'supersecret' + - !not 'supersecret'🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 57-57: trailing spaces
(trailing-spaces)
59-59: Add newline at end of file.Standard coding practice is to end files with a newline character.
- !not 'ATMOS_GITHUB_TOKEN' + exit_code: 0🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
internal/exec/go_getter_utils.go(2 hunks)tests/test-cases/vendoring-ssh-dryrun.yaml(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
internal/exec/go_getter_utils.go (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: internal/exec/go_getter_utils.go:74-75
Timestamp: 2025-02-13T07:30:28.946Z
Learning: In the `CustomGitDetector.Detect` method of `internal/exec/go_getter_utils.go`, verbose debug logging of raw URLs is intentionally kept for debugging purposes, despite potential credential exposure risks.
🪛 YAMLlint (1.35.1)
tests/test-cases/vendoring-ssh-dryrun.yaml
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (9)
tests/test-cases/vendoring-ssh-dryrun.yaml (1)
1-59: Well-structured test cases for SSH URL handling in dry run mode.The test cases nicely cover different scenarios:
- Basic vendor pull with SSH
- Component-specific vendoring with SSH
- Security testing to ensure credentials aren't leaked
The test configuration is clearly organized and aligns well with the improved functionality in the
CustomGitDetector.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
internal/exec/go_getter_utils.go (8)
66-68: Great improvement to generalize the detector.Renaming from
CustomGitHubDetectortoCustomGitDetectorcorrectly reflects the expanded scope to handle multiple Git hosting services.
74-124: The Detect method is well-structured with good URL handling.The refactored implementation properly handles different URL formats, including SCP-style URLs, and performs appropriate token injection. The masking of sensitive information in logs is a strong security practice.
126-139: Good use of named constants for clarity.Using named constants for regex match indices and hosts improves readability and maintainability.
144-161: Effective handling of SCP-style URLs.The
ensureSchememethod properly handles different URL formats, including converting SCP-style URLs to valid SSH URLs, which is central to the PR's objective.
163-186: Well-implemented SCP URL rewriting.The regex pattern and transformation logic correctly convert SCP-style URLs (like
[email protected]:org/repo.git) to proper SSH URLs (ssh://[email protected]/org/repo.git).
211-243: Comprehensive token resolution across services.Good implementation that checks multiple environment variables across different Git hosting services. This improves flexibility for users with different authentication setups.
245-265: Well-handled username selection for different hosts.The code correctly identifies the appropriate default username for authentication based on the Git hosting service.
267-278: Robust subdirectory handling for repository paths.The
adjustSubdirmethod ensures proper path handling, especially for top-level repositories without explicit subdirectories.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1076 +/- ##
==========================================
- Coverage 17.22% 17.09% -0.14%
==========================================
Files 169 170 +1
Lines 18736 18886 +150
==========================================
Hits 3228 3228
- Misses 14937 15087 +150
Partials 571 571
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
what
This PR upgrades the output of dry run mode for vendor pull command.
After the change is applied, the dry run mode shows
yet there's no actual files download.
Before ( component vendoring dry-run output)
After ( component vendoring dry-run output)
component.yaml, refer to the respective test case for details
Before ( generic vendoring dry-run output)
After ( generic vendoring dry-run output)
vendor.yaml, refer to the respective test case for detaails
why
Received feedback from in the parent PR
references
Upscopes from #1061 and
Summary by CodeRabbit
Summary by CodeRabbit
New Features
Documentation
Tests