Convert SCP-style URLs (no explicit scheme) into proper SSH URLs #1061
Conversation
📝 WalkthroughWalkthroughThe changes refactor Git URL handling by renaming and updating the Git detector type and its methods. They add support for multiple Git hosting services with enhanced logging and token injection, along with new helper functions for URL normalization and error handling. A new Git getter type is introduced to manage post-clone symlink removal. Additionally, new YAML configuration files and test cases improve the vendor pull process and extend documentation with environment variables for Bitbucket and GitLab. A utility function to mask basic auth credentials in URLs has also been added. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Detector as CustomGitDetector
participant Getter as CustomGitGetter
User->>Detector: Detect(src, _)
Detector->>Detector: Call ensureScheme & rewriteSCPURL
Detector->>Detector: Normalize URL path and check host
Detector->>Detector: Inject token if credentials absent
Detector-->>User: Return formatted URL, flag, error
alt Clone operation needed
User->>Getter: Get(dst, URL)
Getter->>Getter: Remove symlinks post clone
Getter-->>User: Return success or error
end
Possibly related PRs
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (5)
internal/exec/go_getter_utils.go (5)
15-15: Consider using a more descriptive alias
Short aliases likelmight hamper clarity. Renaming tologorloggercould improve readability.
70-70: Suggest clarifying the field name
sourcecould be named more explicitly, such assourceURIororiginalSource, for improved code clarity.
90-112: Add optional support for custom ports
SCP-style URLs sometimes specify a custom port (e.g.,git@host:port/repo). The current regex won’t match those.- scpPattern := regexp.MustCompile(`^(([\w.-]+)@)?([\w.-]+\.[\w.-]+):([\w./-]+)(\.git)?(.*)$`) + scpPattern := regexp.MustCompile(`^(([\w.-]+)@)?([\w.-]+\.[\w.-]+)(:[0-9]+)?:([\w./-]+)(\.git)?(.*)$`)
136-138: Convert TBC comment into a TODO
Documenting pending enhancements is good. Consider adding a// TODO:or opening an issue.Would you like me to open an issue for broadening token injection support?
139-164: Ensure consistent config toggles
Only GitHub token injection is governed byInjectGithubToken. Consider adding similar toggles for Bitbucket and GitLab for uniformity.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
internal/exec/go_getter_utils.go(3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (17)
- GitHub Check: [mock-macos] tests/fixtures/scenarios/complete
- GitHub Check: [mock-macos] examples/demo-vendoring
- GitHub Check: [mock-macos] examples/demo-context
- GitHub Check: [mock-macos] examples/demo-component-versions
- GitHub Check: [mock-macos] examples/demo-atlantis
- GitHub Check: [mock-windows] examples/demo-context
- GitHub Check: [mock-windows] examples/demo-component-versions
- GitHub Check: [mock-windows] examples/demo-atlantis
- GitHub Check: [mock-linux] tests/fixtures/scenarios/complete
- GitHub Check: [mock-linux] examples/demo-vendoring
- GitHub Check: [mock-linux] examples/demo-context
- GitHub Check: Acceptance Tests (windows-latest, windows)
- GitHub Check: Acceptance Tests (ubuntu-latest, linux)
- GitHub Check: Docker Lint
- GitHub Check: [k3s] demo-helmfile
- GitHub Check: [localstack] demo-localstack
- GitHub Check: Summary
🔇 Additional comments (11)
internal/exec/go_getter_utils.go (11)
11-11: Looks good
No immediate concerns. Theregexpimport is necessary for the new SCP-style detection.
61-61: Confirmed
Addinggit::sshto the list of valid schemes aligns with go-getter usage.
66-68: Renaming is consistent
This rename clarifies support for multiple Git hosting platforms.
81-88: Excellent documentation
The inline comments clearly explain how SCP-style URLs are handled.
115-115: Duplicate concern: potential credential exposure
Similar to the previous logging ofsrc, sensitive info may be leaked.
123-126: Clear error handling
Returning an error when no SSH agent is available is straightforward.
128-133: Non-standard host scenario well-handled
Skipping token injection for unrecognized hosts is logical.
166-175: Token injection logic is robust
Injecting the token only if credentials aren’t already present is correct.
177-185: Subdir detection
Appending//.for top-level repos is a known go-getter approach. Looks good.
187-197: Default shallow clone
Specifyingdepth=1improves performance but may break use cases needing full history. Confirm this suits your workflows.
208-208: Registering the new CustomGitDetector
Ensuring the custom detector runs first is correct.
|
To test vendoring of SSH style URLs, without SSH key, test instead |
|
Windows tests are failing: === RUN TestCLICommands/atmos_vendor_pull_(no_tty)
cli_test.go:901: Stderr diff mismatch for "D:\\a\\atmos\\atmos\\tests\\snapshots\\TestCLICommands_atmos_vendor_pull_(no_tty).stdout.golden":
--- expected
+++ actual
@@ -1,7 +1,12 @@
INFO Vendoring from 'vendor.yaml'
WARN No TTY detected. Falling back to basic output. This can happen when no terminal is attached or when commands are pipelined.
-INFO ✓ github/stargazers (main)
-INFO ✓ weather (main)
-INFO ✓ ipinfo (main)
+ERRO Failed to vendor github/stargazers: error : failed to download package: subdir "examples%5Cdemo-library%5Cgithub%5Cstargazers" not found
+INFO x github/stargazers (main)
+ERRO Failed to vendor weather: error : failed to download package: subdir "examples%5Cdemo-library%5Cweather" not found
+INFO x weather (main)
+ERRO Failed to vendor ipinfo: error : failed to download package: subdir "examples%5Cdemo-library%5Cipinfo" not found
+INFO x ipinfo (main)
+INFO Vendored 0 components. Failed to vendor 3 components.
+ |
Have added the test, but it is in disabled state now as current dry run implementation in internal\exec\vendor_component_utils.go exits before hitting custom detectors and go getters. Let me know there are any ojectiosn to adjust vendor_component_utils.go code for the dry run to operate differently. |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tests/test-cases/demo-vendoring.yaml (1)
58-59: Clean up YAML formatting.Fix the following formatting issues:
- Line 58: Remove trailing spaces
- Line 59: Add newline at end of file
- - "No SSH authentication method found" - exit_code: 0 \ No newline at end of file + - "No SSH authentication method found" + exit_code: 0 +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 58-58: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
internal/exec/go_getter_utils.go(3 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden(1 hunks)tests/test-cases/demo-vendoring.yaml(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- internal/exec/go_getter_utils.go
🧰 Additional context used
🪛 YAMLlint (1.35.1)
tests/test-cases/demo-vendoring.yaml
[error] 58-58: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (1)
tests/test-cases/demo-vendoring.yaml (1)
43-59: LGTM! Well-structured test case for SSH URL handling.The test case effectively validates:
- SSH-style URL handling
- Dry-run behavior
- Error messaging for missing SSH authentication
- Debug logging output
This aligns perfectly with the PR objectives for SCP-style URL support.
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 58-58: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
Done |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (2)
9-17: SSH Vendoring Source Block VerificationThe SSH source URL
"git::[email protected]:cloudposse/terraform-null-label.git?ref={{ .Version }}"and its related fields (component, version, targets, and tags) are well-defined in this fixture. Given the PR’s objective to properly convert SCP-style URLs into fully qualified SSH URLs, please ensure that the URL conversion logic within the application treats this pattern (and similar edge cases) correctly. Additionally, consider expanding your test suite with varied SCP-style URL scenarios to further validate the conversion process.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 17-17: no new line character at the end of file
(new-line-at-end-of-file)
17-17: File Formatting: Newline at End-of-FileYAMLlint has flagged that there is no newline at the end of the file. Adding a newline at the end will ensure compliance with common formatting standards.
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 17-17: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml:20-22
Timestamp: 2025-02-18T15:20:49.080Z
Learning: Hardcoded credentials are acceptable in test fixtures when they are specifically testing credential handling, masking, or injection behavior. For example, in `tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml`, credentials like "myuser:supersecret" are used to test that direct credentials in URLs are not overwritten by token injection.
🪛 YAMLlint (1.35.1)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml
[error] 17-17: no new line character at the end of file
(new-line-at-end-of-file)
⏰ Context from checks skipped due to timeout of 90000ms (3)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Analyze (go)
- GitHub Check: Summary
🔇 Additional comments (1)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
1-8: Configuration Metadata Clarity and ConsistencyThe metadata and basic configuration structure are clear and consistent. The definition of the API version, kind, metadata, and an explicitly empty
importsarray enhance readability and maintainability.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
internal/exec/go_getter_utils.go (4)
97-99: Consider early return for unsupported hosts.The code logs when a host is unsupported, but continues execution. For clarity, consider returning early if token injection isn't supported.
if host != hostGitHub && host != hostBitbucket && host != hostGitLab { log.Debug("Skipping token injection for a unsupported host", "host", parsedURL.Host) + // Continue without token injection }
116-121: Error handling improvement needed.The error from
MaskBasicAuthis logged but doesn't affect execution flow. Consider returning the error or implementing a fallback mechanism.maskedFinal, err := u.MaskBasicAuth(strings.TrimPrefix(finalURL, "git::")) if err != nil { log.Debug("Masking failed", "error", err) + // Consider safer alternatives here, like not logging the URL at all } else { log.Debug("Final transformation", "url", "git::"+maskedFinal) }
211-243: Inconsistent token resolution order across hosts.For GitHub, ATMOS_GITHUB_TOKEN is checked first when InjectGithubToken is true, but for Bitbucket and GitLab, the non-prefixed version is checked first. Consider standardizing this approach.
func (d *CustomGitDetector) resolveToken(host string) (string, string) { var token, tokenSource string switch host { case hostGitHub: if d.AtmosConfig.Settings.InjectGithubToken { tokenSource = "ATMOS_GITHUB_TOKEN" token = os.Getenv(tokenSource) if token == "" { tokenSource = "GITHUB_TOKEN" token = os.Getenv(tokenSource) } } else { tokenSource = "GITHUB_TOKEN" token = os.Getenv(tokenSource) } case hostBitbucket: - tokenSource = "BITBUCKET_TOKEN" + tokenSource = "ATMOS_BITBUCKET_TOKEN" token = os.Getenv(tokenSource) if token == "" { - tokenSource = "ATMOS_BITBUCKET_TOKEN" + tokenSource = "BITBUCKET_TOKEN" token = os.Getenv(tokenSource) } case hostGitLab: - tokenSource = "GITLAB_TOKEN" + tokenSource = "ATMOS_GITLAB_TOKEN" token = os.Getenv(tokenSource) if token == "" { - tokenSource = "ATMOS_GITLAB_TOKEN" + tokenSource = "GITLAB_TOKEN" token = os.Getenv(tokenSource) } } return token, tokenSource }
330-343: Consider making symlink removal configurable.The current implementation unconditionally removes all symlinks from cloned repositories. This could break repositories that rely on symlinks for their structure.
Consider adding a configuration option to control this behavior:
type CustomGitGetter struct { getter.GitGetter + RemoveSymlinks bool } func (c *CustomGitGetter) Get(dst string, url *url.URL) error { // Normal clone if err := c.GitGetter.Get(dst, url); err != nil { return err } // Remove symlinks + if c.RemoveSymlinks { return removeSymlinks(dst) + } + return nil }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
internal/exec/go_getter_utils.go(4 hunks)
🧰 Additional context used
🧠 Learnings (1)
internal/exec/go_getter_utils.go (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: internal/exec/go_getter_utils.go:74-75
Timestamp: 2025-02-13T07:30:28.946Z
Learning: In the `CustomGitDetector.Detect` method of `internal/exec/go_getter_utils.go`, verbose debug logging of raw URLs is intentionally kept for debugging purposes, despite potential credential exposure risks.
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (5)
internal/exec/go_getter_utils.go (5)
73-76: Log entry now more secure.The improved log entry no longer includes the raw URL, which is good for security. The detailed URL logging is moved to after the URL is properly masked.
81-83: Good addition of explicit scheme handling.Adding the
ensureSchememethod effectively addresses the core issue of handling SCP-style URLs.
143-161: Good implementation of SCP-style URL conversion.The
ensureSchememethod properly handles URLs without explicit schemes by first trying to interpret them as SCP-style, and defaulting to HTTPS if that fails.
247-265: Inconsistent default username handling.The code has different approach for Bitbucket compared to other hosts. Consider standardizing or documenting why this difference exists.
Is there a specific reason why Bitbucket allows configuring the username via environment variables while GitHub and GitLab use fixed values?
311-320: Good implementation of custom getters.The map of getters with overrides for the
gitgetter is a clean approach to extending functionality.
|
Please document the ways to specify vendoring sources for SSH |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (8)
tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml (3)
1-1: Blank Lines Warning: Remove Extra Blank Lines.
There appear to be excess blank lines at the very start of the file. Consider removing them to keep the YAML neat and consistent.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
13-13: Trailing Whitespace: Remove Unnecessary Spaces.
Trailing whitespace detected on this line; please remove the extra spaces to satisfy linting standards.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 13-13: trailing spaces
(trailing-spaces)
21-21: Blank Lines Warning: Extra Blank Line Detected.
A superfluous blank line is present here—removing it will improve the overall formatting.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 21-21: too many blank lines
(1 > 0) (empty-lines)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
17-18: Typographical Issues in Comments.
Consider correcting the spelling in the comment: change "vednoring" to "vendoring" and "spcified" to "specified" for clarity.tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml (2)
18-18: Trailing Whitespace Detected.
There's trailing space on this line; please remove the extra spaces to comply with YAML standards.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 18-18: trailing spaces
(trailing-spaces)
35-35: Missing Newline at End-of-File.
Adding a newline at the end will ensure POSIX compliance and cleaner diffs.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
tests/test-cases/vendoring-ssh-dryrun.yaml (2)
57-57: Trailing Whitespace Detected.
There is trailing whitespace on this line. Removing it will improve consistency.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 57-57: trailing spaces
(trailing-spaces)
59-59: Missing Newline at EOF.
Please add a newline character at the end of the file to adhere to best practices.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
pkg/utils/url_utils.go(1 hunks)tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden(1 hunks)tests/test-cases/vendoring-ssh-dryrun.yaml(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- pkg/utils/url_utils.go
- tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden
🧰 Additional context used
🧠 Learnings (1)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml:20-22
Timestamp: 2025-02-18T15:20:49.080Z
Learning: Hardcoded credentials are acceptable in test fixtures when they are specifically testing credential handling, masking, or injection behavior. For example, in `tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml`, credentials like "myuser:supersecret" are used to test that direct credentials in URLs are not overwritten by token injection.
🪛 YAMLlint (1.35.1)
tests/test-cases/vendoring-ssh-dryrun.yaml
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
[warning] 21-21: too many blank lines
(1 > 0) (empty-lines)
tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml
[error] 18-18: trailing spaces
(trailing-spaces)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (6)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
1-25: Configuration Structure is Well-Formed.
The schema for both implicit and explicit SSH vendoring is clearly defined and aligns nicely with the expected behavior.tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml (2)
1-35: Vendor Configuration is Comprehensive and Clear.
This configuration cleanly distinguishes between token-injected HTTPS sources, direct credential usage, and pre-existing credentials, providing robust test coverage for credential handling.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 18-18: trailing spaces
(trailing-spaces)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
20-21: Direct Credentials in URL: Confirm Test Fixture Intent.
The hardcoded credentials in the URL are intentional for verifying credential masking behavior and token injection skipping. This is acceptable in a test fixture context.tests/test-cases/vendoring-ssh-dryrun.yaml (3)
4-21: SSH Dry-Run Test Case Validated.
The "atmos vendor pull using SSH" test is well-defined with appropriate dry-run parameters and expected outputs. Everything looks in order here.
22-40: Component Vendor Pull Test is Configured Well.
The test for pulling a specific component using SSH is clear and concise, with proper arguments and expectations.
41-59: Credentials Leakage Test: Verification of Sensitive Data Handling.
This test case effectively checks that sensitive credentials aren’t leaked in logs. Please double-check that the assertions (ensuring absence of "supersecret" and "ATMOS_GITHUB_TOKEN") are reliably capturing the sensitive data removal.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
tests/test-cases/vendoring-ssh-dryrun.yaml (3)
22-40: Test Case "atmos vendor pull with custom detector and handling credentials leakage" – Configuration Review.
The test is effective in ensuring that sensitive credentials (the token) are not inadvertently logged. For clarity and consistency with the previous test case, consider quoting the token value (e.g.,"supersecret") in the environment variable.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 38-38: trailing spaces
(trailing-spaces)
[error] 40-40: no new line character at the end of file
(new-line-at-end-of-file)
38-38: Remove Trailing Whitespace on Line 38.
There are extra spaces at the end of line 38 as reported by YAML lint. Please remove these trailing spaces.- - !not 'supersecret' + - !not 'supersecret'🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 38-38: trailing spaces
(trailing-spaces)
40-40: Ensure File Ends With a Newline Character.
YAML lint has flagged that the file does not end with a newline character. Please add a newline at the end of the file to improve file formatting.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 40-40: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/test-cases/vendoring-ssh-dryrun.yaml(1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
tests/test-cases/vendoring-ssh-dryrun.yaml
[error] 38-38: trailing spaces
(trailing-spaces)
[error] 40-40: no new line character at the end of file
(new-line-at-end-of-file)
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Analyze (go)
- GitHub Check: website-deploy-preview
- GitHub Check: Summary
🔇 Additional comments (1)
tests/test-cases/vendoring-ssh-dryrun.yaml (1)
4-20: Test Case "atmos vendor pull using SSH" is well configured.
This test case clearly simulates a dry-run for vendoring via an SSH-style URL. The work directory, command, args, environment, and expected outcome (an empty diff and exit code 0) are all appropriately specified.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
website/docs/cli/commands/vendor/vendor-pull.mdx (4)
121-141: Well-documented URL Rewriting and Explicit SSH UsageThe documentation effectively demonstrates how Atmos rewrites a non-schemed Git URL into a fully qualified SSH URL and then handles explicit SSH schemes. One minor suggestion: consider using typographic quotation marks (e.g., “git” instead of "git") when referring to default usernames for enhanced readability.
🧰 Tools
🪛 LanguageTool
[typographical] ~130-~130: Consider using typographic quotation marks here.
Context: ...case, Atmos injects a default username ("git"). ### Explicit SSH When the SSH schem...(EN_QUOTES)
156-169: HTTPS Vendoring Section ClarityThis part explains how Atmos transforms ambiguous repository names into HTTPS URLs with excellent clarity. The examples provided are practical and easy to follow. As a minor stylistic tweak, you might consider using a typographical ellipsis in the "full HTTPS URL" example for a polished touch.
170-200: Comprehensive Authentication & Token Usage ExplanationThe authentication section does a great job of outlining the priority and usage of environment variables for GitHub, Bitbucket, and GitLab. For better presentation and consistency, consider adopting typographic quotes (e.g., “x-access-token” and “x-token-auth”) when specifying default usernames.
🧰 Tools
🪛 LanguageTool
[typographical] ~188-~188: Consider using a typographic close quote here.
Context: ...e and does not accept dummy values like"x-access-token". | **Default Username ...(EN_QUOTES)
[typographical] ~193-~193: Consider using a typographic close quote here.
Context: ...s set, it is used. - If neither is set,"x-token-auth"is used. ### GitLab | En...(EN_QUOTES)
203-213: Refine HTTPS URL Resolution RulesThe step-by-step rules for resolving HTTPS URLs are logically presented and provide valuable guidance. A little refinement could be made by varying the sentence beginnings in the list to avoid a repetitive "If" at the start of several points, and, if possible, using a typographical ellipsis for consistency in the examples.
🧰 Tools
🪛 LanguageTool
[style] ~205-~205: Consider using a different verb to strengthen your wording.
Context: ...en resolving HTTPS-based sources, Atmos follows these rules: 1. If a *full HTTPS URL...(FOLLOW_OBEY)
[style] ~207-~207: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. 2. If a **reposito...(ELLIPSIS)
[style] ~209-~209: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...po.git), it defaults tohttps://`. 3. If a username and repo name are provid...(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
website/docs/cli/commands/vendor/vendor-pull.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
website/docs/cli/commands/vendor/vendor-pull.mdx
[typographical] ~130-~130: Consider using typographic quotation marks here.
Context: ...case, Atmos injects a default username ("git"). ### Explicit SSH When the SSH schem...
(EN_QUOTES)
[typographical] ~188-~188: Consider using a typographic close quote here.
Context: ...e and does not accept dummy values like "x-access-token". | **Default Username ...
(EN_QUOTES)
[typographical] ~193-~193: Consider using a typographic close quote here.
Context: ...s set, it is used. - If neither is set, "x-token-auth" is used. ### GitLab | En...
(EN_QUOTES)
[style] ~205-~205: Consider using a different verb to strengthen your wording.
Context: ...en resolving HTTPS-based sources, Atmos follows these rules: 1. If a *full HTTPS URL...
(FOLLOW_OBEY)
[style] ~207-~207: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. 2. If a **reposito...
(ELLIPSIS)
[style] ~209-~209: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...po.git), it defaults to https://`. 3. If a username and repo name are provid...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
⏰ Context from checks skipped due to timeout of 90000ms (6)
- GitHub Check: Build (macos-latest, macos)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Build (ubuntu-latest, linux)
- GitHub Check: Analyze (go)
- GitHub Check: website-deploy-preview
- GitHub Check: Summary
🔇 Additional comments (3)
website/docs/cli/commands/vendor/vendor-pull.mdx (3)
111-120: Clear Introduction for SSH VendoringThis section clearly introduces SSH-based vendoring, explaining how non-public Git repositories can be accessed without interactive prompts. The implicit SSH example is straightforward and helpful.
142-155: Important Notes on URL Format ValidityThe guidelines for valid versus invalid URL formats are detailed and supported with clear code examples. This will help users avoid common pitfalls when dealing with SCP-style URLs. The formatting within the code blocks looks consistent; great work here.
214-255: Clear and Detailed Additional DocumentationThe latter portion of the document—including configuration notes, examples, and the flags table—is very well organized and aligns with the overall style. These details clearly aid in understanding the command's functionality.
🧰 Tools
🪛 LanguageTool
[grammar] ~239-~239: The word ‘vendor’ is a noun or an adjective. A verb or adverb is missing or misspelled here, or maybe a comma is missing.
Context: ...componentis not specified, Atmos will vendor all the artifacts defined in thevendo...(PRP_MD_NN)
[style] ~240-~240: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...efined in thevendor.yamlmanifest. If the flag--componentis passed in, At...(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[grammar] ~240-~240: The word ‘vendor’ is a noun or an adjective. A verb or adverb is missing or misspelled here, or maybe a comma is missing.
Context: ...--componentis passed in, Atmos will vendor only that component - Ifvendor.yaml...(PRP_MD_NN)
[grammar] ~240-~240: Please add a punctuation mark at the end of paragraph.
Context: ... passed in, Atmos will vendor only that component - Ifvendor.yamlis not found, Atmos...(PUNCTUATION_PARAGRAPH_END)
[style] ~242-~242: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...mlmanifest in the component's folder. Ifcomponent.yaml` is not found, an err...(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[grammar] ~243-~243: Please add a punctuation mark at the end of paragraph.
Context: ... flag--componentis required in this case ::: ## Flags | Flag | Descr...(PUNCTUATION_PARAGRAPH_END)
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
website/docs/cli/commands/vendor/vendor-pull.mdx (3)
116-122: Clarification on Implicit SSH URL Rewriting
The "Implicit SSH" subsection neatly explains that when no explicit scheme is provided, Atmos rewrites the URL (e.g. convertinggit::[email protected]:cloudposse/terraform-null-label.git?ref={{.Version}}into an SSH URL with depth and ref parameters). For improved readability, consider using typographic quotation marks for inline values (e.g. changing"git"to “git”).
170-202: Authentication & Token Usage Details
The section outlining token usage for GitHub, Bitbucket, and GitLab is comprehensive and well-organized. The tables clearly indicate which environment variables take precedence and what default usernames are used. As a stylistic improvement, consider adopting typographic quotation marks for credential values (for example, replacing"x-access-token"and"x-token-auth"with “x-access-token” and “x-token-auth”) to enhance readability.🧰 Tools
🪛 LanguageTool
[typographical] ~188-~188: Consider using a typographic close quote here.
Context: ...e and does not accept dummy values like"x-access-token". | **Default Username ...(EN_QUOTES)
[typographical] ~193-~193: Consider using a typographic close quote here.
Context: ...s set, it is used. - If neither is set,"x-token-auth"is used. ### GitLab | En...(EN_QUOTES)
203-211: Resolution Rules for HTTPS URLs
The rules for resolving HTTPS-based sources are clearly enumerated. A couple of minor style suggestions:
- Consider using a typographical ellipsis ( … ) in the examples instead of the three-dot notation.
- Rephrase the successive rule sentences to avoid starting each with the same word for a smoother reading experience.
🧰 Tools
🪛 LanguageTool
[style] ~205-~205: Consider using a different verb to strengthen your wording.
Context: ...en resolving HTTPS-based sources, Atmos follows these rules: 1. If a *full HTTPS URL...(FOLLOW_OBEY)
[style] ~207-~207: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. 2. If a **reposito...(ELLIPSIS)
[style] ~209-~209: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...po.git), it defaults tohttps://`. 3. If a username and repo name are provid...(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
website/docs/cli/commands/vendor/vendor-pull.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
website/docs/cli/commands/vendor/vendor-pull.mdx
[typographical] ~130-~130: Consider using typographic quotation marks here.
Context: ...case, Atmos injects a default username ("git"). ### Explicit SSH When the SSH schem...
(EN_QUOTES)
[typographical] ~188-~188: Consider using a typographic close quote here.
Context: ...e and does not accept dummy values like "x-access-token". | **Default Username ...
(EN_QUOTES)
[typographical] ~193-~193: Consider using a typographic close quote here.
Context: ...s set, it is used. - If neither is set, "x-token-auth" is used. ### GitLab | En...
(EN_QUOTES)
[style] ~205-~205: Consider using a different verb to strengthen your wording.
Context: ...en resolving HTTPS-based sources, Atmos follows these rules: 1. If a *full HTTPS URL...
(FOLLOW_OBEY)
[style] ~207-~207: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. 2. If a **reposito...
(ELLIPSIS)
[style] ~209-~209: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...po.git), it defaults to https://`. 3. If a username and repo name are provid...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
⏰ Context from checks skipped due to timeout of 90000ms (6)
- GitHub Check: Analyze (javascript-typescript)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: website-deploy-preview
- GitHub Check: Build (ubuntu-latest, linux)
- GitHub Check: Analyze (go)
- GitHub Check: Summary
🔇 Additional comments (7)
website/docs/cli/commands/vendor/vendor-pull.mdx (7)
111-115: New "Vendoring from SSH" Section Overview
The added section clearly introduces SSH-based vendoring, highlighting its benefits (i.e. access to non-public repositories without interactive prompts) and setting the stage for the SCP-style URL conversion improvements described in the PR objectives.
123-131: Display of Rewritten Implicit SSH URL
The code snippet showing the rewritten URL (from SCP-style to a fully qualified SSH URL) is well presented and directly supports the explanation above. This concrete example should help users better grasp the transformation process.🧰 Tools
🪛 LanguageTool
[typographical] ~130-~130: Consider using typographic quotation marks here.
Context: ...case, Atmos injects a default username ("git"). ### Explicit SSH When the SSH schem...(EN_QUOTES)
132-141: Explicit SSH Explanation is Clear
The "Explicit SSH" subsection correctly explains that URLs explicitly prefixed withssh://are left as provided. The example reinforces that no rewriting occurs in this scenario, aligning well with user expectations.
142-149: Important Note on URL Format Validation
The "Important Notes" block efficiently warns users about invalid URL formats (e.g. using a colon without a scheme) that cause misinterpretation by go-getter. This direct guidance helps avoid common pitfalls when specifying Git URLs.
150-155: Default HTTPS Behavior Clarification
The explanation that URLs without any scheme default to HTTPS—and that Atmos will inject credentials if available—is clear and concise. This ensures users understand the fallback behavior when an explicit protocol is not provided.
156-167: HTTPS Vendoring Section Clarity
The HTTPS vendoring explanation, including the transformation of partial URLs to fully qualified HTTPS URLs (with the injection of a default depth and the ref parameter), is detailed and informative. The provided code snippet reinforces how this conversion occurs.
212-255: Overall Documentation Clarity and Consistency
The remaining sections (including the examples and flags) integrate well with the newly added SSH/HTTPS guidelines. The detailed examples and step-by-step instructions ensure that users can confidently apply the vendoring commands. Great work aligning the documentation with the enhanced URL handling functionality.🧰 Tools
🪛 LanguageTool
[grammar] ~239-~239: The word ‘vendor’ is a noun or an adjective. A verb or adverb is missing or misspelled here, or maybe a comma is missing.
Context: ...componentis not specified, Atmos will vendor all the artifacts defined in thevendo...(PRP_MD_NN)
[style] ~240-~240: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...efined in thevendor.yamlmanifest. If the flag--componentis passed in, At...(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[grammar] ~240-~240: The word ‘vendor’ is a noun or an adjective. A verb or adverb is missing or misspelled here, or maybe a comma is missing.
Context: ...--componentis passed in, Atmos will vendor only that component - Ifvendor.yaml...(PRP_MD_NN)
[grammar] ~240-~240: Please add a punctuation mark at the end of paragraph.
Context: ... passed in, Atmos will vendor only that component - Ifvendor.yamlis not found, Atmos...(PUNCTUATION_PARAGRAPH_END)
[style] ~242-~242: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...mlmanifest in the component's folder. Ifcomponent.yaml` is not found, an err...(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[grammar] ~243-~243: Please add a punctuation mark at the end of paragraph.
Context: ... flag--componentis required in this case ::: ## Flags | Flag | Descr...(PUNCTUATION_PARAGRAPH_END)
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1061 +/- ##
=======================================
Coverage ? 17.09%
=======================================
Files ? 170
Lines ? 18878
Branches ? 0
=======================================
Hits ? 3228
Misses ? 15079
Partials ? 571
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
website/docs/cli/commands/vendor/vendor-pull.mdx (4)
130-130: Consider Typographic Quotation Marks
On this line, the default username is shown as straight quotes ("git"). For improved readability, consider using typographic quotation marks.🧰 Tools
🪛 LanguageTool
[typographical] ~130-~130: Consider using typographic quotation marks here.
Context: ...case, Atmos injects a default username ("git"). ### Explicit SSH When the SSH schem...(EN_QUOTES)
169-201: Comprehensive Authentication & Token Usage Information
The section covering environment variables for GitHub, Bitbucket, and GitLab is comprehensive and well organized with tables. It might be worth reviewing for typographical consistency in the quotes for the default usernames.🧰 Tools
🪛 LanguageTool
[typographical] ~187-~187: Consider using a typographic close quote here.
Context: ...e and does not accept dummy values like"x-access-token". | **Default Username ...(EN_QUOTES)
[typographical] ~192-~192: Consider using a typographic close quote here.
Context: ...s set, it is used. - If neither is set,"x-token-auth"is used. ### GitLab | En...(EN_QUOTES)
187-192: Typographic Quote Improvements
On these lines, especially where the default username values are shown (for example,"x-access-token"and"x-token-auth"), consider replacing straight quotes with typographically appropriate closing quotes.🧰 Tools
🪛 LanguageTool
[typographical] ~187-~187: Consider using a typographic close quote here.
Context: ...e and does not accept dummy values like"x-access-token". | **Default Username ...(EN_QUOTES)
[typographical] ~192-~192: Consider using a typographic close quote here.
Context: ...s set, it is used. - If neither is set,"x-token-auth"is used. ### GitLab | En...(EN_QUOTES)
204-206: Enhance Wording in HTTPS Resolution Rules
The description here is clear; however, on line 204, consider using a more assertive verb than “follows” (e.g., “applies” or “implements”) for stronger clarity. Additionally, on line 206, replacing the three consecutive periods with a typographical ellipsis ( … ) would improve the style.🧰 Tools
🪛 LanguageTool
[style] ~204-~204: Consider using a different verb to strengthen your wording.
Context: ...en resolving HTTPS-based sources, Atmos follows these rules: 1. If a *full HTTPS URL...(FOLLOW_OBEY)
[style] ~206-~206: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. No token data is a...(ELLIPSIS)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
website/docs/cli/commands/vendor/vendor-pull.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
website/docs/cli/commands/vendor/vendor-pull.mdx
[typographical] ~130-~130: Consider using typographic quotation marks here.
Context: ...case, Atmos injects a default username ("git"). ### Explicit SSH When the SSH schem...
(EN_QUOTES)
[typographical] ~187-~187: Consider using a typographic close quote here.
Context: ...e and does not accept dummy values like "x-access-token". | **Default Username ...
(EN_QUOTES)
[typographical] ~192-~192: Consider using a typographic close quote here.
Context: ...s set, it is used. - If neither is set, "x-token-auth" is used. ### GitLab | En...
(EN_QUOTES)
[style] ~204-~204: Consider using a different verb to strengthen your wording.
Context: ...en resolving HTTPS-based sources, Atmos follows these rules: 1. If a *full HTTPS URL...
(FOLLOW_OBEY)
[style] ~206-~206: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. No token data is a...
(ELLIPSIS)
⏰ Context from checks skipped due to timeout of 90000ms (7)
- GitHub Check: Build (macos-latest, macos)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Analyze (javascript-typescript)
- GitHub Check: Build (ubuntu-latest, linux)
- GitHub Check: Analyze (go)
- GitHub Check: website-deploy-preview
- GitHub Check: Summary
🔇 Additional comments (8)
website/docs/cli/commands/vendor/vendor-pull.mdx (8)
111-112: New SSH Vendoring Section Documentation
The newly added “Vendoring from SSH” section clearly explains how Atmos handles non-public Git repositories via SSH. This lays a good foundation for users needing to access private repos without interactive prompts.
113-117: Implicit SSH Description Clarity
The introductory text explains that SSH is the preferred method for non-public repositories and sets the stage nicely. It may help to briefly remind users that the rewrite occurs only when no explicit scheme is provided.
120-122: Implicit SSH Example is Clear
The code block example for an implicit SSH URL is concise and shows the expected input format (e.g. usinggit::[email protected]:cloudposse/terraform-null-label.git?ref={{.Version}}).
124-128: SSH URL Rewriting Example
The subsequent code block clearly demonstrates how Atmos rewrites an implicit SSH URL to include the explicitssh://scheme and adds parameters likedepth=1. This example should help users understand the URL transformation.
132-140: Explicit SSH Section Clarity
The section describing explicit SSH usage is straightforward. It clearly differentiates between implicit rewriting and when the URL is used as provided (if it already includesssh://).
142-148: Invalid URL Example is Informative
The “Important Notes” section effectively demonstrates with a code block which URL format (using a colon without a scheme) is invalid. This is a helpful clarification for users.
150-155: HTTPS Default Behavior Explanation
These lines clearly explain that URLs without an explicit scheme default to HTTPS and that Atmos will inject credentials if available. The explanation is direct and easy to follow.
156-168: HTTPS Vendoring Section Detail
The example provided for converting a bare repository string to a full HTTPS URL is detailed and well structured. It conveys the transformation process effectively.
what
Done:
This is a spin off of #984 that futher extends custom detector logic
Testing
Use this to run only test cases relevant for this PR
$ go test -v -run '^TestCLICommands/(atmos_vendor_pull_using_SSH|atmos_vendor_pull_with_custom_detector_and_handling_credentials_leakage)$' github.com/cloudposse/atmos/testsnon-standard SCP-style links handling
Token injections were tested wtih bitbucket and gitlab (http) for private and public repos + ssh vendoring for both.
Listing them here as there are no dedicated tests/repos available for testing at bitbucket/gitlab.
why
git::[email protected]:cloudposse/terraform-null-label.git?ref={{.Version}}references
Summary by CodeRabbit
New Features
Documentation
Tests
Refactor