You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat: find security variants with ephemeral SAST rules and save them as custom rules - #49
Adds a workflow that takes an uncommitted security fix, writes a SAST rule for the mistake it fixes, runs that rule locally, and, once the user approves it, saves it as a Datadog custom rule.
datadog_ephemeral_sast_scan / ephemeral-scan: runs one caller-supplied rule with datadog-static-analyzer, without publishing it and without API keys.
datadog_promote_sast_rule / promote-sast-rule: saves an approved rule, its examples and description through the existing custom-rule API (default ruleset security-variants).
dd-codesec-find-security-variants skill: drives the workflow, with a confirmation pause before searching and before saving.
The ephemeral scan passes --rules-with-config: the analyzer runs only the supplied rule but keeps the repository config's ignored paths, only paths, file size limit and gitignore setting. Support is detected from --help (development builds report the last release's version); older analyzers get an upgrade error with install commands.
paths must stay inside directory; the scan has a 10-minute timeout; findings are kept when the analyzer exits non-zero.
Saving refuses to publish over an existing rule unless overwrite / --overwrite is set after the user agrees, and warns if Datadog returns no revision id.
Telemetry: findings_count counts violations instead of files, and the new errors map to curated, path-free messages. docs/TELEMETRY.md lists both operations.
README, CLAUDE.md and AGENTS.md document both tools, the CLI commands and DD_AUTH_DOMAIN.
Reviewer notes
E2E Test 10 uses only public fixtures in testdata/ephemeral/ and no credentials. It checks that only the ephemeral rule reports and that generated/ (ignored by the fixture config, not by .gitignore) stays ignored. It requires datadog-static-analyzer 0.9.10 or later (the first release with --rules-with-config) and fails on older analyzers.
testdata/ephemeral/src/handler.go intentionally contains an exec.Command call so the test can prove go-security rules from the config do not run; SAST flags it.
A pre-existing issue found while testing, dd-auth keys not refreshing in long MCP sessions, is fixed separately in K9CODESEC-8183.
Testing
go test -race ./..., golangci-lint (0 issues), gofmt.
./scripts/test-e2e.sh --ci with a locally built analyzer from the K9CODESEC-8033 branch: 11 passed, 0 failed. With analyzer 0.9.8: the upgrade error is returned as expected.
Saving against the real API: create (revision 2787), refusal without overwrite, overwrite (revision 2788, rule_existed: true).
Release order
Release datadog-static-analyzer with --rules-with-config (K9CODESEC-8033).
Add the minimum analyzer version to the upgrade error and the README, and make E2E Test 10 fail instead of warn when the flag is missing.
Release the MCP (the ephemeral scan returns the upgrade error to everyone until the analyzer release is out).
Merge the public docs update with the minimum analyzer version.
Adds a security-variant workflow to the local Code Security MCP server, turning an uncommitted fix into a temporary SAST rule that users can approve for publication.
Changes:
Adds ephemeral scanning and custom-rule publishing through CLI commands and MCP tools.
Ships a guided variant-search skill with confirmation steps.
Adds validation, telemetry, documentation, and automated coverage.
File
Description
testdata/ephemeral/static-analysis.datadog.yml
Configures rule-isolation and ignored-path checks.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Rule-execution failures can appear as successful scans, and input handling still needs corrections.
No open issues 🎉
4 previously missed issues
In code that hasn't changed since last review
Use normalized authentication configuration for client construction
cmd/datadog-code-security-mcp/promote.go:265
Use the normalized auth configuration rather than reading DD_SITE again. With environment-provided keys, setAuthCredentials returns without applying the normalized config. Thus DD_SITE=" datadoghq.com " passes MCP startup validation but promotion constructs an invalid host; the CLI path applies the normalized site and succeeds. Construct the client from auth.LoadConfig() so both interfaces use the validated values.
Surface rule execution errors when findings are empty
internal/ephemeral/run.go:95
A generated visitor that throws on every match can produce [] without a failing process exit. The analyzer filters out results with no violations even when they contain execution errors (see the analyzer's result filtering). Accepting that JSON here reports a broken rule as a successful zero-finding scan. Surface and check rule-execution diagnostics before reporting success, and add a throwing-visitor regression case. Preserve partial findings separately from execution status.
Reject explicit path lists that resolve to no targets
internal/ephemeral/run.go:175
An explicit scope containing only blank entries, such as paths: [" "] or --path "", returns no targets without an error. analyzerArgs then omits every -u, unexpectedly scanning the entire directory. Reject a supplied path list that resolves to no targets rather than silently widening the scan, and add a regression case for an all-blank list.
Count Unicode characters instead of UTF-8 bytes
internal/promote/client.go:236
len(short) counts UTF-8 bytes rather than the documented 60 characters. A description containing 31 é characters is therefore rejected before any API call. Count runes instead, and cover a non-ASCII description below the limit in a regression test.
…rity fix
The agent can discover it from a natural-language question, and the generated rule stays in the system temp directory instead of the repository.
Co-authored-by: Cursor <cursoragent@cursor.com>
…rule API
The tool uploads the exact source after the user confirms. The skill names the IDE refresh only when the session is Cursor or VS Code.
Co-authored-by: Cursor <cursoragent@cursor.com>
The promote call was reading the environment only, so a Datadog user configured with DD_AUTH_DOMAIN could not save a rule.
Co-authored-by: Cursor <cursoragent@cursor.com>
The custom-rule detail page decodes creation_message as base64 and treats a decode error as a missing rule.
Co-authored-by: Cursor <cursoragent@cursor.com>
The invariant check and the config offer were still free text, so the flow waited. Refresh is a fallback when the rule is not visible yet.
Co-authored-by: Cursor <cursoragent@cursor.com>
A chat can drop text written before a question card in the same turn, and the model often keeps that text in its reasoning. Pauses that need reading now end the turn with plain markdown and a direct question. A no to the extra search still leads to the report and the save question. The fix covers arrow functions and methods, the save question names the rule, and a saved run ends with the list of places to fix. Save and config steps move to references/.
Co-authored-by: Cursor <cursoragent@cursor.com>
The id sent in the create request was returned as the revision id, but Datadog stores its own. Read data.id from the response, fall back to the rule's last_revision, and return an empty id rather than the request id if both fail. The skill now shows that id.
Co-authored-by: Cursor <cursoragent@cursor.com>
…ick fix
The IDE kept an older revision of a rule that already existed, and the refresh brought the fix back.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Pass --rules-with-config so the analyzer runs only the supplied rule while
keeping the repository config's ignored paths, only paths, file size and
gitignore settings. Detect support from --help and ask for an upgrade
when the analyzer lacks it.
- Keep scan paths inside the scan directory, bound the scan with a timeout
and keep findings when the analyzer exits non-zero.
- Refuse to overwrite an existing custom rule unless overwrite is set after
the user agrees, and warn when Datadog returns no revision id.
- Rename the default ruleset to security-variants.
- Count violations, not files, in ephemeral telemetry and classify the new
errors with curated messages.
- Add a public E2E fixture for the ephemeral scan and document both tools.
Refs: K9CODESEC-8034, K9CODESEC-7800
Co-authored-by: Cursor <cursoragent@cursor.com>
The analyzer calls visit, beforeAll, and afterAll as plain functions in
strict mode, so this is undefined. The guard called them as methods, so a
rule storing state on this passed locally and threw once published.
The outside-directory error includes the supplied path, but this classification leaves verbatim false. decorateMessage therefore treats any exit status followed by digits in that path as a process exit code. For example, paths: ["../exit status 4111111111111111"] produces error.message = "path outside scan directory (exit status 4111111111111111)", exposing filename content despite the path-free telemetry contract. Set verbatim: true for this validation error and add a regression with an exit-status-like string in the rejected path.
The guard ran the rule's top-level code outside any try/catch, so a
throwing initializer stopped the script and the scan looked clean. It now
catches that too, and visit reports the exception once per file.
The ephemeral scan now runs the user's rule unchanged and passes
--debug yes. The analyzer prints a line for each file where the rule
throws, does not compile, or times out, and the scan reports those files
as an error. The previous try/catch wrapper changed how the rule ran and
missed syntax errors and timeouts.
The analyzer output is streamed: only the failure lines and the last
2000 bytes are kept.
… new errors
- promote: check confirmation, slug, and examples before fetching credentials
- promote: explain the empty rule left behind when the revision save fails
- promote: accept only ASCII slugs
- ephemeral: report cancellation and timeout as such; compare resolved paths
- telemetry: curated messages for rule input, invalid JSON, and probe errors
- docs: AGENTS.md tree, analyzer default skips in the variants skill
- CheckExamples reads the analyzer's 'Apply rule' debug lines and reports an
example it never scanned (wrong extension, generated, or minified) instead
of counting it as 0
- Example filenames with surrounding spaces are rejected, not trimmed
- ScannerError carries the analyzer log; telemetry matches it by type so the
caller's rule or query text cannot change the classification
- Tool and CLI descriptions mention --rules-with-config and file paths
Overlapping targets are forwarded separately: paths: ["src", "src/handlers"], or the same target supplied relatively and absolutely, produces multiple -u arguments covering the same files. Analyzer 0.9.10's get_files walks each target independently and appends files without deduplication, so overlapping files can produce duplicate findings and inflate findings_count. Collapse equal targets and targets covered by an ancestor before invoking the analyzer. Update the test that expects src twice, and cover nested targets in both input orders.
- decorateMessage reads the code from *exec.ExitError, not from error text,
so a rejected path or rule name cannot put digits into telemetry
- SBOM results keep the original error (not serialized) so SCA failures
still report their exit code
- Rule and ruleset name errors are typed and classified as invalid arguments
- Tests create real exit codes with the test binary, so they run on Windows
Validation, API, and scanner-output errors now carry typed reasons, so
caller paths, rule names, and API response bodies cannot steer the
telemetry kind. API errors are classified by status; bodies are capped.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Caller-supplied rule execution and remote publication require final human review of analyzer compatibility and safeguards after the outstanding corrections.
Extracting only the function can remove context required by the generated rule. Step 3 supports import edits and class methods, but a query that captures an import alongside the function cannot match an example without that import. CheckExamples scans the supplied text unchanged, so promotion can reject a valid rule with expected 1, got 0 and send the agent back to rewriting it. Preserve the necessary imports and enclosing declarations in both examples.
- Match the analyzer's name for a rule already prefixed with its ruleset
- Reject control characters in rule and ruleset names
- Join failure lines split by a newline in a filename; an unfinished one
still counts as a failure
- Use the validated DD_SITE for the custom rule client
- Reject MCP paths that are not an array of strings
- Bound the wait for analyzer output with WaitDelay; test cancellation
during the scan
- Test that a real SBOM exit code reaches telemetry
- E2E: count the failing-rule checks once, guard jq, keep their outputs
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Caller-supplied rule execution, diagnostic parsing, and shared-rule publishing require final human integration review alongside the requested corrections.
No open issues 🎉
2 previously missed issues
In code that hasn't changed since last review
Deduplicate overlapping analyzer targets
internal/ephemeral/run.go:221
Equivalent or overlapping targets are passed to the analyzer more than once. For example, paths: ["src", "./src"] becomes two -u src arguments. Analyzer 0.9.10 walks each target separately without deduplicating its file list, so matching files are scanned twice, duplicate findings are returned, and findings_count is inflated. Remove equivalent targets and descendants already covered by another target before invoking the analyzer. Update TestScanPassesRulesWithConfig, which currently expects duplicate arguments, and cover ancestor/descendant targets in both orders.
Function-only examples can reject a working rule. Step 3 supports class methods, assigned arrow functions, and import edits, but CheckExamples writes the supplied text verbatim. Removing an enclosing class, variable declaration, or relevant import can make a query stop matching or its visitor fail. The pre-save check then blocks publication and tells the agent to rewrite a valid rule. Keep the minimal valid surrounding syntax and context needed by the query, visitor, and fix in both examples, with only the security fix differing.
- Ephemeral scans pass each target once and skip targets inside another,
so the analyzer does not report the same finding twice
- save-rule.md keeps the enclosing class, assignment, and imports the rule
needs in both examples
A validation error leaves an empty rule, and saving the same request again fails the same way. Lead with the server's detail, and reserve the overwrite retry for failures that can succeed on another try.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a workflow that takes an uncommitted security fix, writes a SAST rule for the mistake it fixes, runs that rule locally, and, once the user approves it, saves it as a Datadog custom rule.
datadog_ephemeral_sast_scan/ephemeral-scan: runs one caller-supplied rule withdatadog-static-analyzer, without publishing it and without API keys.datadog_promote_sast_rule/promote-sast-rule: saves an approved rule, its examples and description through the existing custom-rule API (default rulesetsecurity-variants).dd-codesec-find-security-variantsskill: drives the workflow, with a confirmation pause before searching and before saving.Jira: K9CODESEC-8034 (MCP side), umbrella K9CODESEC-7800. Analyzer side: K9CODESEC-8033.
Production hardening (last commit)
--rules-with-config: the analyzer runs only the supplied rule but keeps the repository config's ignored paths,onlypaths, file size limit and gitignore setting. Support is detected from--help(development builds report the last release's version); older analyzers get an upgrade error with install commands.pathsmust stay insidedirectory; the scan has a 10-minute timeout; findings are kept when the analyzer exits non-zero.overwrite/--overwriteis set after the user agrees, and warns if Datadog returns no revision id.findings_countcounts violations instead of files, and the new errors map to curated, path-free messages.docs/TELEMETRY.mdlists both operations.DD_AUTH_DOMAIN.Reviewer notes
testdata/ephemeral/and no credentials. It checks that only the ephemeral rule reports and thatgenerated/(ignored by the fixture config, not by.gitignore) stays ignored. It requiresdatadog-static-analyzer0.9.10 or later (the first release with--rules-with-config) and fails on older analyzers.testdata/ephemeral/src/handler.gointentionally contains anexec.Commandcall so the test can provego-securityrules from the config do not run; SAST flags it.Testing
go test -race ./...,golangci-lint(0 issues),gofmt../scripts/test-e2e.sh --ciwith a locally built analyzer from the K9CODESEC-8033 branch: 11 passed, 0 failed. With analyzer 0.9.8: the upgrade error is returned as expected.rule_existed: true).Release order
datadog-static-analyzerwith--rules-with-config(K9CODESEC-8033).