Support fieldref combined with contains, startswith, and endswith - #506
Merged
Merged
Conversation
Rules such as fieldref|contains failed to compile because fieldref was equality only. Follow pySigma: the string modifier must follow fieldref, and the referenced name cannot contain wildcards.
The lint ignored fieldref, so combinations the compiler rejects stayed quiet, and it treated neq as a numeric comparison even though neq negates any value.
Postgres compares the referenced text with strpos and right so percent and underscore stay literal. Fibratus equality is case-insensitive, and substring comparisons use the matching icontains, istartswith, and iendswith operators.
Field references render with ~=, so a cross-process guard such as evt.pid != thread.pid arrives as not (evt.pid ~= thread.pid). The macro recognizer now accepts that form and still folds create_remote_thread.
A missing referenced field counts as not equal when the left field is present. PostgreSQL uses IS NOT TRUE together with an IS NOT NULL check on the left field so a NULL comparison still matches. Other backends negate the comparison directly.
Document the contains, startswith, and endswith combinations, the PostgreSQL and Fibratus renderings, and the lint rule that accepts them.
The neq wrap ran after the regex, cidr, numeric, and timestamp branches returned, so those combinations compiled without the negation. Negate first, then compile the remaining modifiers.
LynxDB renders regex and cidr as deferred where clauses. Negating only the inline part dropped the negation, so neq now negates the deferred parts it produced.
Field|neq: [a, b] matched when the field differed from either value, which is almost always. pySigma negates the whole item, so the list now matches only when the field is none of the values. Backends render the negated list as one grouped NOT, and Fibratus negates its collapsed list clause.
Sigma string comparison is case-insensitive, and the evaluator and the substring forms already fold case. Equality now compares lower() of both sides unless |cased is set.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #505.
fieldrefmay be followed by one ofcontains,startswith, orendswith. The comparison is case-insensitive unless|casedis set. A wildcard in the referenced name is rejected, and a string modifier beforefieldrefis rejected. Thanks to @Karib0u for reporting these three combinations.strposandright, so%and_stay literal, andfieldrefequality compareslower()of both sides unless|casedis set. Fibratus renders equality with~=and the substring forms withicontains,istartswith, andiendswith.|neqnegates the whole detection item, as pySigma does.Field|neq: [a, b]matches when the field is neitheranorb; before, it matched when the field differed from either value.re|neq,cidr|neq,fieldref|neq, andneqwith a timestamp part also compiled without the negation before.|neqconverts. A missing referenced field counts as not equal when the left field is present; PostgreSQL expresses that as(comparison) IS NOT TRUE AND "field" IS NOT NULL. LynxDB negates its deferredwhereclauses (!~,NOT cidrmatch).incompatible_modifiersaccepts the new combinations and warns on the ones that still conflict.neqcombines with string comparisons as well as with numbers.FieldRefcarries the string operator. The HIR cache schema is 2, so a cache written by an older build is rejected and recompiled.Test plan
cargo test --workspace --exclude rstix --no-fail-fast --lib --testscargo clippy --workspace --all-targets --all-features -- -D warningscargo fmt --all -- --checknpm run docs:buildandnpm run docs:validatefromdocs/