fix(checks): descendants JSONPath returns scalar vs list depending on match count - #2634
Closed
Qalipso wants to merge 1 commit into
Closed
fix(checks): descendants JSONPath returns scalar vs list depending on match count#2634Qalipso wants to merge 1 commit into
Qalipso wants to merge 1 commit into
Conversation
… match count _is_list_expression handles Descendants in the same recursive branch as Child, so `trace..ctx` -- whose children are both single-field Fields -- returns False. resolve() then falls back to its match-count heuristic (`len(matches) > 1 or _is_list_expression(expression)`), and a `..` expression yields a bare scalar when the data holds exactly one match and a list when it holds two or more. This is the defect Giskard-AI#2605 fixed for dot-wildcards and multi-field selectors; `..` compiles to Descendants and was not covered. It breaks ComparisonCheck's match=any/all/none, which requires a list/set/tuple: Equals(key="trace..ctx", expected_value="doc-1", match="any") fails with a type error when one interaction carries the field and passes when two do, for an identical check configuration. Give Descendants its own branch returning True and leave Child recursing. A `..` searches a whole subtree, so its match count is a property of the data, not of the expression. Consequence: `trace..missing` now returns [] rather than NoMatch, which matches how list-expressions already behave (`trace.last.metadata.*` on empty metadata returns [] today). Ordinary paths such as `trace.last.nope` still return NoMatch. Fixes Giskard-AI#2633
Qalipso
force-pushed
the
fix/jsonpath-descendants-list
branch
from
July 26, 2026 03:28
6cef49c to
1733080
Compare
davidberenstein1957
self-requested a review
July 29, 2026 14:13
davidberenstein1957
approved these changes
Jul 29, 2026
Comment on lines
+68
to
+72
| # `..` searches a whole subtree, so how many values it finds is a property | ||
| # of the data, not of the expression. Treat it as a list-expression the way | ||
| # bracket wildcards are, or resolve() falls back to its match-count | ||
| # heuristic and returns a bare scalar whenever the data happens to hold one | ||
| # match. |
Member
There was a problem hiding this comment.
Suggested change
| # `..` searches a whole subtree, so how many values it finds is a property | |
| # of the data, not of the expression. Treat it as a list-expression the way | |
| # bracket wildcards are, or resolve() falls back to its match-count | |
| # heuristic and returns a bare scalar whenever the data happens to hold one | |
| # match. |
3 tasks
Member
|
Opened a same-repo follow-up with the same fix plus format fixes so CI lint can pass (fork PRs cannot receive the auto-format push): this supersedes the branch here. Thanks for the clear repro and tests — credited as co-author on the fix commit. |
2 tasks
Member
Contributor
Author
|
Thanks — happy to have #2645 carry it, and no objection to closing this once that lands. One heads-up: #2645 currently shows as conflicting with |
kevinmessiaen
added a commit
that referenced
this pull request
Aug 5, 2026
* style: auto-format README code blocks for current ruff Unblocks check-format on PRs; same drift as CI style: auto-format. * fix(checks): descendants JSONPath always returns a list Treat Descendants as a list-expression so resolve() does not fall back to the match-count heuristic and return a bare scalar for a single hit. Fixes #2633. Supersedes #2634. Co-authored-by: Qalipso <quadwailt@gmail.com> * fix(checks): drop unnecessary pyright ignore for MISSING The reportInvalidTypeForm ignore is flagged as unnecessary after chore(pyright): flag and remove unused ignore comments (#2690) merged from main, which blocked the lint CI job on PR #2644. --------- Co-authored-by: Qalipso <quadwailt@gmail.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Kevin Messiaen <kevinmessiaen@users.noreply.github.com> Co-authored-by: Kevin Messiaen <114553769+kevinmessiaen@users.noreply.github.com>
Member
|
Duplicate of #2644 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2633.
Problem
_is_list_expressionhandlesDescendantsin the same recursive branch asChild:For
trace..ctxboth children are single-fieldFields, so this returnsFalseandresolve()falls back to its match-count heuristic (len(matches) > 1 or _is_list_expression(expression)). A..expression then yields a bare scalar when the data holds exactly one match and a list when it holds two or more.This is the defect #2605 fixed for dot-wildcards and multi-field selectors. That fix added a
Fieldsbranch;..compiles toDescendantsand was not covered...is a supported expression —test_extraction.pyalready assertstrace..outputsvalidates.It breaks
ComparisonCheck'smatch=any/all/none, which requires a list/set/tuple:Identical check configuration; the outcome depends on incidental data shape.
Fix
Give
Descendantsits own branch returningTrue, and leaveChildrecursing. A..searches a whole subtree, so its match count is a property of the data, not of the expression — the same reasoning the bracket-wildcard andFieldsbranches already encode.Behavior change worth flagging
trace..missingnow returns[]instead ofNoMatch. That matches how list-expressions already behave —trace.last.metadata.*on empty metadata returns[]today — while ordinary paths such astrace.last.nopekeep returningNoMatch. Covered by a test.Verification
uv run pytest libs/giskard-checks -m "not functional"→ 761 passed, 4 skipped (757 passed, 4 skipped onmainbefore this change; the 4 added tests are the difference).extraction.pyreproduces'doc-1' == ['doc-1'](bare scalar) and fails 2 of the new tests; reapplying passes all 5 in the class.ruff check/ruff format --checkon both changed files: clean.basedpyrightonextraction.py: 0 errors, 28 warnings — byte-identical to the count onmain, so nothing new was introduced.Python 3.13,
jsonpath-ngfrom the pinned range.Separate observation, not addressed here
Trace.model_dump()includes alastkey duplicating the final interaction, sotrace..outputson a single-interaction trace returns['hello', 'hello']. Different problem, different blast radius — noted in #2633 and left alone.