Skip to content

Fix ValuesExpression pruning dropping outer columns referenced from VALUES cells - #38779

Merged
AndriySvyryd merged 2 commits into
dotnet:mainfrom
yvesleguennec:fix/38700-values-pruning-column-refs
Sep 10, 2026
Merged

AndriySvyryd merged 2 commits into
dotnet:mainfrom
yvesleguennec:fix/38700-values-pruning-column-refs

Conversation

@yvesleguennec

Copy link
Copy Markdown
Contributor

Fixes #38700

SqlTreePruner.PruneValues (from #36159) copied VALUES row cells without
Visit(), so ColumnExpressions embedded in inline collections were never
registered. Outer join projections were then over-pruned (e.g. only Id
projected while EXISTS still referenced ServiceId).

Fix

  • Always Visit() nested RowValues / ValuesParameter in PruneValues
    (same pattern as OpenJSON pruning)
  • Fix column-name backfill to use ColumnNames[j] instead of repeating the
    first unreferenced column's name

Tests

  • Relational AdHoc query filters regression for the reported scenario
  • SqlServer SQL baseline asserting ServiceId remains in the join projection

Note: approach described on #38700; no maintainer reply after ~1.5 weeks, opening PR for review.


  • I've read the guidelines for contributing and seen the walkthrough
  • I've posted a comment on an issue with a detailed description of how I am planning to contribute and got approval from a member of the team
  • The code builds and tests pass locally (also verified by our automated build checks)
  • Commit messages follow this format:

        Summary of the changes
        - Detail 1
        - Detail 2

        Fixes #bugnumber
  • Tests for the changes have been added (for bug fixes / features)
  • Code follows the same patterns and style as existing code in this repo

@yvesleguennec

Copy link
Copy Markdown
Contributor Author

@dotnet-policy-service agree

@AndriySvyryd
AndriySvyryd requested a balanced review from Copilot August 12, 2026 00:29
@AndriySvyryd AndriySvyryd self-assigned this Aug 12, 2026
@AndriySvyryd AndriySvyryd added this to the 12.0.0 milestone Aug 12, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes VALUES pruning so nested outer-column references are retained.

Changes:

  • Visits VALUES cells during pruning.
  • Corrects pruned column-name backfilling.
  • Adds relational regression coverage and a SQL Server baseline.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/EFCore.Relational/Query/SqlTreePruner.cs Corrects VALUES traversal and column pruning.
test/EFCore.Relational.Specification.Tests/Query/AdHocQueryFiltersQueryRelationalTestBase.cs Adds the regression scenario.
test/EFCore.SqlServer.FunctionalTests/Query/AdHocQueryFiltersQuerySqlServerTest.cs Verifies the generated SQL projection.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/EFCore.Relational/Query/SqlTreePruner.cs
Comment thread test/EFCore.Relational.Tests/Query/SqlTreePrunerTest.cs Outdated
…ALUES cells

- Visit nested RowValues/ValuesParameter in PruneValues so embedded
  ColumnExpressions are registered before outer join projections are pruned
- Fix column-name backfill to use ColumnNames[j] instead of repeating the
  first unreferenced column's name

Fixes dotnet#38700
…tnet#38700)

Cover the jk-aau case (inline-array Max + AsSplitQuery + pagination),
forcing VALUES+MAX on SQL Server compat < 160.
@yvesleguennec
yvesleguennec force-pushed the fix/38700-values-pruning-column-refs branch from 60b1d5b to eeee33b Compare August 21, 2026 05:45

@AndriySvyryd AndriySvyryd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!

@AndriySvyryd
AndriySvyryd merged commit 0e21341 into dotnet:main Sep 10, 2026
15 checks passed
@jk-aau

jk-aau commented Sep 10, 2026

Copy link
Copy Markdown

@AndriySvyryd Will this PR be patched into the current version (EF 10) or do we have to wait until EF 12?

@AndriySvyryd

Copy link
Copy Markdown
Member

@jk-aau Currently the issue doesn't have enough votes (👍 reactions) to be considered for 10 or 11 backport.

@yvesleguennec

yvesleguennec commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor Author

@AndriySvyryd Following up on the backport question.

You already labeled #38700 as regression — this isn’t a new query-filter capability, it’s invalid SQL from previously-valid LINQ after 9 → 10 (#36159 / PruneValues not visiting VALUES cells). Same code, net10.0 on both sides: works on 9.0.18, throws on 10.0.0–10.0.11 (Unknown column / no such column). jk-aau hit the same hole via new[]{...}.Max() + split-query pagination after an 8 → 10.0.11 upgrade with no application changes.

We’re a .NET 10 / EF 10.0.11 production app. The issue repro is our access-control engine (new int?[] { navigation.ServiceId }.Any(...), often two navigations). On 10.x those queries fail at runtime; we had to downgrade that branch to EF 9.x (still net10.0) and cannot merge it onto our EF 10 line until this is patched. The Contains workaround only covers a single non-nullable column, not a shared IEnumerable<int?> (nullable paths, OR of several navigations, line-level service ids).

The fix is already in this PR (small Visit() + regression tests). .NET 10 is LTS — could this be considered for 10.0.x servicing on that basis, rather than vote count alone?

AndriySvyryd pushed a commit that referenced this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Regression 9.0 → 10.0] Query filter with inline array of navigation columns generates SQL referencing pruned columns

4 participants