Skip to content

Conversation

marzaccaro
Copy link

@marzaccaro marzaccaro commented Sep 24, 2025

resolves #1041

Problem

The dbt_utils.recency test incorrectly passes when a where clause filters
out all records.

Current behavior: When a where clause results in no matching records,
most_recent becomes NULL. The test only checks where most_recent < {{ threshold }}, but since NULL < threshold evaluates to NULL (not true), no
rows are returned and the test incorrectly passes.

Expected behavior: The test should fail when no recent data exists (even
when filtered by a where clause), indicating missing recent data.

Solution

Modified the recency macro in macros/generic_tests/recency.sql to include
or most_recent is null in the WHERE clause. This ensures that when a where
clause filters out all records (causing most_recent to be null), the test
correctly fails by returning the null row.

Changes made:

  • Updated the WHERE condition from where most_recent < {{ threshold }} to
    where most_recent < {{ threshold }} or most_recent is null
  • Added integration test recency_with_where_filter that reproduces the
    original bug and verifies the fix
  • Test uses data with old timestamps (10-15 days) and a where clause that
    filters to non-existent records

Testing approach:

  • Before fix: Test incorrectly passes (returns 0 rows)
  • After fix: Test correctly fails (returns 1 row with null most_recent)
  • Verified existing recency tests continue to work as expected

Checklist

  • This code is associated with an
    issue which has been
    triaged and [accepted for development](https://docs.getdbt.com/docs/contributin
    g/oss-expectations#pull-requests).
  • I have read the contributing
    guide
    and
    understand what's expected of me
  • I have run this code in development and it appears to resolve the stated
    issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the README.md (if applicable)

@marzaccaro marzaccaro requested a review from a team as a code owner September 24, 2025 14:08
@marzaccaro marzaccaro changed the title Recency test to fail when no recent record is found Recency test to fail when no recent records are found Sep 24, 2025
@marzaccaro marzaccaro force-pushed the fix/recency-test-working-with-where branch from 13e913a to 8eabbe5 Compare September 24, 2025 14:21
@marzaccaro marzaccaro force-pushed the fix/recency-test-working-with-where branch from 8eabbe5 to 3d865af Compare September 24, 2025 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue in recency test when using where in the config

1 participant