[plusplusjiajia]

Purpose

As in Java LeafBinaryFunction and LeafTernaryFunction, a comparison with a null literal matches no row and no file. The Python predicates compared values, stats and Arrow columns with None instead, so a filter such as not_equal('v', None) built with PredicateBuilder gave wrong results or failed:

• notEqual returned every non-null row when a primary-key read filters the merged rows. Java returns no rows.
• equal, lessThan, lessOrEqual, greaterThan, greaterOrEqual, between, notBetween, startsWith, endsWith and contains raised TypeError, in the row filter or when pruning files by value stats, for example in an append table with metadata.stats-mode: full.
• like turned the null pattern into the string 'None' and matched it.
• notBetween with a null bound returned rows where the Arrow expression is the final filter, as in an append table read: Arrow evaluates x < a OR x > NULL as true for x < a.
• System tables did the same for notBetween with a null bound, both in the Arrow filter and in the row filter that [python] Support string filters on system tables #10631 added for string predicates: not_between('id', 3, None) returned the rows with id < 3.

A null literal now fails the row test and the stats test, as in Java, and converts to an Arrow expression that matches no row. The system-table row filter drops its own null handling and relies on this. in and notIn already follow Java since #9132 and are unchanged.

Tests

• PredicateTest#test_null_literal_matches_nothing: all 12 comparisons with a null literal return no rows from four tables: an append table with full stats, an append table without stats so that the Arrow filter decides, a primary-key table, and a primary-key table with two commits to one bucket so that rows are filtered after the merge.
• PredicateTest#test_null_literal_by_value_stats_and_arrow: the row test, the stats test and the Arrow expression of each comparison reject a null literal, and like does not match the string 'None'.
• On master, 40 subtests of these two tests fail.
• system_string_filter_test#test_mixed_null_range_and_set_comparisons: not_between with a null bound now expects no rows, alone and combined with a string predicate.
• The predicate, pushdown, filter, stats, Arrow, decimal and timestamp tests pass locally: 1029 tests, and the system table tests: 224 tests. flake8 is clean.

[JingsongLi]

+1