[SPARK-54914] [SQL] Fixing DROP operator in pipe syntax to support qualified column names

[Shubhambhusate]

What changes were proposed in this pull request?

This PR fixes the DROP operator in pipe syntax (|>) to support qualified column names.

Why are the changes needed?

Previously, the DROP operator in pipe syntax only accepted simple identifiers due to the use of identifierSeq in the grammar. This prevented users from dropping struct fields using qualified names

Does this PR introduce any user-facing change?

Yes. Users can now use qualified column names with the DROP pipe operator.

Before changes:

After changes:

How was this patch tested?

Test 1:
Updated SQL test cases in pipe-operators.sql with positive test for dropping struct fields using qualified names
Updated expected results in pipe-operators.sql.out and analyzer-results/pipe-operators.sql.out

Test 2:

spark.sql("""
| SELECT 2 AS lhs_a, 1 AS lhs_b
| |> AS lhs
| |> JOIN (VALUES (2, 1)) AS rhs(a, b)
| ON lhs.lhs_a = rhs.a
| |> Drop lhs.lhs_a
| """).show()

Was this patch authored or co-authored using generative AI tooling?

No

[github-actions]

JIRA Issue Information

=== New Feature SPARK-54914 ===
Summary: Pipe Syntax - Qualified Column Names in DROP Operator
Assignee: None
Status: Open
Affected: ["4.1.0"]

---

This comment was automatically generated by GitHub Actions