Add keypath method and initializer syntax under an experimental feature flag `keypathWithMethodMembers` #2950

amritpan · 2025-01-22T02:48:05Z

This accompanies the swift-foundation and compiler implementation for Method and Initializer Keypaths which extends keypath usage to include references to methods and initializers:

struct S {
  static let millenium = 3
  var year = 2024
  init() {}
  init(val value: Int = 2024) { year = value }
  
  func add(this: Int) -> Int { this + this}
  func add(that: Int) -> Int { that + that }
  static func subtract(_ val: Int) -> Int { return millenium - val }
  nonisolated func nonisolatedNextYear() -> Int { return year + 1 }
  consuming func consume() { print(year) }

  subscript(index: Int) -> Int { return year + index }
}

let kp1: KeyPath<S, () -> S> = \S.Type.init
let kp2: KeyPath<S, S> = \S.Type.init()
let kp3: KeyPath<S, (Int) -> S> = \S.Type.init(val:)
let kp4: KeyPath<S, S> = \S.Type.init(val: 2025)
let kp5: KeyPath<S, (Int) -> Int> = \S.add(this:)
let kp6: KeyPath<S, Int> = \S.add(that: 1)
let kp7: KeyPath<S, Int> = \S.Type.subtract(1)
let kp8: KeyPath<S, Int> = \S.nonisolatedNextYear()
let kp9: KeyPath<S, Int> = \S.nonisolatedNextYear().signum()
let kp10: KeyPath<S, String> = \S.nonisolatedNextYear().description
let kp11: KeyPath<S, ()> = \S.consume()
let kp12: KeyPath<S, Int> = \S.Type.init()[1]

ahoppen

The implementation looks good to me. I have a few small comments inline.

ahoppen · 2025-01-22T18:40:13Z

CodeGeneration/Sources/SyntaxSupport/ExprNodes.swift

+      Child(
+        name: "leftParen",
+        kind: .token(choices: [.token(.leftParen)]),
+        isOptional: true
+      ),


Why is the left parenthesis optional? Wouldn’t we just have a normal member component if we don’t have parentheses?

I modeled this after .funcCallExpr but you're right and I will make this non optional. @ahoppen I also have 2 related questions:

.keyPathPropertyComponent has a .genericArgumentClause but I don't see any examples of it being used in the tests. Do you happen to know when that is used? It seems like a generic on the root (eg: \Box<Int>.item) is already being handled by KeyPathExpr.

Is there a way to make .labeledExprList optional? I forgot to account for this - partially applied methods parse their method name and arg list into DeclReferenceExpr and do not have a LabeledExprList. I'm inclined to create a second node to handle this (eg: .keyPathPartiallyAppliedComponent).

.keyPathPropertyComponent has a .genericArgumentClause but I don't see any examples of it being used in the tests. Do you happen to know when that is used? It seems like a generic on the root (eg: \Box<Int>.item) is already being handled by KeyPathExpr.

I don’t think a generic clause is actually valid in key path properties and maybe we should remove it. @rintaro Can you think of a reason why KeyPathPropertyComponentSyntax should have a generic arguments clause?

Is there a way to make .labeledExprList optional? I forgot to account for this - partially applied methods parse their method name and arg list into DeclReferenceExpr and do not have a LabeledExprList. I'm inclined to create a second node to handle this (eg: .keyPathPartiallyAppliedComponent).

I’m not sure I understand. If there are no parentheses, we should be able to parse the component as a KeyPathPropertyComponentSyntax. If there are parentheses and no arguments, arguments can just be an empty list. And if there are arguments … then it doesn’t need to be optional.

I’m not sure I understand. If there are no parentheses, we should be able to parse the component as a KeyPathPropertyComponentSyntax. If there are parentheses and no arguments, arguments can just be an empty list. And if there are arguments … then it doesn’t need to be optional.

Should partial arguments should also be KeyPathPropertyComponentSyntax? method(_:) and method(arg:) for unapplied keypath methods are being parsed as DeclReferenceExpr with DeclNameArgumentListSyntax arguments instead of LabeledExprList. I have pushed changes that reflect this, unsure if there are any ramifications of doing it this way, but the Swift and c++ parser outputs match.

CodeGeneration/Sources/SyntaxSupport/ExprNodes.swift

Sources/SwiftParser/Expressions.swift

Tests/SwiftParserTest/ExpressionTests.swift

amritpan · 2025-02-04T23:06:00Z

Added additional tests to ensure Swift and c++ parsers match.

CodeGeneration/Sources/SyntaxSupport/ExprNodes.swift

Sources/SwiftParser/Expressions.swift

ahoppen

That looks great and is much simpler as well. Just two minor comments, everything else is great.

Sources/SwiftParser/Expressions.swift

Tests/SwiftParserTest/ExpressionTests.swift

Release Notes/602.md

ahoppen

Looks good to me 🚀

ahoppen · 2025-02-06T01:43:58Z

@swift-ci Please test

amritpan · 2025-02-06T21:04:43Z

Not sure why this test is failing - is it due to my imports in Expressions.swift?

ahoppen · 2025-02-06T21:39:46Z

Oh, I just didn’t test it with your PR in the compiler repo. Let’s try again.

swiftlang/swift#78823

@swift-ci Please test

ahoppen · 2025-02-06T22:29:27Z

Looks like you need to rebase this PR + re-generate the sources to adjust it for #2926.

amritpan · 2025-02-07T00:00:12Z

Rebased and regenerated, but running into this failure locally:

./swift-syntax-dev-utils local-pr-precheck
Building for debugging...
[1/1] Write swift-version--58304C5D6DBC2206.txt
Build of product 'swift-syntax-dev-utils' complete! (0.14s)
** Running code generation **
** Building target SwiftSyntax-all **
Building for debugging...
[55/55] Emitting module SwiftSyntaxBuilder
Build of target: 'SwiftSyntax-all' complete! (9.33s)
** Building target Examples-all **
Building for debugging...
error: couldn't build /.../swift-project/swift-syntax/Examples/.build/arm64-apple-macosx/debug/SwiftSyntax.build/sources because of missing inputs: /.../swift-project/swift-syntax/Sources/SwiftSyntax/Raw/RawSyntaxArena.swift, /.../swift-project/swift-syntax/Sources/SwiftSyntax/ArenaAllocatedBuffer.swift

ahoppen · 2025-02-07T01:20:34Z

Could you try if deleting Examples/.build fixes the issue you’re seeing locally?

ahoppen · 2025-02-07T01:20:49Z

swiftlang/swift#78823

@swift-ci Please test

ahoppen · 2025-02-07T02:10:26Z

swiftlang/swift#78823

@swift-ci Please test

amritpan · 2025-02-07T21:36:55Z

Yes, deleting Examples/.build fixed the issue locally.

amritpan · 2025-02-27T15:37:57Z

swiftlang/swift-foundation#1179

swiftlang/swift#78823

@swift-ci Please test windows platform.

amritpan · 2025-02-27T17:21:23Z

swiftlang/swift-foundation#1179

swiftlang/swift#78823

@swift-ci please test windows platform

amritpan · 2025-02-27T21:11:59Z

@ahoppen Unsure why the Windows platform test keeps failing.

ahoppen · 2025-02-27T21:19:02Z

Because the tests that you added to the swift repository are failing, specifically Interpreter/keypath.swift and SILGen/keypaths.swift. Looks like some kind of path formatting issue.

amritpan · 2025-02-27T21:25:35Z

Are you able to share any failure output? I only see "Build finished. No test results found." and nothing is failing for me locally.

bnbarham · 2025-02-28T05:24:27Z

Are you able to share any failure output? I only see "Build finished. No test results found." and nothing is failing for me locally.

The failing job was https://ci-external.swift.org/job/swift-syntax-PR-windows/4074/ with two test failures:

Failed Tests (2):
  Swift(windows-x86_64) :: Interpreter/keypath.swift
  Swift(windows-x86_64) :: SILGen/keypaths.swift

SILGen/keypaths.swift is because:

# | C:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\swift\test\SILGen\keypaths.swift:677:12: error: CHECK: expected string not found in input
# |  // CHECK: %0 = keypath $WritableKeyPath<K.Type, () -> K>, (objc "self"; root $K.Type; gettable_property $() -> K, id @$s8keypaths1KVACycfC : $@convention(method) (@thin K.Type) -> K, getter @$s8keypaths1KVACycfcACmTkm : $@convention(keypath_accessor_getter) (@in_guaranteed @thick K.Type) -> @out @callee_guaranteed @substituted <τ_0_0> () -> @out τ_0_0 for <K>)

Seems like the difference is the lack of objc "self".

Interpreter/keypath.swift is because:

# | C:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\swift\test\Interpreter\keypath.swift:240:11: error: CHECK: expected string not found in input
# | // CHECK: \S.Type.<computed {{.*}} (S)>.subscript(_: Int)

That one is less obvious to me though - I don't see a subscript (in the logged output anyway).

amritpan · 2025-03-05T13:57:54Z

Thank you. The above tests should now pass, but I have a new test in the compiler that is failing due to this swift-syntax PR. Is there a way to suppress Parser diagnostics for all keypath methods and initializers so that invalid expressions are only caught and diagnosed in semantic analysis?

This test preserving current diagnostics in the compiler fails as it forces me to add Parser diagnostics and expects the following instead:

_ = \Int.byteSwapped.signum()
  // expected-error@-1 {{key path cannot refer to instance method 'signum()'}}
  // expected-error@-2 {{consecutive statements on a line must be separated by newline or ';'}}
  // expected-note@-3 {{insert newline}}
  // expected-note@-4 {{insert ';'}}

I would like only the first semantic diagnostic.

This test is missing the compiler experimental flag that is related to this feature.

Can I add the swift-syntax changes in this PR without the swift-syntax experimental flag to prevent these diagnostics? @ahoppen @bnbarham

bnbarham · 2025-03-05T18:16:41Z

You can add -disable-experimental-parser-round-trip to this particular test case. I would have said that we should have a proper diagnostic here (to eg. add () as unexpected), but given that error wouldn't exist any more with your proposal, that's not really worth doing.

amritpan · 2025-03-05T18:59:40Z

@bnbarham Would it be // RUN: %target-typecheck-verify-swift -disable-experimental-parser-round-trip ? This is still producing the parser diagnostics.

bnbarham · 2025-03-05T19:18:41Z

Yeah, that should definitely work 🤔. Locally I see eg.

> swift-frontend t2.swift -typecheck
t2.swift:1:28: error: consecutive statements on a line must be separated by newline or ';'
1 | _ = \Int.byteSwapped.signum()
  |                            |- error: consecutive statements on a line must be separated by newline or ';'
  |                            |- note: insert newline
  |                            `- note: insert ';'
2 |

And then:

> swift-frontend t2.swift -typecheck -disable-experimental-parser-round-trip
t2.swift:1:22: error: invalid component of Swift key path
1 | _ = \Int.byteSwapped.signum()
  |                      `- error: invalid component of Swift key path
2 |

target-typecheck-verify-swift just expands to swift-frontend -typecheck -verify -disable-objc-attr-requires-foundation-module %s

amritpan · 2025-03-05T19:37:20Z

Still getting errors. I also tried the expanded version.

Could the experimental flags be causing issues here?

mapFeature(.KeyPathWithMethodMembers, to: .keypathWithMethodMembers)

ahoppen · 2025-03-05T21:00:02Z

What are the errors you’re seeing now? It might be easiest if you could push your latest revisions.

amritpan · 2025-03-05T23:40:06Z

~~The errors are the same as before adding the flag. Here is full output:~~ I've added -disable-experimental-parser-round-trip to the test and pushed changes as well.

My mistake: I have no idea what I was doing before but this test now passes! Running CI now...

amritpan · 2025-03-05T23:54:48Z

swiftlang/swift#78823

@swift-ci Please test.

amritpan · 2025-03-06T04:02:26Z

swiftlang/swift#78823

@swift-ci Please test.

amritpan · 2025-03-06T15:57:54Z

swiftlang/swift#78823

@swift-ci Please test.

amritpan · 2025-03-06T19:53:05Z

Unsure if I'm looking at the correct part of the failure output for the Windows test, but I don't see any failed tests. I see this:

FAILED: tools/clang/lib/AST/CMakeFiles/obj.clangAST.dir/ExprObjC.cpp.obj 
C:\PROGRA~1\MICROS~2\2022\COMMUN~1\VC\Tools\MSVC\1442~1.344\bin\Hostx64\x64\cl.exe  /nologo /TP -DGTEST_HAS_RTTI=0 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_GLIBCXX_ASSERTIONS -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -IT:\5\tools\clang\lib\AST -IC:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\llvm-project\clang\lib\AST -IC:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\llvm-project\clang\include -IT:\5\tools\clang\include -IT:\5\include -IC:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\llvm-project\llvm\include /GS- /Gw /Gy /Oi /Oy /Zc:inline /Zc:__cplusplus /Zc:inline /Zc:preprocessor /Zc:__cplusplus /Oi /bigobj /permissive- /W4 -wd4141 -wd4146 -wd4244 -wd4267 -wd4291 -wd4351 -wd4456 -wd4457 -wd4458 -wd4459 -wd4503 -wd4624 -wd4722 -wd4100 -wd4127 -wd4512 -wd4505 -wd4610 -wd4510 -wd4702 -wd4245 -wd4706 -wd4310 -wd4701 -wd4703 -wd4389 -wd4611 -wd4805 -wd4204 -wd4577 -wd4091 -wd4592 -wd4319 -wd4709 -wd5105 -wd4324 -w14062 -we4238 /Gw /O2 /Ob2  -std:c++17 -MD  /EHs-c- /GR- -UNDEBUG /showIncludes /Fotools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ExprObjC.cpp.obj /Fdtools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ /FS -c C:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\llvm-project\clang\lib\AST\ExprObjC.cpp
C:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\llvm-project\clang\lib\AST\ExprObjC.cpp(360): error C3861: 'bzero': identifier not found

but I'm unsure how to fix this. Do I need to rebase llvm or clang? @ahoppen

ahoppen · 2025-03-06T22:57:29Z

That’s an unrelated test failure which was fixed by swiftlang/llvm-project#10189. Let’s try again

@swift-ci Please test Windows

amritpan · 2025-03-07T16:31:15Z

Do update-checkout errors need a rebase? I'm now seeing:

Error on repo "C:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\llvm-project": Traceback (most recent call last):
  File "C:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\swift\utils\update_checkout\update_checkout\update_checkout.py", line 167, in update_single_repository
    config, repo_name, scheme_name, scheme_map, cross_repos_pr)
  File "C:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\swift\utils\update_checkout\update_checkout\update_checkout.py", line 145, in get_branch_for_repo
    .format(pr_id, repo_branch), "--tags"], echo=True)
  File "C:\Users\swift-ci\jenkins\workspace\swift-syntax-PR-windows\swift\utils\swift_build_support\swift_build_support\shell.py", line 257, in run
    raise eout
Exception: ['git', 'fetch', 'origin', 'pull/10189/merge:ci_pr_10189', '--tags']

ahoppen · 2025-03-07T17:53:46Z

No, seems to be some infrastructure issue. Let’s try again. And sorry for those unrelated CI failures.

@swift-ci Please test Windows

amritpan · 2025-03-07T20:44:14Z

Oh, we forgot to link the compiler PR.

swiftlang/swift#78823

@swift-ci Please test Windows.

amritpan · 2025-03-09T02:04:16Z

swiftlang/swift#78823

@swift-ci Please test Windows.

amritpan · 2025-03-09T06:52:51Z

swiftlang/swift#78823

@swift-ci Please test Windows.

amritpan requested review from ahoppen and bnbarham as code owners January 22, 2025 02:48

ahoppen reviewed Jan 22, 2025

View reviewed changes

amritpan mentioned this pull request Jan 22, 2025

[Sema/SILGen/IRGen] Implement method & initializer keypaths. swiftlang/swift#78823

Merged

amritpan force-pushed the method-keypaths branch from e0c6ccb to 7762c01 Compare January 30, 2025 20:06

amritpan requested a review from ahoppen January 30, 2025 20:39

amritpan force-pushed the method-keypaths branch 2 times, most recently from dfa57bf to dd259f6 Compare February 4, 2025 23:05

ahoppen reviewed Feb 4, 2025

View reviewed changes

amritpan force-pushed the method-keypaths branch 4 times, most recently from 6cf481b to 6790783 Compare February 5, 2025 23:06

amritpan requested a review from ahoppen February 5, 2025 23:06

ahoppen reviewed Feb 6, 2025

View reviewed changes

Sources/SwiftParser/Expressions.swift Show resolved Hide resolved

Tests/SwiftParserTest/ExpressionTests.swift Outdated Show resolved Hide resolved

Release Notes/602.md Outdated Show resolved Hide resolved

amritpan force-pushed the method-keypaths branch from 6790783 to fa3596e Compare February 6, 2025 01:31

amritpan requested a review from ahoppen February 6, 2025 01:32

ahoppen approved these changes Feb 6, 2025

View reviewed changes

amritpan force-pushed the method-keypaths branch from fa3596e to 5c3310a Compare February 6, 2025 23:57

amritpan requested a review from ahoppen February 7, 2025 00:00

amritpan added 3 commits March 2, 2025 17:22

Add experimental flag and keyPathMethodComponent node.

b97ca6e

Add supporting code.

9b5dc51

Parse method and initializer keypaths.

8b5df29

amritpan force-pushed the method-keypaths branch from 646b14f to 8b5df29 Compare March 5, 2025 13:49

amritpan merged commit ae69dd5 into swiftlang:main Mar 9, 2025
3 checks passed

DougGregor mentioned this pull request Mar 9, 2025

[ASTGen] Stub out support for method keypaths swiftlang/swift#79873

Merged

amritpan deleted the method-keypaths branch March 10, 2025 17:03

Add keypath method and initializer syntax under an experimental feature flag keypathWithMethodMembers #2950

Add keypath method and initializer syntax under an experimental feature flag keypathWithMethodMembers #2950

Conversation

amritpan commented Jan 22, 2025 • edited Loading

ahoppen left a comment

Choose a reason for hiding this comment

ahoppen Jan 22, 2025

Choose a reason for hiding this comment

amritpan Jan 28, 2025

Choose a reason for hiding this comment

ahoppen Jan 29, 2025

Choose a reason for hiding this comment

amritpan Jan 30, 2025

Choose a reason for hiding this comment

amritpan commented Feb 4, 2025

ahoppen left a comment

Choose a reason for hiding this comment

ahoppen left a comment

Choose a reason for hiding this comment

ahoppen commented Feb 6, 2025

amritpan commented Feb 6, 2025

ahoppen commented Feb 6, 2025

ahoppen commented Feb 6, 2025

amritpan commented Feb 7, 2025

ahoppen commented Feb 7, 2025

ahoppen commented Feb 7, 2025

ahoppen commented Feb 7, 2025

amritpan commented Feb 7, 2025

amritpan commented Feb 27, 2025

amritpan commented Feb 27, 2025

amritpan commented Feb 27, 2025

ahoppen commented Feb 27, 2025

amritpan commented Feb 27, 2025

bnbarham commented Feb 28, 2025

amritpan commented Mar 5, 2025 • edited Loading

bnbarham commented Mar 5, 2025

amritpan commented Mar 5, 2025

bnbarham commented Mar 5, 2025

amritpan commented Mar 5, 2025

ahoppen commented Mar 5, 2025

amritpan commented Mar 5, 2025 • edited Loading

amritpan commented Mar 5, 2025

amritpan commented Mar 6, 2025

amritpan commented Mar 6, 2025

amritpan commented Mar 6, 2025

ahoppen commented Mar 6, 2025

amritpan commented Mar 7, 2025

ahoppen commented Mar 7, 2025

amritpan commented Mar 7, 2025

amritpan commented Mar 9, 2025

amritpan commented Mar 9, 2025

Add keypath method and initializer syntax under an experimental feature flag `keypathWithMethodMembers` #2950

Add keypath method and initializer syntax under an experimental feature flag `keypathWithMethodMembers` #2950

amritpan commented Jan 22, 2025 •

edited

Loading

amritpan commented Mar 5, 2025 •

edited

Loading

amritpan commented Mar 5, 2025 •

edited

Loading