Set OtherLDFlags for dynamic framework type #206

S-Shimotori · 2025-04-14T13:30:35Z

Closes #116

This is a draft that contains FIXME and doesn't have required tests.
I created this PR to disclose codes given by ikesyo san. I will close this issue if anyone can implement completely.

Issue

According to #116, current Scipio cannot create dynamic frameworks for these libraries:

Atomics in swift-atomics
NIOConcurrencyHelpers in swift-nio

The linker needs dependencies with -framework flag.

Idea

PIFGenerator

Pass -framework to OTHER_LDFLAGS when frameworkType=dynamic
- Thank you ikesyo san
- I need to solve swiftlint warning

Tests

I think these tests are required but I'm sorry couldn't create them by myself yet.

framework type condition

static

-

Scipio/Tests/ScipioKitTests/IntegrationTests.swift

Line 38 in dd8e118

"Atomics": .init(frameworkType: .static),

mergeable - Current Scipio seems not to be able to handle "Atomics": .init(frameworkType: .mergeable)

dynamic - swift-atomics and swift-nio are this type. We can test this type with swift-atomics or swift-nio at least.
~~I tried to create a package that contains a simple internal target, but current Scipio can create XCFramework without -framework. Maybe we should prepare a special target (e.g. Clang module?)~~
[2025-05-01] It seems that the linker requires -framework for @usableFromInline and Clang module.

dynamic platform condition We need to make PIFLibraryTargetModifier internal for checking OTHER_LDFLAGS.

dynamic configuration condition not available yet (SE-0273)

dynamic trait condition SE-0450

I think "OtherLDFlagsTestPackage" is not a good name but I need to decide what tests we need before giving name.

S-Shimotori · 2025-04-29T14:58:29Z

I used swift-atomics for test on cd10ffc, but I found Scipio fails with @usableFromInline and a C function.

(package)/
┣━━ Package.swift
┗━━ Sources/
    ┣━━ (Clang module target)/
    ┃   ┣━ include/
    ┃   ┃  ┣━ (C header file)
    ┃   ┃  ┗━ module.modulemap
    ┃   ┗━ src/
    ┃      ┗━ (C source file)
    ┗━━ (swift target)/
        ┗━ (Swift source file)

void this_is_c_function();

#include "ClangModule.h"

void this_is_c_function() {
}

import ClangModule

@usableFromInline
struct Sample {
    @usableFromInline
    static func sample() {
        this_is_c_function()
    }
}

Please see 6bf01ab for the concrete example.

S-Shimotori · 2025-04-30T07:08:07Z

We can check which libraries created xcframeworks depends on with otool -l.

Load command 10
          cmd LC_LOAD_DYLIB
      cmdsize 96
         name @rpath/ClangModuleForMacOS.framework/Versions/A/ClangModuleForMacOS (offset 24)

I think we need to make PIFGenerator or PIFLibraryTargetModifier internal to confirm that they pass minimum required -framework flags for each platform.

S-Shimotori · 2025-05-02T13:50:15Z

Summary

Cause of Issue

The linker cannot link Clang modules for @usableFromInline by itself.

Changes

PIFGenerator.swift

PIFLibraryTargetModifier will make lists of dependencies and pass them with -framework to OTHER_LDFLAGS.
I refactored PIFGenerator.swift to reduce the number of lines.
I left FIXME comment for trait condition. I'm sorry, but I would be happy if someone familiar with trait condition could implement it.

DynamicFrameworkTests.swift

DynamicFrameworkTests checks whether a generated xcframework contains minimum necessary dependencies.
- Ideally, we should check the list of -frameworks in OTHER_LDFLAGS because the generated xcframework won't contain unnecessary dependencies even if Scipio passes additional modules.

IntegrationTests.swift

testDynamicFramework() checks whether Scipio can generate xcframeworks from a package that contains platform condition and @usableFromInline.

UsableFromInlinePackage, DynamicFrameworkOtherLDFlagsTestPackage

This package contains not only @usableFromInline but also platform condition (condition: .when(platforms:)) for coverage.

giginet · 2025-06-03T05:27:01Z

@S-Shimotori Sorry for waiting you. It's ready to review. Could you rebase the current main?

S-Shimotori · 2025-06-21T05:47:50Z

@giginet @Ryu0118

I have to fix this PR because it still depends on SwiftPM, not PIFKit.

Could you review my idea and give me some advice?

What I should do to merge PIFKit logic into this PR

Retrieve dependencies

When `PIFGenerator` depended on SwiftPM

descriptionPackage.graph.allModules.first(where: { $0.c99name == c99Name })

Scipio/Sources/ScipioKit/Producer/PIF/PIFGenerator.swift

Line 172 in 2be9610

    
           guard let resolvedTarget = descriptionPackage.graph.allModules.first(where: { $0.c99name == c99Name }) else {

PIFCompiler passed descriptionPackage: DescriptionPackage to PIFGenerator, but it was deleted with #183.

What to do with PIFKit

Pass ResolvedModule to PIFGenerator.init(...)
Pass c99Name to PIFGenerator.init(...)
- Maybe buildProduct.target.c99name in createXCFramework(buildProduct:outputDirectory:overwrite:) ?

Configure build settings

When `PIFGenerator` depended on SwiftPM

Scipio used PIF.BuildSettings as a container.

PIF.BuildSettings has consists of some dictionaries for platform-specific settings.
c.f. swift-package-manager/Sources/XCBuildSupport/PIF.swift at 29e562052de0305417c9bd71d8a7c64bee2759fb · swiftlang/swift-package-manager

What to do with PIFKit

PIFKit.BuildConfiguration's buildSettings is a simple dictionary.

Option 1: Scipio doesn't consider PackageCondition nor set platform-specific values there.
Option 2: PIFGenerator configures them somehow to handle PackageConditions.

Do you have any ideas? If we choose the latter option, how should I update updateBuildConfiguration(_:) ?

giginet · 2025-06-27T12:22:32Z

@S-Shimotori I'm sorry for the long wait.

As you mentioned, we should configure settings considering the Platform Filter, but PIFKit currently does not have that interface.

As far as I understand, the new ResolvedModule also has an interface to retrieve the PlatformFilter.
Since we only need an interface to set __platform_filter for PIFKit, I added it in #225.

With these, I believe we can reproduce the original implementation. What do you think?

S-Shimotori · 2025-07-04T07:57:48Z

@giginet

Thank you. I updated this PR with #225

giginet · 2025-06-03T05:28:50Z

Sources/ScipioKit/Producer/PIF/PIFGenerator.swift

+        case .mergeable:
            settings[.OTHER_LDFLAGS, default: ["$(inherited)"]]
                .append("-Wl,-make_mergeable")
+        case .dynamic:


I believe we need to take the same path for mergeable as well.

giginet · 2025-06-03T05:34:01Z

Tests/ScipioKitTests/DynamicFrameworkTests.swift

+private let fixturePath = URL(fileURLWithPath: #filePath)
+    .deletingLastPathComponent()
+    .appendingPathComponent("Resources")
+    .appendingPathComponent("Fixtures")


[nits] Please use the modern API

Suggested change

private let fixturePath = URL(fileURLWithPath: #filePath)

.deletingLastPathComponent()

.appendingPathComponent("Resources")

.appendingPathComponent("Fixtures")

private let fixturePath = URL(filePath: #filePath)

.deletingLastPathComponent()

.appending(components: "Resources", "Fixtures")

giginet · 2025-07-08T04:00:27Z

Sources/ScipioKit/Producer/PIF/PIFGenerator.swift

+        case .mergeable:
            configuration.buildSettings["OTHER_LDFLAGS"]
                .append("-Wl,-make_mergeable")
+        case .dynamic:


These flag settings will be required also for mergeable

giginet · 2025-07-08T04:02:06Z

Tests/ScipioKitTests/DynamicFrameworkTests.swift

+        let outputDir = fileManager.temporaryDirectory
+            .appendingPathComponent("Scipio")
+            .appendingPathComponent(packageName)


Suggested change

let outputDir = fileManager.temporaryDirectory

.appendingPathComponent("Scipio")

.appendingPathComponent(packageName)

let outputDir = fileManager.temporaryDirectory

.appending(components: "Scipio", packageName)

giginet · 2025-07-08T04:02:20Z

Tests/ScipioKitTests/DynamicFrameworkTests.swift

+            .appendingPathComponent(packageName)
+
+        defer {
+            print("remove output directory: \(outputDir.path)")


giginet · 2025-07-08T04:03:14Z

Tests/ScipioKitTests/DynamicFrameworkTests.swift

+                verbose: false
+            )
+        )
+        let packageDir = fixturePath.appendingPathComponent(packageName)


Suggested change

let packageDir = fixturePath.appendingPathComponent(packageName)

let packageDir = fixturePath.appending(component: packageName)

giginet · 2025-07-08T04:03:28Z

Tests/ScipioKitTests/DynamicFrameworkTests.swift

+        dependencies: [String: Set<Destination>],
+        outputDir: URL
+    ) async throws {
+        let xcFrameworkPath = outputDir.appendingPathComponent(framework.xcFrameworkName)


Suggested change

let xcFrameworkPath = outputDir.appendingPathComponent(framework.xcFrameworkName)

let xcFrameworkPath = outputDir.appending(component: framework.xcFrameworkName)

giginet · 2025-07-08T04:04:01Z

Tests/ScipioKitTests/DynamicFrameworkTests.swift

+            let binaryPath = xcFrameworkPath
+                .appendingPathComponent(arch.rawValue)
+                .appendingPathComponent("\(framework.name).framework")
+                .appendingPathComponent(framework.name)


Suggested change

let binaryPath = xcFrameworkPath

.appendingPathComponent(arch.rawValue)

.appendingPathComponent("\(framework.name).framework")

.appendingPathComponent(framework.name)

let binaryPath = xcFrameworkPath

.appending(components: arch.rawValue, "\(framework.name).framework", framework.name)

S-Shimotori · 2025-07-13T10:57:23Z

Changes:

Added fallthrough to apply this logic to mergeable type
Improve file path
Added test to check both dynamic and mergeable

giginet · 2025-07-14T02:30:30Z

Tests/ScipioKitTests/DynamicFrameworkTests.swift

+    private let fileManager: FileManager = .default
+
+    @Test(
+        arguments: [FrameworkType.dynamic, .mergeable]


Set OtherLDFlags for dynamic framework type

cd10ffc

ikesyo mentioned this pull request Apr 15, 2025

XCFramework fails to build when depending on SwiftNIO #205

Closed

Use local package instead of open source for test

6bf01ab

Merge remote-tracking branch 'upstream/main' into issue/116

b14091a

S-Shimotori added 8 commits April 30, 2025 23:46

Add platform condition test

11054ec

Refactor

f7907c9

Refactor

28b3fe0

Insert flag for Xcode 16.2

01d4bc0

Downgrade test package for Xcode 16.2

f6b0328

Fix test package

c840696

Rename

32ee48a

Add test failure message

b558dfa

Simplify test case

a8d3913

S-Shimotori marked this pull request as ready for review May 4, 2025 02:25

giginet requested review from giginet and ikesyo June 3, 2025 05:26

Merge remote-tracking branch 'upstream/main' into issue/116

6dd486c

This was referenced Jun 22, 2025

Could not build IGListKit as dynamic/mergeable framework #217

Closed

Add feature to set platform_filter for PIF #225

Merged

S-Shimotori added 4 commits July 4, 2025 12:26

Merge remote-tracking branch 'upstream/main' into issue/116

9f20164

Support platform condition using PIFKit

43849ac

Fix lint warnings

2dc5ce6

Remove unnecessary serialized config from test

c5aa23d

giginet reviewed Jul 8, 2025

View reviewed changes

S-Shimotori added 5 commits July 13, 2025 18:44

Refactor

21205c7

Apply "-framework" logic to mergeable framework

ec185db

Merge remote-tracking branch 'upstream/main' into issue/116

eb83025

Fix "import ScipioKit" in test

37b1ed6

Add test to check OTHER_LDFLAGS for mergeable

aa35631

giginet reviewed Jul 14, 2025

View reviewed changes

giginet approved these changes Jul 14, 2025

View reviewed changes

	let packageDir = fixturePath.appendingPathComponent(packageName)
	let packageDir = fixturePath.appending(component: packageName)

	let xcFrameworkPath = outputDir.appendingPathComponent(framework.xcFrameworkName)
	let xcFrameworkPath = outputDir.appending(component: framework.xcFrameworkName)

Set OtherLDFlags for dynamic framework type #206

Are you sure you want to change the base?

Set OtherLDFlags for dynamic framework type #206

Uh oh!

Conversation

S-Shimotori commented Apr 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue

Idea

PIFGenerator

Tests

Uh oh!

S-Shimotori commented Apr 29, 2025

Uh oh!

S-Shimotori commented Apr 30, 2025

Uh oh!

S-Shimotori commented May 2, 2025

Summary

Cause of Issue

Changes

PIFGenerator.swift

DynamicFrameworkTests.swift

IntegrationTests.swift

UsableFromInlinePackage, DynamicFrameworkOtherLDFlagsTestPackage

Uh oh!

giginet commented Jun 3, 2025

Uh oh!

S-Shimotori commented Jun 21, 2025

What I should do to merge PIFKit logic into this PR

Retrieve dependencies

When PIFGenerator depended on SwiftPM

What to do with PIFKit

Configure build settings

When PIFGenerator depended on SwiftPM

What to do with PIFKit

Uh oh!

giginet commented Jun 27, 2025

Uh oh!

S-Shimotori commented Jul 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

S-Shimotori commented Jul 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

S-Shimotori commented Apr 14, 2025 •

edited

Loading

When `PIFGenerator` depended on SwiftPM

When `PIFGenerator` depended on SwiftPM