Override API connection via step inputs (#303)

* Override API connection via step inputs

* Refactor

* Fix method name

* Unset test app commit env var

* Remove commit ref from test

* Update go-xcode

* Clear Bitrise API envs for testing

* yamllint fix

* Fix script

* Update readme with connection override inputs

* Don't try connection if API env vars are not defined

* Add debug log statements

* Debug

* Debug

* Fix nil pointer dereference

* Remove debug logs

Author: ofalvai

README.md

@@ -42,6 +42,8 @@ Under **Automatic code signing**:
 3. **The minimum days the Provisioning Profile should be valid**: If this input is set to >0, the managed Provisioning Profile will be renewed if it expires within the configured number of days. Otherwise the Step renews the managed Provisioning Profile if it is expired.
 4. The **Code signing certificate URL**, the **Code signing certificate passphrase**, the **Keychain path**, and the **Keychain password** inputs are automatically populated if certificates are uploaded to Bitrise's **Code Signing** tab. If you store your files in a private repo, you can manually edit these fields.
 
+If you want to set the Apple service connection credentials on the step-level (instead of using the one configured in the App Settings), use the Step inputs in the **App Store Connect connection override** category. Note that this only works if **Automatic code signing method** is set to `api-key`.
+
 Under **IPA export configuration**:
 1. **Developer Portal team**: Add the Developer Portal team's name to use for this export. This input defaults to the team used to build the archive.
 2. **Rebuild from bitcode**: For non-App Store exports, should Xcode re-compile the app from bitcode?
@@ -131,6 +133,9 @@ Build a development IPA with custom xcconfig file path:
 | `export_all_dsyms` | Export additional dSYM files besides the app dSYM file for Frameworks. | required | `yes` |
 | `artifact_name` | This name will be used as basename for the generated Xcode Archive, App, IPA and dSYM files.  If not specified, the Product Name (`PRODUCT_NAME`) Build settings value will be used. If Product Name is not specified, the Scheme will be used. |  |  |
 | `cache_level` | Defines what cache content should be automatically collected.  Available options:  - `none`: Disable collecting cache content - `swift_packages`: Collect Swift PM packages added to the Xcode project | required | `swift_packages` |
+| `api_key_path` | Local path or remote URL to the private key (p8 file) for App Store Connect API. This overrides the Bitrise-managed API connection, only set this input if you want to control the API connection on a step-level. Most of the time it's easier to set up the connection on the App Settings page on Bitrise. The input value can be a file path (eg. `$TMPDIR/private_key.p8`) or an HTTPS URL. This input only takes effect if the other two connection override inputs are set too (`api_key_id`, `api_key_issuer_id`). |  |  |
+| `api_key_id` | Private key ID used for App Store Connect authentication. This overrides the Bitrise-managed API connection, only set this input if you want to control the API connection on a step-level. Most of the time it's easier to set up the connection on the App Settings page on Bitrise. This input only takes effect if the other two connection override inputs are set too (`api_key_path`, `api_key_issuer_id`). |  |  |
+| `api_key_issuer_id` | Private key issuer ID used for App Store Connect authentication. This overrides the Bitrise-managed API connection, only set this input if you want to control the API connection on a step-level. Most of the time it's easier to set up the connection on the App Settings page on Bitrise. This input only takes effect if the other two connection override inputs are set too (`api_key_path`, `api_key_id`). |  |  |
 | `verbose_log` | If this input is set, the Step will print additional logs for debugging. | required | `no` |
 </details>
 

e2e/bitrise.yml

@@ -14,9 +14,44 @@ app:
   - BITFALL_APPLE_PROVISIONING_PROFILE_URL_LIST: $BITFALL_APPLE_PROVISIONING_PROFILE_URL_LIST
 
   - BITFALL_APPSTORECONNECT_API_KEY_URL: $BITFALL_APPSTORECONNECT_API_KEY_URL
+  - BITFALL_APPSTORECONNECT_API_KEY_ID: $BITFALL_APPSTORECONNECT_API_KEY_ID
   - BITFALL_APPSTORECONNECT_API_KEY_ISSUER_ID: $BITFALL_APPSTORECONNECT_API_KEY_ISSUER_ID
 
 workflows:
+
+  test_override_api_key_signing:
+    description: Test API key based signing on a project where code signing is not managed automatically, by step inputs
+    envs:
+    - TEST_APP_URL: https://github.com/bitrise-io/Fruta
+    - TEST_APP_BRANCH: manual-signing
+    - TEST_APP_COMMIT: ""
+    - BITRISE_PROJECT_PATH: Fruta.xcodeproj
+    - BITRISE_SCHEME: Fruta iOS
+    - CODE_SIGNING_METHOD: api-key
+    - API_KEY_PATH: $BITFALL_APPSTORECONNECT_API_KEY_URL
+    - API_KEY_ID: $BITFALL_APPSTORECONNECT_API_KEY_ID
+    - API_KEY_ISSUER_ID: $BITFALL_APPSTORECONNECT_API_KEY_ISSUER_ID
+    - MIN_DAYS_PROFILE_VALID: 0
+    - IPA_EXPORT_METHOD: development
+    - LOG_FORMATTER: xcodebuild
+    steps:
+    - script:
+        title: Unset Bitrise API connection env vars
+        inputs:
+        - content: |-
+            #!/usr/bin/env bash
+            set -ex
+            envman add --key BITRISE_BUILD_URL_BACKUP --value $BITRISE_BUILD_URL
+            envman add --key BITRISE_BUILD_API_TOKEN_BACKUP --value $BITRISE_BUILD_API_TOKEN
+            envman unset --key BITRISE_BUILD_URL
+            envman unset --key BITRISE_BUILD_API_TOKEN
+
+    after_run:
+    - _run
+    - _check_outputs
+    - _check_exported_artifacts
+    - _restore_api_envs
+
   test_single_certificate_no_passphrase:
     steps:
     - script:
@@ -412,6 +447,9 @@ workflows:
         - keychain_path: $BITRISE_KEYCHAIN_PATH
         - keychain_password: $BITRISE_KEYCHAIN_PASSWORD
         - fallback_provisioning_profile_url_list: $FALLBACK_PROFILES
+        - api_key_path: $API_KEY_PATH
+        - api_key_id: $API_KEY_ID
+        - api_key_issuer_id: $API_KEY_ISSUER_ID
 
   _check_outputs:
     steps:
@@ -540,3 +578,13 @@ workflows:
             xcode_major_version=${BASH_REMATCH[1]}
             echo "Xcode major version: $xcode_major_version"
             envman add --key XCODE_MAJOR_VERSION --value $xcode_major_version
+
+  _restore_api_envs:
+    steps:
+    - script:
+        title: Restore Bitrise API connection env vars
+        inputs:
+        - content: |-
+            set -ex
+            envman add --key BITRISE_BUILD_URL --value $BITRISE_BUILD_URL_BACKUP
+            envman add --key BITRISE_BUILD_API_TOKEN --value $BITRISE_BUILD_API_TOKEN_BACKUP

go.mod

@@ -5,13 +5,12 @@ go 1.16
 require (
 	github.com/bitrise-io/go-steputils/v2 v2.0.0-alpha.2
 	github.com/bitrise-io/go-utils v1.0.2
-	github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.7
-	github.com/bitrise-io/go-xcode v1.0.8
-	github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.17
+	github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.11
+	github.com/bitrise-io/go-xcode v1.0.9
+	github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.20
 	github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
 	github.com/stretchr/testify v1.7.1
 	golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect
-	golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a // indirect
 	golang.org/x/term v0.0.0-20220526004731-065cf7ba2467 // indirect
 	gopkg.in/yaml.v3 v3.0.1
 	howett.net/plist v1.0.0

go.sum

@@ -9,13 +9,12 @@ github.com/bitrise-io/go-utils v1.0.1/go.mod h1:ZY1DI+fEpZuFpO9szgDeICM4QbqoWVt0
 github.com/bitrise-io/go-utils v1.0.2 h1:w4Mz2IvrgDzrFJECuHdvsK1LHO30cdtuy9bBa7Lw2c0=
 github.com/bitrise-io/go-utils v1.0.2/go.mod h1:ZY1DI+fEpZuFpO9szgDeICM4QbqoWVt0RSY3tRI1heY=
 github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.1/go.mod h1:sy+Ir1X8P3tAAx/qU/r+hqDjHDcrMjIzDEvId1wqNc4=
-github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.7 h1:d0XDESvQwOO+V9afZrI8QGR7bJGDkmE4Q9ezIBB4TLw=
-github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.7/go.mod h1:6i0Gt0JRIbXpsrFDJT1YWghFfdN8qF26/fnpc/6d/88=
-github.com/bitrise-io/go-xcode v1.0.6/go.mod h1:Y0Wu2dXm0MilJ/4D3+gPHaNMlUcP+1DjIPoLPykq7wY=
-github.com/bitrise-io/go-xcode v1.0.8 h1:fNB5TmbfsqWRh5ibtKdFdxPEua8gaNBmDt1rPM/GDuo=
-github.com/bitrise-io/go-xcode v1.0.8/go.mod h1:Y0Wu2dXm0MilJ/4D3+gPHaNMlUcP+1DjIPoLPykq7wY=
-github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.17 h1:aLlblow6vgnFKBRJRPz03O/M6x6LYY57wDkRCh66XL0=
-github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.17/go.mod h1:oYILBt4j8jn69avylknuqsUO/BSiRx9i+JaxcNOtWMA=
+github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.11 h1:IacLMHL7hhgVcqtx15Bysq738P8FRCp6ckGk1NvioWo=
+github.com/bitrise-io/go-utils/v2 v2.0.0-alpha.11/go.mod h1:SJqGxzwjIAx2LVQxNGS4taN7X//eDPJLrFxJ1MpOuyA=
+github.com/bitrise-io/go-xcode v1.0.9 h1:+sbqOYidQ+aiFfCTDpf2LdGSQEM5RfbtDsiG27zJG+s=
+github.com/bitrise-io/go-xcode v1.0.9/go.mod h1:Y0Wu2dXm0MilJ/4D3+gPHaNMlUcP+1DjIPoLPykq7wY=
+github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.20 h1:MIH3eGNcAsc5VBACiU+EFcmUfqCyT6/fMSi2UjYR9+s=
+github.com/bitrise-io/go-xcode/v2 v2.0.0-alpha.20/go.mod h1:8WBcRgrVXY8tzR7NcjE4fw6WguOIfB3YcC7ZTcQYUEY=
 github.com/bitrise-io/pkcs12 v0.0.0-20211108084543-e52728e011c8 h1:kmvU8AxrNTxXsVPKepBHD8W+eCVmeaKyTkRuUJB2K38=
 github.com/bitrise-io/pkcs12 v0.0.0-20211108084543-e52728e011c8/go.mod h1:UiXKNs0essbC14a2TvGlnUKo9isP9m4guPrp8KJHJpU=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -74,8 +73,8 @@ golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211205182925-97ca703d548d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a h1:dGzPydgVsqGcTRVwiLJ1jVbufYwmzD3LfVPLKsKg+0k=
-golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220712014510-0a85c31ab51e h1:NHvCuwuS43lGnYhten69ZWqi2QOj/CiDNcKbVqwVoew=
+golang.org/x/sys v0.0.0-20220712014510-0a85c31ab51e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/term v0.0.0-20220526004731-065cf7ba2467 h1:CBpWXWQpIRjzmkkA+M7q9Fqnwd2mZr3AFqexg8YTfoM=

main.go

@@ -106,6 +106,9 @@ type Inputs struct {
 	RegisterTestDevices             bool            `env:"register_test_devices,opt[yes,no]"`
 	MinDaysProfileValid             int             `env:"min_profile_validity,required"`
 	FallbackProvisioningProfileURLs string          `env:"fallback_provisioning_profile_url_list"`
+	APIKeyPath                      stepconf.Secret `env:"api_key_path"`
+	APIKeyID                        string          `env:"api_key_id"`
+	APIKeyIssuerID                  string          `env:"api_key_issuer_id"`
 	BuildURL                        string          `env:"BITRISE_BUILD_URL"`
 	BuildAPIToken                   stepconf.Secret `env:"BITRISE_BUILD_API_TOKEN"`
 }
@@ -355,15 +358,22 @@ func (s XcodeArchiveStep) createCodesignManager(config Config) (codesign.Manager
 		return codesign.Manager{}, fmt.Errorf("issue with input: %s", err)
 	}
 
-	var serviceConnection *devportalservice.AppleDeveloperConnection = nil
 	devPortalClientFactory := devportalclient.NewFactory(logger)
-	if authType == codesign.APIKeyAuth || authType == codesign.AppleIDAuth {
+
+	var serviceConnection *devportalservice.AppleDeveloperConnection = nil
+	if config.BuildURL != "" && config.BuildAPIToken != "" {
 		if serviceConnection, err = devPortalClientFactory.CreateBitriseConnection(config.BuildURL, string(config.BuildAPIToken)); err != nil {
 			return codesign.Manager{}, err
 		}
 	}
 
-	appleAuthCredentials, err := codesign.SelectConnectionCredentials(authType, serviceConnection, logger)
+	connectionInputs := codesign.ConnectionOverrideInputs{
+		APIKeyPath:     config.Inputs.APIKeyPath,
+		APIKeyID:       config.Inputs.APIKeyID,
+		APIKeyIssuerID: config.Inputs.APIKeyIssuerID,
+	}
+
+	appleAuthCredentials, err := codesign.SelectConnectionCredentials(authType, serviceConnection, connectionInputs, logger)
 	if err != nil {
 		return codesign.Manager{}, err
 	}
@@ -390,10 +400,14 @@ func (s XcodeArchiveStep) createCodesignManager(config Config) (codesign.Manager
 	}
 
 	client := retry.NewHTTPClient().StandardClient()
+	var testDevices []devportalservice.TestDevice
+	if serviceConnection != nil {
+		testDevices = serviceConnection.TestDevices
+	}
 	return codesign.NewManagerWithProject(
 		opts,
 		appleAuthCredentials,
-		serviceConnection,
+		testDevices,
 		devPortalClientFactory,
 		certdownloader.NewDownloader(codesignConfig.CertificatesAndPassphrases, client),
 		profiledownloader.New(codesignConfig.FallbackProvisioningProfiles, client),
@@ -509,24 +523,17 @@ func (s XcodeArchiveStep) xcodeArchive(opts xcodeArchiveOpts) (xcodeArchiveOutpu
 	logger.Println()
 	logger.TInfof("Creating the Archive ...")
 
-	isWorkspace := false
-	ext := filepath.Ext(opts.ProjectPath)
-	if ext == ".xcodeproj" {
-		isWorkspace = false
-	} else if ext == ".xcworkspace" {
-		isWorkspace = true
+	var actions []string
+	if opts.PerformCleanAction {
+		actions = []string{"clean", "archive"}
 	} else {
-		return out, fmt.Errorf("project file extension should be .xcodeproj or .xcworkspace, but got: %s", ext)
+		actions = []string{"archive"}
 	}
 
-	archiveCmd := xcodebuild.NewCommandBuilder(opts.ProjectPath, isWorkspace, xcodebuild.ArchiveAction)
+	archiveCmd := xcodebuild.NewCommandBuilder(opts.ProjectPath, actions...)
 	archiveCmd.SetScheme(opts.Scheme)
 	archiveCmd.SetConfiguration(opts.Configuration)
 
-	if opts.PerformCleanAction {
-		archiveCmd.SetCustomBuildAction("clean")
-	}
-
 	if opts.XcconfigContent != "" {
 		xcconfigWriter := xcconfig.NewWriter(s.pathProvider, s.fileManager, s.pathChecker, s.pathModifier)
 		xcconfigPath, err := xcconfigWriter.Write(opts.XcconfigContent)

step.yml

@@ -36,6 +36,8 @@ description: |-
   3. **The minimum days the Provisioning Profile should be valid**: If this input is set to >0, the managed Provisioning Profile will be renewed if it expires within the configured number of days. Otherwise the Step renews the managed Provisioning Profile if it is expired.
   4. The **Code signing certificate URL**, the **Code signing certificate passphrase**, the **Keychain path**, and the **Keychain password** inputs are automatically populated if certificates are uploaded to Bitrise's **Code Signing** tab. If you store your files in a private repo, you can manually edit these fields.
 
+  If you want to set the Apple service connection credentials on the step-level (instead of using the one configured in the App Settings), use the Step inputs in the **App Store Connect connection override** category. Note that this only works if **Automatic code signing method** is set to `api-key`.
+
   Under **IPA export configuration**:
   1. **Developer Portal team**: Add the Developer Portal team's name to use for this export. This input defaults to the team used to build the archive.
   2. **Rebuild from bitcode**: For non-App Store exports, should Xcode re-compile the app from bitcode?
@@ -392,6 +394,45 @@ inputs:
     - swift_packages
     is_required: true
 
+# App Store Connect connection override
+
+- api_key_path:
+  opts:
+    category: App Store Connect connection override
+    title: App Store Connect API private key
+    summary: Local path or remote URL to the private key (p8 file). This overrides the Bitrise-managed API connection.
+    description: |-
+      Local path or remote URL to the private key (p8 file) for App Store Connect API.
+      This overrides the Bitrise-managed API connection, only set this input if you want to control the API connection
+      on a step-level. Most of the time it's easier to set up the connection on the App Settings page on Bitrise.
+      The input value can be a file path (eg. `$TMPDIR/private_key.p8`) or an HTTPS URL.
+      This input only takes effect if the other two connection override inputs are set too (`api_key_id`, `api_key_issuer_id`).
+    is_required: false
+
+- api_key_id:
+  opts:
+    category: App Store Connect connection override
+    title: App Store Connect API key ID
+    summary: Private key ID used for App Store Connect authentication. This overrides the Bitrise-managed API connection.
+    description: |-
+      Private key ID used for App Store Connect authentication.
+      This overrides the Bitrise-managed API connection, only set this input if you want to control the API connection
+      on a step-level. Most of the time it's easier to set up the connection on the App Settings page on Bitrise.
+      This input only takes effect if the other two connection override inputs are set too (`api_key_path`, `api_key_issuer_id`).
+    is_required: false
+
+- api_key_issuer_id:
+  opts:
+    category: App Store Connect connection override
+    title: App Store Connect API issuer ID
+    summary: Private key issuer ID used for App Store Connect authentication. This overrides the Bitrise-managed API connection.
+    description: |-
+      Private key issuer ID used for App Store Connect authentication.
+      This overrides the Bitrise-managed API connection, only set this input if you want to control the API connection
+      on a step-level. Most of the time it's easier to set up the connection on the App Settings page on Bitrise.
+      This input only takes effect if the other two connection override inputs are set too (`api_key_path`, `api_key_id`).
+    is_required: false
+
 # Debugging
 
 - verbose_log: "no"