diff --git a/docs/migration/account-migration.mdx b/docs/migration/account-migration.mdx index 542543915..7d15af215 100644 --- a/docs/migration/account-migration.mdx +++ b/docs/migration/account-migration.mdx @@ -14,7 +14,7 @@ import BulkImportUsersCountRequest from "./_blocks/count-staged-users-request.md import BulkImportUsersGetRequest from "./_blocks/get-staged-users-request.mdx" import BulkImportUsersAddRequest from "./_blocks/add-users-for-bulk-import-request.mdx" -# Account Migration +# Account migration The following guide will show you how to move your users from your current authentication solution to **SuperTokens** @@ -24,25 +24,25 @@ import BulkImportUsersAddRequest from "./_blocks/add-users-for-bulk-import-reque ## Overview -The process of migrating your accounts can be broken down into two parts: +The process of migrating your accounts breaks down into two parts: ### Creating new users on the fly -In order to ensure a smooth migration process, with no downtime, you need to be able to directly create new users from the legacy sign up flow. -This is necessary since there will be a time gap between when you export all your data for bulk import and when you go live with **SuperTokens**. +To ensure a smooth migration process, with no downtime, you need to be able to directly create new users from the legacy sign up flow. +This is necessary since there is a time gap between when you export all your data for bulk import and when you go live with **SuperTokens**. New users might get created in that interval through your legacy authentication provider. -Hence, you will also need to create them in **SuperTokens** to keep the data in sync. +Hence, you also need to create them in **SuperTokens** to keep the data in sync. ### Adding most of your users through a bulk import After you have set in place the lazy migration process you can move on to adding most of your users. -This will happened through the bulk import API. +This happens through the bulk import API. The process is asynchronous and can work with large amounts of data. -## Before You Start +## Before you start This guide assumes that you have already integrated **SuperTokens** with your existing stack. If you have not, please check the [Quickstart Guide](/docs/quickstart/introduction) and explore all the supported [authentication methods](/docs/authentication/overview). @@ -53,15 +53,138 @@ If you have not, please check the [Quickstart Guide](/docs/quickstart/introducti ### 1. Update the legacy sign up flow Modify the legacy sign up flow logic to also create new users in **SuperTokens**. -This can be done through the `Import User` endpoint that allows you to directly create accounts. +You can do this through the `Import User` endpoint that allows you to directly create accounts. Call the endpoint from the authentication flow used by your legacy provider. -After you have added the new sign up logic you can deploy the changes and move to the next step. + +## Auth0 Instructions + + +:::warning no-title +Auth0 does not export password hashes by default. You will have to contact their support and request them. +::: + +##### 1. Access the Auth0 Dashboard +##### 2. From the navigation menu go to *Actions* > *Library* +##### 3. Click *Create Action* > *Create custom action* +##### 4. Specify a custom name for your action and then select **Login/Post Login** as the trigger +##### 5. After the action has been created, click the *Add Secret* button and save your `SUPERTOKENS_CORE_API_KEY` as a secret +##### 5. Paste the following code in the editor + +```typescript +exports.onExecutePostLogin = async (event, api) => { + const SUPERTOKENS_CORE_URL = ""; + const SUPERTOKENS_API_KEY = event.secrets.SUPERTOKENS_API_KEY; + + try { + // Check if user already migrated + if (event.user.app_metadata?.migrated_to_supertokens) { + console.log(`User ${event.user.user_id} already migrated`); + return; + } + + const connectionStrategy = event.connection.strategy; + + const userPayload = { + externalUserId: event.user.user_id, + userMetadata: { + auth0_user_id: event.user.user_id, + auth0_connection: event.connection.name, + name: event.user.name, + nickname: event.user.nickname, + picture: event.user.picture + }, + loginMethods: [] + }; + + // Handle different login methods + if (connectionStrategy === 'google-oauth2' || connectionStrategy === 'facebook' || + connectionStrategy === 'github' || connectionStrategy === 'apple' || + connectionStrategy.includes('oauth')) { + const provider = mapProvider(connectionStrategy); + const thirdPartyUserId = event.user.user_id.split('|')[1]; + + userPayload.loginMethods.push({ + recipeId: "thirdparty", + thirdPartyId: provider, + thirdPartyUserId: thirdPartyUserId, + email: event.user.email, + isVerified: event.user.email_verified || false, + isPrimary: true, + timeJoinedInMSSinceEpoch: new Date(event.user.created_at).getTime() + }); + + } else if (connectionStrategy === 'auth0' || connectionStrategy === 'Username-Password-Authentication') { + // Auth0 does not export passworded hashes by default + // You will have to contact their support and request them + userPayload.loginMethods.push({ + recipeId: "emailpassword", + email: event.user.email, + passwordHash: getPasswordHash(event.user.email), + hashingAlgorithm: "bcrypt", + isVerified: event.user.email_verified || false, + isPrimary: true, + timeJoinedInMSSinceEpoch: new Date(event.user.created_at).getTime() + }); + } else if (connectionStrategy === 'sms' || connectionStrategy === 'email') { + const isEmail = connectionStrategy === 'email'; + userPayload.loginMethods.push({ + recipeId: "passwordless", + email: isEmail ? event.user.email : undefined, + phoneNumber: !isEmail ? event.user.phone_number : undefined, + isVerified: true, + isPrimary: true, + timeJoinedInMSSinceEpoch: new Date(event.user.created_at).getTime() + }); + } + + // Import user to SuperTokens + const response = await fetch(`${SUPERTOKENS_CORE_URL}/bulk-import/import`, { + method: 'POST', + headers: { + 'api-key': SUPERTOKENS_API_KEY, + 'Content-Type': 'application/json; charset=utf-8' + }, + body: JSON.stringify(userPayload) + }); + + const result = await response.json(); + + if (response.ok && result.status === 'OK') { + console.log(`User migrated: ${event.user.email || event.user.user_id}`); + api.user.setAppMetadata('migrated_to_supertokens', true); + api.user.setAppMetadata('supertokens_user_id', result.user?.id); + } else { + console.error(`Migration failed: ${JSON.stringify(result)}`); + } + + } catch (error) { + console.error('Migration error:', error.message); + } +}; + +function mapProvider(strategy) { + const mapping = { + 'google-oauth2': 'google', + 'facebook': 'facebook', + 'github': 'github', + 'apple': 'apple', + 'windowslive': 'microsoft', + 'linkedin': 'linkedin' + }; + return mapping[strategy] || strategy; +} +``` + + + +
:::info -If your application does not have a sign up process or if new users are created manually you can skip this step +If your application does not have a sign up process or if new users get created manually you can skip this step ::: + @@ -268,28 +391,185 @@ curl --location --request POST '^{coreInfo.uri}/bulk-import/import' \ -:::info -If your current authentication logic includes a password change flow, you will also have to update it, to keep the user data in sync. -::: + ### 2. Export the accounts from your legacy provider Export the users from your legacy authentication provider and adjust the data to match the request body schema used in the [**`Add Users for Bulk Import`**](#add-users-for-bulk-import-http-request) endpoint. + +## Auth0 Instructions +:::warning no-title +Auth0 does not export password hashes or `TOTP` device information. +You will have to contact their support and request them. +::: + +#### 1. Create a management API application in Auth0 +##### 1.1 Navigate to Auth0 Dashboard and the select `Applications` > `APIs` +##### 1.2 Select `Auth0 Management API` +##### 1.3 Go to `Machine to Machine Applications` tab +##### 1.4 Authorize your application or create a new one +##### 1.5 Grant the following permissions: `read:users`, `update:users`, `create:users`, `read:user_app_metadata` +##### 1.6 Save your `Domain`, `Client ID`, and `Client Secret` + +#### 2. Get the management API access token +You need a valid Management API Access Token to export users. +Use the following `cURL` command to get the token: +```bash +curl --request POST \ + --url 'https://YOUR_DOMAIN.auth0.com/oauth/token' \ + --header 'content-type: application/json' \ + --data '{ + "client_id": "YOUR_CLIENT_ID", + "client_secret": "YOUR_CLIENT_SECRET", + "audience": "https://YOUR_DOMAIN.auth0.com/api/v2/", + "grant_type": "client_credentials" + }' +``` + +#### 3. Create the export job +Use the `POST /api/v2/jobs/users-exports` endpoint to create a job that exports all users. + +```bash +curl --request POST \ + --url 'https://YOUR_DOMAIN.auth0.com/api/v2/jobs/users-exports' \ + --header 'authorization: Bearer YOUR_MGMT_API_TOKEN' \ + --header 'content-type: application/json' \ + --data '{ + "format": "json", + "fields": [ + {"name": "user_id"}, + {"name": "email"}, + {"name": "email_verified"}, + {"name": "name"}, + {"name": "nickname"}, + {"name": "picture"}, + {"name": "created_at"}, + {"name": "updated_at"}, + {"name": "identities"}, + {"name": "app_metadata"}, + {"name": "user_metadata"}, + {"name": "phone_number"}, + {"name": "phone_verified"} + ] + }' +``` + +#### 4. Check the export job status + +Check if the export job has finalized with this request: + +```bash +curl --request GET \ + --url 'https://YOUR_DOMAIN.auth0.com/api/v2/jobs/job_abc123xyz' \ + --header 'authorization: Bearer YOUR_MGMT_API_TOKEN' +``` + +#### 5. Download the export file + +The previous request returns a `location` attribute in the response body if the export job has finalized. +Use it do access your data. + +```bash +curl -o auth0_users.json.gz "LOCATION_URL_FROM_RESPONSE" +gunzip auth0_users.json.gz +``` + +Auth0 exports data in `NDJSON` format. +Convert it to JSON first: + +```bash +cat auth0_users.json | jq -s '.' > auth0_users_array.json +``` + +#### 6. Transform the data to the SuperTokens format + +```typescript +const fs = require('fs'); + +const auth0Users = JSON.parse(fs.readFileSync('auth0_users_array.json', 'utf8')); + +const superTokensUsers = auth0Users.map(auth0User => { + const loginMethods = []; + + auth0User.identities.forEach(identity => { + if (identity.provider === 'auth0') { + // Auth0 does not export passworded hashes by default + // You will have to contact their support and request them + loginMethods.push({ + recipeId: "emailpassword", + email: auth0User.email, + passwordHash: getPasswordHash(auth0User.email), + hashingAlgorithm: "bcrypt", + isVerified: auth0User.email_verified || false, + isPrimary: true, + timeJoinedInMSSinceEpoch: new Date(auth0User.created_at).getTime() + }); + } else if (['google-oauth2', 'facebook', 'github', 'apple'].includes(identity.provider)) { + loginMethods.push({ + recipeId: "thirdparty", + thirdPartyId: mapProvider(identity.provider), + thirdPartyUserId: identity.user_id, + email: auth0User.email, + isVerified: auth0User.email_verified || false, + isPrimary: true, + timeJoinedInMSSinceEpoch: new Date(auth0User.created_at).getTime() + }); + } else if (identity.provider === 'sms') { + loginMethods.push({ + recipeId: "passwordless", + phoneNumber: auth0User.phone_number, + isVerified: auth0User.phone_verified || false, + isPrimary: true, + timeJoinedInMSSinceEpoch: new Date(auth0User.created_at).getTime() + }); + } + }); + + return { + externalUserId: auth0User.user_id, + userMetadata: { + auth0_user_id: auth0User.user_id, + name: auth0User.name, + nickname: auth0User.nickname, + picture: auth0User.picture, + ...auth0User.user_metadata + }, + loginMethods: loginMethods + }; +}); + +fs.writeFileSync('supertokens_users.json', JSON.stringify({ users: superTokensUsers }, null, 2)); + +function mapProvider(auth0Provider) { + const mapping = { + 'google-oauth2': 'google', + 'facebook': 'facebook', + 'github': 'github', + 'apple': 'apple' + }; + return mapping[auth0Provider] || auth0Provider; +} + +console.log(`Transformed ${superTokensUsers.length} users`); +``` + + + ### 3. Perform the bulk migration process :::warning If your application has a sign up process please make sure that you have completed the [**first step**](#1-update-the-legacy-sign-up-flow). -Otherwise, new accounts that get created after you have exported your users will not be available in **SuperTokens**. +Otherwise, new accounts that get created after you have exported your users are not available in **SuperTokens**. ::: -#### 3.1 Add the accounts that should be imported +#### 3.1 Add the accounts to import Using the data that you have generated in the previous step, call the `Add Users for Bulk Import` endpoint. -This step stages the data that will be imported later on by the background job. +This step stages the data that the background job imports later. Keep in mind that the endpoint has a limit of **10000 users** per request. @@ -482,32 +762,32 @@ curl --location --request POST '^{coreInfo.uri}/bulk-import/users' \ :::info The Bulk Import Cron Job -Every 5 minutes the **SuperTokens** core service will run a cron job that goes through the staged users and tries to import them. -If a user gets imported successfully it will get removed from the staged list. +Every 5 minutes the **SuperTokens** core service runs a cron job that goes through the staged users and tries to import them. +If a user gets imported successfully it gets removed from the staged list. ::: #### 3.2 Monitor the progress of the job -In order to determine if all the users have been processed by the import flow call the [`Count Staged Users`](#count-staged-users-http-request) API. +To determine if the import flow has processed all the users call the [`Count Staged Users`](#count-staged-users-http-request) API. -Before doing that, let's first understand the different states in which a staged user can be. +Before doing that, first understand the different states in which a staged user can be. During the import process, the user can have one of the following statuses: -- **NEW**: The user has not yet been picked up by the import process. +- **NEW (not yet started)**: The import process has not yet picked up the user. - **PROCESSING**: The import process has selected the user for import. - **FAILED**: The import process has failed for that user. -If a user gets imported successfully it will then be removed from the staged list. Hence, no status is needed for that state. +If a user gets imported successfully it then gets removed from the staged list. Hence, no status exists for that state. -With this new information let's get back to the `count users` endpoint. -The request counts the users that are staged for import. -Pass a status filter as a query parameter (e.g. `status=NEW`) to count only the users with that status. +With this new information, get back to the `count users` endpoint. +The request counts the users that await import. +Pass a status filter as a query parameter (for example, `status=NEW`) to count only the users with that status. -Given that information, to check if your import is finalized do the following: -1. Call the `count users` API once without any filters. If the count is 0, then the import process is done. -2. If the count is not 0, then check if you still have rows that are getting processed (`status=PROCESSING`) or if there are rows that have not yet been picked up by the import job (`status=NEW`) -3. If the only rows that are left are the ones with the `FAILED` status, then proceed to step `3.3`. There you will see how to debug those issues. +Given that information, to check if your import is complete do the following: +1. Call the `count users` API once without any filters. If the count is 0, then the import process is complete. +2. If the count is not 0, then check if you still have rows that are getting processed (`status=PROCESSING`) or if there are rows that the import job has not yet picked up (`status=NEW`) +3. If the only rows that remain are the ones with the `FAILED` status, then proceed to step `3.3`. There you can see how to debug those issues. @@ -603,7 +883,7 @@ Given that information, to check if your import is finalized do the following: #### 3.3 Handle staged users that failed to import Go through this step only if you have staged users that failed to import. -This can happen for a number of reasons. Some of the most common ones: +This can happen for a number of reasons. Some common ones: - `Email` / `phoneNumber` already exists - `externalUserId` is being already used by other user - A primary user already exists for the email but with a different login method @@ -700,8 +980,8 @@ If at the end of the previous step you have determined that you have staged user -The response will include the import error messages for each specific user. -Use them to determine what needs to be corrected in your import data. +The response includes the import error messages for each specific user. +Use them to determine what you need to correct in your import data. After you have fixed the issues, run the process again, from step `3.1`, only for the corrected data. @@ -709,7 +989,7 @@ After you have fixed the issues, run the process again, from step `3.1`, only fo :::success You have successfully migrated your accounts -If you all your data has been imported then you can now consider the account migration process as done. +If all your data has imported then you can consider the account migration process complete. Go on to the [session migration](/docs/migration/session-migration) step to complete the entire migration flow. :::