Skip to content

Commit 8537fbe

Browse files
Add live migration from postgres (#2759)
* Add live migration from postgres * Apply suggestions from code review Co-authored-by: James Guthrie <[email protected]> Signed-off-by: alejandrodnm <[email protected]> * Add prepare source section * Minor tweaks and clarification * Populate live migration index page --------- Signed-off-by: alejandrodnm <[email protected]> Co-authored-by: James Guthrie <[email protected]> Co-authored-by: James Guthrie <[email protected]>
1 parent 7655b73 commit 8537fbe

File tree

5 files changed

+477
-2
lines changed

5 files changed

+477
-2
lines changed

migrate/dual-write-and-backfill/dual-write-from-postgres.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ import SourceTargetNote from "versionContent/_partials/_migrate_source_target_no
1818
import DumpDatabaseRoles from "versionContent/_partials/_migrate_dual_write_dump_database_roles.mdx";
1919
import Step6eTurnOnCompressionPolicies from "versionContent/_partials/_migrate_dual_write_6e_turn_on_compression_policies.mdx";
2020
import Step6aThroughc from "versionContent/_partials/_migrate_dual_write_6a_through_c.mdx";
21+
import ExplainPgDumpFlags from "versionContent/_partials/_migrate_explain_pg_dump_flags.mdx";
2122

2223
# Dual-write and backfill from PostgreSQL database
2324

migrate/index.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,9 @@ migrating from PostgreSQL.
3535

3636
If you're using PostgreSQL, you may also have heard of logical replication
3737
being the recommended strategy for migrations with low downtime. Currently,
38-
TimescaleDB doesn't work with logical replication, so this is not a viable
39-
option, but we are actively working on making this possible.
38+
TimescaleDB supports logical replication only from sources that are using
39+
PostgreSQL without Timescale. There's active work being made for supporting
40+
Timescale sources.
4041

4142
If you're migrating from something other than PostgreSQL, and don't want to use
4243
the dual-write and backfill approach, then the easiest way to move your data to

migrate/live-migration/index.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
title: Live migration
3+
excerpt: Migrate an entire database with low downtime
4+
products: [cloud]
5+
keywords: [migration, low-downtime, backup]
6+
tags: [recovery, logical backup, replication]
7+
---
8+
9+
# Live migration
10+
11+
Live migration is a migration strategy to move a large amount of data
12+
(100&nbsp;GB-10&nbsp;TB+) with low downtime (on the order of minutes of
13+
downtime). It is significantly more complicated to execute than a migration
14+
with downtime using [pg_dump/restore][pg-dump-and-restore], but supports more
15+
use-cases and has less requirements than the [dual-write and backfill] method.
16+
17+
<SourceTargetNote />
18+
19+
Live migration leverages Postgres' built-in replication functionality to
20+
provide a seamless migration with very little application downtime.
21+
22+
Roughly, it consists of four steps:
23+
24+
1. Prepare and create replication slot in source database.
25+
2. Copy schema from source to target, optionally enabling hypertables.
26+
3. Copy data from source to target while capturing changes.
27+
4. Apply captured changes from source to target.
28+
29+
Currently live migration only supports migrating from PostgreSQL, but we are
30+
actively working on supporting TimescaleDB.
31+
32+
Live migration works well when:
33+
- Large, busy tables have primary keys, or don't have many `UPDATE` or
34+
`DELETE` statements.
35+
- The insert workload does not exceed 20'000 rows per second, and
36+
inserts are batched. If your application exceeds this, you should use
37+
the [dual-write and backfill] migration method.
38+
39+
For more information, consult the step-by-step migration guide:
40+
41+
- [Live migration from PostgreSQL][from-postgres]
42+
43+
[from-postgres]: /migrate/:currentVersion:/live-migration/live-migration-from-postgres/
44+
[pg-dump-and-restore]: /migrate/:currentVersion:/pg-dump-and-restore/
45+
[dual-write and backfill]: /migrate/:currentVersion:/dual-write-and-backfill/

0 commit comments

Comments
 (0)