Skip to content

Commit

Permalink
Update docs for LDR v25.1
Browse files Browse the repository at this point in the history
  • Loading branch information
kathancox committed Feb 7, 2025
1 parent 690649b commit 366c902
Show file tree
Hide file tree
Showing 7 changed files with 396 additions and 46 deletions.
149 changes: 149 additions & 0 deletions src/current/_includes/v25.1/ldr/create_logically_replicated_stmt.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
<?xml version="1.0" encoding="UTF-8"?>
<div style="overflow-x:auto;">
<svg xmlns="http://www.w3.org/2000/svg" width="923" height="301">
<polygon points="9 17 1 13 1 21"/>
<polygon points="17 17 9 13 9 21"/>
<rect x="31" y="3" width="72" height="32" rx="10"/>
<rect x="29"
y="1"
width="72"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="39" y="21">CREATE</text>
<rect x="123" y="3" width="98" height="32" rx="10"/>
<rect x="121"
y="1"
width="98"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="131" y="21">LOGICALLY</text>
<rect x="241" y="3" width="106" height="32" rx="10"/>
<rect x="239"
y="1"
width="106"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="249" y="21">REPLICATED</text>
<rect x="387" y="3" width="62" height="32" rx="10"/>
<rect x="385"
y="1"
width="62"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="395" y="21">TABLE</text>
<rect x="469" y="3" width="126" height="32"/>
<rect x="467" y="1" width="126" height="32" class="nonterminal"/>
<text class="nonterminal" x="477" y="21">db_object_name</text>
<rect x="387" y="47" width="72" height="32" rx="10"/>
<rect x="385"
y="45"
width="72"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="395" y="65">TABLES</text>
<rect x="479" y="47" width="26" height="32" rx="10"/>
<rect x="477"
y="45"
width="26"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="487" y="65">(</text>
<rect x="525" y="47" width="230" height="32"/>
<rect x="523" y="45" width="230" height="32" class="nonterminal"/>
<text class="nonterminal" x="533" y="65">logical_replication_resources_list</text>
<rect x="775" y="47" width="26" height="32" rx="10"/>
<rect x="773"
y="45"
width="26"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="783" y="65">)</text>
<rect x="841" y="3" width="60" height="32" rx="10"/>
<rect x="839"
y="1"
width="60"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="849" y="21">FROM</text>
<rect x="86" y="113" width="62" height="32" rx="10"/>
<rect x="84"
y="111"
width="62"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="94" y="131">TABLE</text>
<rect x="168" y="113" width="126" height="32"/>
<rect x="166" y="111" width="126" height="32" class="nonterminal"/>
<text class="nonterminal" x="176" y="131">db_object_name</text>
<rect x="86" y="157" width="72" height="32" rx="10"/>
<rect x="84"
y="155"
width="72"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="94" y="175">TABLES</text>
<rect x="178" y="157" width="26" height="32" rx="10"/>
<rect x="176"
y="155"
width="26"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="186" y="175">(</text>
<rect x="224" y="157" width="230" height="32"/>
<rect x="222" y="155" width="230" height="32" class="nonterminal"/>
<text class="nonterminal" x="232" y="175">logical_replication_resources_list</text>
<rect x="474" y="157" width="26" height="32" rx="10"/>
<rect x="472"
y="155"
width="26"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="482" y="175">)</text>
<rect x="540" y="113" width="40" height="32" rx="10"/>
<rect x="538"
y="111"
width="40"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="548" y="131">ON</text>
<rect x="600" y="113" width="182" height="32"/>
<rect x="598" y="111" width="182" height="32" class="nonterminal"/>
<text class="nonterminal" x="608" y="131">source_connection_string</text>
<rect x="802" y="113" width="58" height="32" rx="10"/>
<rect x="800"
y="111"
width="58"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="810" y="131">WITH</text>
<rect x="597" y="267" width="278" height="32"/>
<rect x="595" y="265" width="278" height="32" class="nonterminal"/>
<text class="nonterminal" x="605" y="285">logical_replication_create_table_options</text>
<rect x="597" y="223" width="24" height="32" rx="10"/>
<rect x="595"
y="221"
width="24"
height="32"
class="terminal"
rx="10"/>
<text class="terminal" x="605" y="241">,</text>
<path class="line"
d="m17 17 h2 m0 0 h10 m72 0 h10 m0 0 h10 m98 0 h10 m0 0 h10 m106 0 h10 m20 0 h10 m62 0 h10 m0 0 h10 m126 0 h10 m0 0 h206 m-454 0 h20 m434 0 h20 m-474 0 q10 0 10 10 m454 0 q0 -10 10 -10 m-464 10 v24 m454 0 v-24 m-454 24 q0 10 10 10 m434 0 q10 0 10 -10 m-444 10 h10 m72 0 h10 m0 0 h10 m26 0 h10 m0 0 h10 m230 0 h10 m0 0 h10 m26 0 h10 m20 -44 h10 m60 0 h10 m2 0 l2 0 m2 0 l2 0 m2 0 l2 0 m-879 110 l2 0 m2 0 l2 0 m2 0 l2 0 m22 0 h10 m62 0 h10 m0 0 h10 m126 0 h10 m0 0 h206 m-454 0 h20 m434 0 h20 m-474 0 q10 0 10 10 m454 0 q0 -10 10 -10 m-464 10 v24 m454 0 v-24 m-454 24 q0 10 10 10 m434 0 q10 0 10 -10 m-444 10 h10 m72 0 h10 m0 0 h10 m26 0 h10 m0 0 h10 m230 0 h10 m0 0 h10 m26 0 h10 m20 -44 h10 m40 0 h10 m0 0 h10 m182 0 h10 m0 0 h10 m58 0 h10 m2 0 l2 0 m2 0 l2 0 m2 0 l2 0 m-327 154 l2 0 m2 0 l2 0 m2 0 l2 0 m22 0 h10 m278 0 h10 m-318 0 l20 0 m-1 0 q-9 0 -9 -10 l0 -24 q0 -10 10 -10 m298 44 l20 0 m-20 0 q10 0 10 -10 l0 -24 q0 -10 -10 -10 m-298 0 h10 m24 0 h10 m0 0 h254 m23 44 h-3"/>
<polygon points="913 281 921 277 921 285"/>
<polygon points="913 281 905 277 905 285"/>
</svg>
</div>
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
If your table does not contain any user-defined types or [foregin key]({% link {{ page.version.version }}/foreign-key.md %}) dependencies, use the [`CREATE LOGICALLY REPLICATED`]({% link {{ page.version.version }}/create-logically-replicated.md %}) syntax to start the stream for a fast, offline initial scan and automatic destination table setup.
6 changes: 6 additions & 0 deletions src/current/_includes/v25.1/sidebar-data/sql.json
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,12 @@
"/${VERSION}/create-index.html"
]
},
{
"title": "<code>CREATE LOGICALLY REPLICATED</code>",
"urls": [
"/${VERSION}/create-logically-replicated.html"
]
},
{
"title": "<code>CREATE LOGICAL REPLICATION STREAM</code>",
"urls": [
Expand Down
4 changes: 4 additions & 0 deletions src/current/v25.1/create-logical-replication-stream.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ The `CREATE LOGICAL REPLICATION STREAM` statement starts [**logical data replica

This page is a reference for the `CREATE LOGICAL REPLICATION STREAM` SQL statement, which includes information on its parameters and possible options. For a step-by-step guide to set up LDR, refer to the [Set Up Logical Data Replication]({% link {{ page.version.version }}/set-up-logical-data-replication.md %}) page.

{{site.data.alerts.callout_success}}
If the table you're replicating does not contain [user-defined types]({% link {{ page.version.version }}/enum.md %}) or [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) dependencies, we recommend using the [`CREATE LOGICALLY REPLICATED`]({% link {{ page.version.version }}/create-logically-replicated.md %}) syntax that provides a fast, offline initial scan and automatic table setup on the destination cluster.
{{site.data.alerts.end}}

## Required privileges

`CREATE LOGICAL REPLICATION STREAM` requires one of the following privileges:
Expand Down
115 changes: 115 additions & 0 deletions src/current/v25.1/create-logically-replicated.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
---
title: CREATE LOGICALLY REPLICATED
summary: The CREATE LOGICALLY REPLICATED statement starts a new unidirectional or bidirectional LDR stream with a fast, offline scan.
toc: true
---

{{site.data.alerts.callout_info}}
{% include feature-phases/preview.md %}

Logical data replication is only supported in CockroachDB {{ site.data.products.core }} clusters.
{{site.data.alerts.end}}

{% include_cached new-in.html version="v25.1" %} The `CREATE LOGICALLY REPLICATED` statement starts [**logical data replication (LDR)**]({% link {{ page.version.version }}/logical-data-replication-overview.md %}) on a table(s) that runs between a source and destination cluster in an active-active setup. `CREATE LOGICALLY REPLICATED` creates the new table on the destination cluster automatically and conducts a fast, offline initial scan. It accepts `unidirectional` or `bidirectional` as an option to create either one of the setups automatically.

Once the offline initial scan completes, the new table will come online and is ready to serve queries. In a bidirectional setup, the second LDR stream will also initialize after the offline initial scan completes.

{{site.data.alerts.callout_danger}}
If the table to be replicated contains [user-defined types]({% link {{ page.version.version }}/enum.md %}) or [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) dependencies, you must use the [`CREATE LOGICAL REPLICATION STREAM`]({% link {{ page.version.version }}/create-logical-replication-stream.md %}) statement instead.
{{site.data.alerts.end}}

This page is a reference for the `CREATE LOGICALLY REPLICATED` SQL statement, which includes information on its parameters and options. For a step-by-step guide to set up LDR, refer to the [Set Up Logical Data Replication]({% link {{ page.version.version }}/set-up-logical-data-replication.md %}) page.

## Required privileges

`CREATE LOGICALLY REPLICATED` requires one of the following privileges:

- The [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role).
- The [`REPLICATION` system privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#privileges).

Use the [`GRANT SYSTEM`]({% link {{ page.version.version }}/grant.md %}) statement:

{% include_cached copy-clipboard.html %}
~~~ sql
GRANT SYSTEM REPLICATION TO user;
~~~

## Synopsis

<div>
{% include {{ page.version.version }}/ldr/create_logically_replicated_stmt.html %}
</div>

### Parameters

Parameter | Description
----------+------------
`db_object_name` | The fully qualified name of the table on the source or destination cluster. Refer to [Examples](#examples).
`logical_replication_resources_list` | A list of the fully qualified table names on the source or destination cluster to include in the LDR stream. Refer to the [LDR with multiple tables](#multiple-tables) example.
`source_connection_string` | The connection string to the source cluster. Use an [external connection]({% link {{ page.version.version }}/create-external-connection.md %}) to store the source cluster's connection URI. To start LDR, you run `CREATE LOGICALLY REPLICATED` from the destination cluster.
`logical_replication_create_table_options` | Options to modify the behavior of the LDR stream. For a list, refer to [Options](#options). **Note:** `bidirectional` or `unidirectional`is a required option.

## Options

Option | Description
-------+------------
`bidirectional` / `unidirectional` | (**Required**) Specifies whether the LDR stream will be unidirectional or bidirectional. With `bidirectional` specified, LDR will set up two LDR streams between the clusters.
`label` | Tracks LDR metrics at the job level. Add a user-specified string with `label`. For more details, refer to [Metrics labels]({% link {{ page.version.version }}/logical-data-replication-monitoring.md %}#metrics-labels).
`mode` | Determines how LDR replicates the data to the destination cluster. Possible values: `immediate`, `validated`. For more details, refer to [LDR modes](#ldr-modes).

## LDR modes

_Modes_ determine how LDR replicates the data to the destination cluster. There are two modes:

- `immediate` (default): {% include {{ page.version.version }}/ldr/immediate-description.md %}
- `validated`: {% include {{ page.version.version }}/ldr/validated-description.md %}

## Examples

`CREATE LOGICALLY REPLICATED` will automatically create the specified source tables on the destination cluster. For unidirectional and bidirectional, you run the statement to start LDR on the destination cluster that does not contain the tables.

### Unidirectional

From the destination cluster of the LDR stream, run:

{% include_cached copy-clipboard.html %}
~~~ sql
CREATE LOGICALLY REPLICATED TABLE {database.public.destination_table_name} FROM TABLE {database.public.source_table_name} ON 'external://source' WITH unidirectional, mode=validated;
~~~

Include the following:

- Fully qualified destination table name.
- Fully qualified source table name.
- [External connection]({% link {{ page.version.version }}/create-external-connection.md %}) for the source cluster. For instructions on creating the external connection for LDR, refer to [Set Up Logical Data Replication]({% link {{ page.version.version }}/set-up-logical-data-replication.md %}#step-2-connect-from-the-destination-to-the-source).
- `unidirectional` option.
- Any other [options](#options).

### Bidirectional

Both clusters will act as a source and destination in bidirectional LDR setups. To start the LDR jobs, you must run this statement from the destination cluster that does not contain the tables:

{% include_cached copy-clipboard.html %}
~~~ sql
CREATE LOGICALLY REPLICATED TABLE {database.public.destination_table_name} FROM TABLE {database.public.source_table_name} ON 'external://source' WITH bidirectional ON 'external://destination', label=track_job;
~~~

Include the following:

- Fully qualified destination table name.
- Fully qualified source table name.
- [External connection]({% link {{ page.version.version }}/create-external-connection.md %}) for the source cluster. For instructions on creating the external connection for LDR, refer to [Set Up Logical Data Replication]({% link {{ page.version.version }}/set-up-logical-data-replication.md %}#step-2-connect-from-the-destination-to-the-source).
- `bidirectional` option defining the external connection for the destination cluster.
- Any other [options](#options).

### Multiple tables

To include multiple tables in an LDR stream, add the fully qualified table names in a list format. Ensure that the table name in the source table list and destination table list are in the same order:

~~~ sql
CREATE LOGICALLY REPLICATED TABLE ({database.public.destination_table_name_1}, {database.public.destination_table_name_2}) FROM TABLE ({database.public.source_table_name_1}, {database.public.source_table_name_2}) ON 'external://source' WITH bidirectional ON 'external://destination', label=track_job;
~~~

## See more

- [`SHOW LOGICAL REPLICATION JOBS`]({% link {{ page.version.version }}/show-logical-replication-jobs.md %})
8 changes: 8 additions & 0 deletions src/current/v25.1/manage-logical-data-replication.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,10 @@ You have a bidirectional LDR setup with a stream between cluster A to cluster B,
CREATE LOGICAL REPLICATION STREAM FROM TABLE {database.public.table_name} ON 'external://{source_external_connection}' INTO TABLE {database.public.table_name};
~~~

{{site.data.alerts.callout_info}}
{% include {{ page.version.version }}/ldr/use-create-logically-replicated.md %}
{{site.data.alerts.end}}

#### Coordinate schema changes for unidirectional LDR

If you have a unidirectional LDR setup, you should cancel the running LDR stream and redirect all application traffic to the source cluster.
Expand All @@ -174,6 +178,10 @@ If you have a unidirectional LDR setup, you should cancel the running LDR stream
CREATE LOGICAL REPLICATION STREAM FROM TABLE {database.public.table_name} ON 'external://{source_external_connection}' INTO TABLE {database.public.table_name};
~~~

{{site.data.alerts.callout_info}}
{% include {{ page.version.version }}/ldr/use-create-logically-replicated.md %}
{{site.data.alerts.end}}

## Jobs and LDR

You can run changefeed and backup [jobs]({% link {{ page.version.version }}/show-jobs.md %}) on any cluster that is involved in an LDR job. Both source and destination clusters in LDR are active, which means they can both serve production reads and writes as well as run [backups]({% link {{ page.version.version }}/backup-and-restore-overview.md %}) and [changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}).
Expand Down
Loading

0 comments on commit 366c902

Please sign in to comment.