MOLT Oracle documentation; refactor MOLT tutorials #19918
Conversation
✅ Deploy Preview for cockroachdb-interactivetutorials-docs canceled.
|
✅ Deploy Preview for cockroachdb-api-docs canceled.
|
Files changed:
|
❌ Deploy Preview for cockroachdb-docs failed. Why did it fail? →
|
✅ Netlify Preview
To edit notification comments on pull requests, go to your Netlify project configuration. |
taroface
requested review from
Jeremyyang920,
ryanluu12345,
noelcrl and
rohan-joshi
I saw this on the preview, but seems like a markdown reference isint resolved properly.
|
| --target $TARGET \ | ||
| --schema-filter 'migration_schema' \ | ||
| --table-filter 'employees|payments|orders' \ | ||
| --replicator-flags '--stagingSchema _replicator_1749699789613149000 --metricsAddr :30005 --userscript table_filter.ts' \ |
There was a problem hiding this comment.
Any reason why the oracle example here is the only one with a userscript?
There was a problem hiding this comment.
Do we need userscript for table filtering on all source databases? I thought this was just oracle.
There was a problem hiding this comment.
Ahh I see. You would need this for MySQL as well.
|
|
||
| #### Table filter userscript | ||
|
|
||
| When migrating only a subset of tables using `--table-filter`, you **must** supply a userscript that filters change events to those tables. For example, save the following as `table_filter.ts`: |
There was a problem hiding this comment.
This could benefit from a bit of clarity that --table-filter helps to filter out tables during initial data load and the userscript is for replication.
| import * as api from "replicator@v1"; | ||
|
|
||
| // List the source tables (matching case) to include in replication | ||
| const allowedTables = ["EMPLOYEES", "PAYMENTS", "ORDERS"]; |
There was a problem hiding this comment.
This could also be rewritten using a set or a dictionary so it's faster, but practically shouldn't make too much of a difference unless allowed tables is huge.
| When migrating from Oracle Multitenant (PDB/CDB), this should be a [common user](https://docs.oracle.com/database/121/ADMQS/GUID-DA54EBE5-43EF-4B09-B8CC-FAABA335FBB8.htm). Prefix the username with `C##` (e.g., `C##MIGRATION_USER`). | ||
| {{site.data.alerts.end}} | ||
|
|
||
| Grant the migration user privileges to connect, read metadata, and `SELECT` and `FLASHBACK` the tables you plan to migrate. The tables should all reside in a single schema (e.g., `migration_schema`). For details, refer to [Schema and table filtering](#schema-and-table-filtering). |
There was a problem hiding this comment.
@noelcrl is "schema" the precise name for this concept in Oracle? Would it also help here to link Oracle docs related to SELECT and FLASHBACK grants or show a sample GRANT query?
| @@ -0,0 +1,22 @@ | |||
| ## Limitations | |||
|
|
|||
| - Migrations must be performed from a single Oracle schema. You **must** include `--schema-filter` so that MOLT Fetch only loads data from the specified schema. Refer to [Schema and table filtering](#schema-and-table-filtering). | |||
There was a problem hiding this comment.
Not something for now, but we should also parse through https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.Oracle.html to understand what other limitations we would have too (since we both use LogMiner under the hood. FYI @taroface and @noelcrl
There was a problem hiding this comment.
@noelcrl In case this needs additional docs work, do you mind filing a new issue for me?
| {% if page.name != "bulk-load.md" %} | ||
| - Replication will not work for tables or column names exceeding 30 characters. This is a [limitation of Oracle LogMiner](https://docs.oracle.com/en/database/oracle/oracle-database/21/sutil/oracle-logminer-utility.html#GUID-7594F0D7-0ACD-46E6-BD61-2751136ECDB4). | ||
|
|
||
| - Running DDL on the source or target while replication is in progress can cause replication failures. |
There was a problem hiding this comment.
We should make this a general warning for any dialect combinations. This is not Oracle specific.
| - User-defined types (UDTs) | ||
| - Nested tables | ||
| - `VARRAY` | ||
| - `LONGBLOB`/`CLOB` columns (over 4000 characters) |
There was a problem hiding this comment.
I thought it does support it but Replicator just doesn't right now.
There was a problem hiding this comment.
Reviewed the rest just now. Looks good, just nits and clarifications. Agreed with Jeremy's comments. Can review after those are all addressed. I also want to ensure rest of the team can take a look here. @taroface it would be helpful if you could also link the exact docs preview section link so folks can see how it presents on the website
|
Addressed all comments. PTAL! The docs preview links are in the PR description. |
|
|
||
| ## Migration failback | ||
| To minimize downtime during migration, MOLT Fetch supports replication streams that sync ongoing changes from the source database to CockroachDB. Instead of performing the entire data load during a planned downtime window, you can perform an initial load followed by continuous replication. Writes are only briefly paused to allow replication to drain before final cutover. The length of the pause depends on the volume of write traffic and the amount of replication lag between the source and CockroachDB. |
There was a problem hiding this comment.
Super nit, should we use "synchronize" instead of "sync" here?
DOC-14130
DOC-14259
DOC-13941
Please review:
The above pages use content from the
_includesfolder. Please review those documents as well.Note that the PG and MySQL flows were updated as well. This is technically outside the scope of this PR, but I'd like to request a review of the SQL user creation steps for each dialect and for CRDB, as this hasn't previously been documented and I was making an educated guess in some places.