Best way to encrypt PII data on TiDB

[sykp241095]

Original author: Sukan_M
Original time: 2023-10-16 03:11:34.613311
Original ID/Slug: 744 / best-way-to-encrypt-pii-data-on-tidb

How to encrypt the PII data on TiDB?

Encryption was enabled on the disk level (DRE), data is coming from upstream MySQL. Can we use any column level masking in DM to support encryption?

[sykp241095]

Original author: TiDB-AI-Assistant
Original time: 2023-10-16 03:11:45.629072

To encrypt PII (Personally Identifiable Information) data on TiDB, you can use column-level encryption or column-level masking. However, currently, TiDB does not provide built-in support for column-level encryption or masking.

If you have enabled encryption on the disk level (DRE), the data is already encrypted at rest. However, if you want to encrypt the data in transit between DM components and the upstream or downstream database, you can enable encrypted data transmission using TLS/SSL .

As for column-level masking, you can use DM (Data Migration) to achieve this. DM provides a feature called “column mapping” that allows you to transform and rewrite data during the synchronization process. By specifying the mapping rules, you can encrypt or mask specific columns before they are synchronized to the downstream database .

Here is an example of how to configure column mapping in DM to encrypt the “id” column:

mapping-rule:
  schema-pattern: "schema_*"
  table-pattern: "table_*"
  expression: "encryption_function(id)"
  source-column: "id"
  target-column: "id"
  arguments: ["encryption_key"]

In the above example, the mapping-rule specifies the schema and table patterns to match, the expression to apply the encryption function to the “id” column, and the encryption key to use. You can customize the encryption function and key according to your requirements.

Please note that DM is responsible for data synchronization between the upstream MySQL and TiDB, and it can perform data transformation during the synchronization process. However, for ongoing encryption or masking of data in TiDB itself, you may need to implement additional measures or use external tools.

It’s important to carefully design and implement encryption or masking solutions to ensure the security and compliance of PII data.

[sykp241095]

Original author: Mini256
Original time: 2023-10-16 04:26:41.062117

Unfortunately, DM does not yet support column level masking/encryption during migration.

It’s welcome to submit a feature issue to pingcap/tiflow repo:

  <a href="https://github.com/pingcap/tiflow" target="_blank" rel="noopener nofollow ugc">GitHub</a>

GitHub - pingcap/tiflow: This repo maintains DM (a data migration platform)...

This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB) - GitHub - pingcap/tiflow: This repo maintains DM (a data migration platform) and TiCDC (change data capt...