Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: 'ClickPipes for MongoDB: Supported data types'
title: 'Supported data types'
slug: /integrations/clickpipes/mongodb/datatypes
description: 'Page describing MongoDB ClickPipe datatype mapping from MongoDB to ClickHouse'
---

MongoDB stores data records as BSON documents. In ClickPipes, you can configure to ingest BSON documents to ClickHouse as either JSON or JSON String. The following table shows the supported BSON to JSON type mapping:
MongoDB stores data records as BSON documents. In ClickPipes, you can configure to ingest BSON documents to ClickHouse as either JSON or JSON String. The following table shows the supported BSON to JSON field type mapping:

| MongoDB BSON Type | ClickHouse JSON Type | Notes |
| ------------------------ | -------------------------------------- | ------------------------ |
Expand All @@ -15,11 +15,15 @@ MongoDB stores data records as BSON documents. In ClickPipes, you can configure
| Double | Float64 | |
| Boolean | Bool | |
| Date | String | ISO 8601 format |
| Regular Expression | \{Options: String, Pattern: String\} | MongoDB regex with fixed fields: Options (regex flags) and Pattern (regex pattern) |
| Timestamp | \{T: Int64, I: Int64\} | MongoDB internal timestamp format with fixed fields: T (timestamp) and I (increment) |
| Regular Expression | \{Options: String, Pattern: String\} | MongoDB regex with fixed fields: Options (regex flags) and Pattern (regex pattern) |
| Timestamp | \{T: Int64, I: Int64\} | MongoDB internal timestamp format with fixed fields: T (timestamp) and I (increment) |
| Decimal128 | String | |
| Array | Array(Nullable(String)) | |
| Binary data | Array(Nullable(Int64)) | Array of byte values |
| Binary data | \{Data: String, Subtype: Int64\} | MongoDB binary data with fixed fields: Data (base64-encoded) and Subtype ([type of binary](https://www.mongodb.com/docs/manual/reference/bson-types/#binary-data)) |
| JavaScript | String | |
| Null | Null | |
| Array | Dynamic | Arrays with homogeneous types become Array(Nullable(T)); arrays with mixed primitive types are promoted to the most general common type; arrays with complex incompatible types become Tuples |
| Object | Dynamic | Each nested field is mapped recursively |

:::info
To learn more about ClickHouse's JSON data types, see [our documentation](https://clickhouse.com/docs/sql-reference/data-types/newjson).
:::
17 changes: 0 additions & 17 deletions docs/integrations/data-ingestion/clickpipes/mongodb/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ import cp_service from '@site/static/images/integrations/data-ingestion/clickpip
import cp_step0 from '@site/static/images/integrations/data-ingestion/clickpipes/cp_step0.png';
import mongodb_tile from '@site/static/images/integrations/data-ingestion/clickpipes/mongodb/mongodb-tile.png'
import mongodb_connection_details from '@site/static/images/integrations/data-ingestion/clickpipes/mongodb/mongodb-connection-details.png'
import ssh_tunnel from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/ssh-tunnel.jpg'
import select_destination_db from '@site/static/images/integrations/data-ingestion/clickpipes/mongodb/select-destination-db.png'
import ch_permissions from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/ch-permissions.jpg'
import Image from '@theme/IdealImage';
Expand Down Expand Up @@ -66,22 +65,6 @@ Make sure you are logged in to your ClickHouse Cloud account. If you don't have

<Image img={mongodb_connection_details} alt="Fill in connection details" size="lg" border/>

#### (Optional) Set up SSH Tunneling {#optional-set-up-ssh-tunneling}

You can specify SSH tunneling details if your source MongoDB database is not publicly accessible.

1. Enable the "Use SSH Tunnelling" toggle.
2. Fill in the SSH connection details.

<Image img={ssh_tunnel} alt="SSH tunneling" size="lg" border/>

3. To use Key-based authentication, click on "Revoke and generate key pair" to generate a new key pair and copy the generated public key to your SSH server under `~/.ssh/authorized_keys`.
4. Click on "Verify Connection" to verify the connection.

:::note
Make sure to whitelist [ClickPipes IP addresses](../clickpipes#list-of-static-ips) in your firewall rules for the SSH bastion host so that ClickPipes can establish the SSH tunnel.
:::

Once the connection details are filled in, click `Next`.

#### Configure advanced settings {#advanced-settings}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: 'ClickPipes for MySQL: Supported data types'
title: 'Supported data types'
slug: /integrations/clickpipes/mysql/datatypes
description: 'Page describing MySQL ClickPipe datatype mapping from MySQL to ClickHouse'
---
Expand Down