Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions docs/_include/links.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
<!-- markdownlint-disable MD053 -->

[Amazon DynamoDB Streams]: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html
[Amazon Kinesis Data Streams]: https://docs.aws.amazon.com/streams/latest/dev/introduction.html
[Apache Airflow]: https://airflow.apache.org/
[Astronomer]: https://www.astronomer.io/
[AWS Database Migration Service (AWS DMS)]: https://aws.amazon.com/dms/
[AWS DMS Integration with CrateDB]: https://cratedb-toolkit.readthedocs.io/io/dms/
[AWS Lambda]: https://aws.amazon.com/lambda/
[BM25]: https://en.wikipedia.org/wiki/Okapi_BM25
[cloud-datashader-colab]: https://colab.research.google.com/github/crate/cratedb-examples/blob/amo/cloud-datashader/topic/timeseries/explore/cloud-datashader.ipynb
[cloud-datashader-github]: https://github.com/crate/cratedb-examples/blob/amo/cloud-datashader/topic/timeseries/explore/cloud-datashader.ipynb
Expand All @@ -17,9 +22,10 @@
[dask-weather-data-github]: https://github.com/crate/cratedb-examples/blob/main/topic/timeseries/dask-weather-data-import.ipynb
[Datashader]: https://datashader.org/
[Dynamic Database Schemas]: https://cratedb.com/product/features/dynamic-schemas
[DynamoDB CDC Relay]: https://cratedb-toolkit.readthedocs.io/io/dynamodb/cdc.html
[DynamoDB CDC Relay with AWS Lambda]: https://cratedb-toolkit.readthedocs.io/io/dynamodb/cdc-lambda.html
[DynamoDB Table Loader]: https://cratedb-toolkit.readthedocs.io/io/dynamodb/loader.html
[DynamoDB]: https://aws.amazon.com/dynamodb/
[DynamoDB CDC Relay]: inv:ctk:*:label#dynamodb-cdc
[DynamoDB CDC Relay with AWS Lambda]: inv:ctk:*:doc#io/dynamodb/cdc-lambda
[DynamoDB Table Loader]: inv:ctk:*:label#dynamodb-loader
[Executable stack with Apache Kafka, Apache Flink, and CrateDB]: https://github.com/crate/cratedb-examples/tree/main/framework/flink/kafka-jdbcsink-java
[Geospatial Data Model]: https://cratedb.com/data-model/geospatial
[Geospatial Database]: https://cratedb.com/geospatial-spatial-database
Expand Down
5 changes: 5 additions & 0 deletions docs/_include/styles.html
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,9 @@
height: 0;
}

/* On tiled link overview index pages, give ul/li elements more space */
.ul-li-wide ul li {
margin-bottom: 1rem;
}

</style>
2 changes: 1 addition & 1 deletion docs/connect/drivers.md
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,6 @@ Ruby on Rails ActiveRecord adapter for CrateDB.


```{tip}
Please visit the :ref:`build-status` page for an overview about the integration
Please visit the {ref}`build-status` page for an overview about the integration
status of the client drivers listed above, and more.
```
5 changes: 4 additions & 1 deletion docs/connect/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

:::{include} /_include/links.md
:::
:::{include} /_include/styles.html
:::

:::::{grid}
:padding: 0
Expand Down Expand Up @@ -71,6 +73,7 @@ protocol.
:gutter: 2

::::{grid-item-card} {material-outlined}`link;2em` How to connect
:class-body: ul-li-wide
- {ref}`connect-configure`

To connect to CrateDB, your application or driver needs to be configured
Expand All @@ -87,7 +90,7 @@ protocol.
Database connectivity options and tools.
::::

::::{grid-item-card} {material-outlined}`not_started;2em` How to use database drivers
::::{grid-item-card} {material-outlined}`link;2em` How to connect
- {ref}`connect-java`
- {ref}`connect-javascript`
- {ref}`connect-php`
Expand Down
97 changes: 86 additions & 11 deletions docs/ingest/cdc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,95 @@
:::

:::{div}
CrateDB provides many options to connect and integrate with third-party
Options to connect and integrate CrateDB with third-party
CDC applications, mostly using [CrateDB's PostgreSQL interface].
CrateDB also provides native adapter components to leverage advanced
features.

This documentation section lists corresponding CDC applications and
frameworks which can be used together with CrateDB, and outlines how
to use them optimally.
Please also take a look at support for {ref}`generic ETL <etl>` solutions.
This documentation section lists CDC applications,
frameworks, and solutions, which can be used together with CrateDB,
and outlines how to use them optimally.
Additionally, see support for {ref}`generic ETL <etl>` solutions.
:::

- {ref}`aws-dms`
- {ref}`aws-dynamodb`
- {ref}`aws-kinesis`
- {ref}`debezium`
- {ref}`mongodb`
- {ref}`streamsets`

## Connectors

Native and specialized connectors for CrateDB, both managed and unmanaged.

:::::{grid} 1
:gutter: 2

::::{grid-item-card} Amazon DynamoDB
:link: aws-dynamodb
:link-type: ref
Load data from DynamoDB, a fully managed NoSQL database service provided by
Amazon Web Services (AWS), which is designed for high-performance, scalable
applications and offers key-value and document data structures.
::::

::::{grid-item-card} Amazon Kinesis
:link: aws-kinesis
:link-type: ref
Load data from Amazon Kinesis Data Streams, a serverless streaming data service
that simplifies the capture, processing, and storage of data streams at any scale.
::::

::::{grid-item-card} MongoDB
:link: mongodb
:link-type: ref
Load data from MongoDB or MongoDB Atlas, a document database, self-hosted
or multi-cloud.
::::

:::::


## Platforms

Support for data integration frameworks and platforms, both managed and unmanaged.

:::::{grid} 1
:gutter: 2

::::{grid-item-card} AWS DMS
:link: aws-dms
:link-type: ref
Use AWS Database Migration Service (AWS DMS), a managed migration and replication
service that helps move your database and analytics workloads between different
kinds of databases.
::::

::::{grid-item-card} Debezium
:link: debezium
:link-type: ref
Use Debezium, an open source distributed platform for change data capture for
loading data into CrateDB.
It is used as a building block by a number of downstream third-party projects and products.
::::

::::{grid-item-card} Estuary
:link: estuary
:link-type: ref
Use Estuary Flow, a managed, real-time, reliable change data capture (CDC) solution,
to load data into CrateDB.
It combines agentless CDC, zero-code pipelines, and enterprise-grade governance to
simplify data integration.
::::

::::{grid-item-card} RisingWave
:link: risingwave
:link-type: ref
Use RisingWave, a stream processing and management platform, to load data into CrateDB.
It provides a Postgres-compatible SQL interface, like CrateDB, and a DataFrame-style
Python interface. It is available for on-premises and as a managed service.
::::

::::{grid-item-card} StreamSets
:link: streamsets
:link-type: ref
Use the StreamSets Data Collector Engine to ingest and transform data from many
sources into CrateDB. It runs on-premises or in any cloud.
::::

:::::
Loading