kafka table engine, modernize table creation, add tip on limitations,… #3666
Conversation
|
@randomizedcoder is attempting to deploy a commit to the ClickHouse Team on Vercel. A member of the Team first needs to authorize it. |
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
2 Skipped Deployments
|
|
@chernser - g'day mate - How does this PR look please? |
There was a problem hiding this comment.
This contains factual inaccuracies i believe, @Blargian to confirm
| :::tip | ||
| The Kafka Engine does have some limitations: | ||
| - [Kafka message header](https://docs.confluent.io/platform/current/schema-registry/fundamentals/serdes-develop/index.html#wire-format) is not currently supported, so schema registry cannot be used | ||
| - Protobuf, ProtobufSingle are supported, while [ProtobufList](https://github.com/ClickHouse/ClickHouse/issues/78746) is not |
There was a problem hiding this comment.
full stop needed but i believe this is supported @Blargian
| Notes | ||
| [Full Kafka Engine Table creation options](https://clickhouse.com/docs/engines/table-engines/integrations/kafka#creating-a-table) | ||
|
|
||
| [kafka_format must be the last setting](https://github.com/ClickHouse/ClickHouse/issues/37895) |
There was a problem hiding this comment.
This is confusing, the link is with respect to INSERT commands not use in this table engine.
|
Thanks for the feedback Dale I've update the PR, please let me know if that looks ok Regarding ProtobufList and Schemas - I've spent quite some time digging into this, and even have an open source repo which replicates the issues, and allows you see what is and is not supported. ( To be honest, I wasted ~1-2 months of elapsed time trying to get ProtobufList to work, which is why I wanted to warn others NOT to waste their time. ) Repo: https://github.com/randomizedcoder/gdp If you have any issues running the example, please let me know. |
Summary
Minor improvements to the kafka table engine page
Checklist