Tidb CDC example integration

[joe-getcouragenow]

GhostDB would be a great match for databases that support CDC

You could use GhostDB to hold your data in a cdb network like cloudflare as a materialised view that is fed from tidb.

Here is an example that feeds data from tidb to Kafka to get an idea :
https://github.com/pingcap/ticdc/tree/master/kafka_consumer.

GhostDB would then act as the read only dB and TiDb cluster as the write dB.

This would be amazingly efficient I think.

[joe-getcouragenow]

The go-sdk looks like it's not ready ? Looks like it's started but needs more work ?

[jakekgrog]

@joe-getcouragenow I'm currently working on the go-sdk at the moment, only started development a couple of days ago.

I'm also open to the idea of providing the ability to integrate with other systems. However, I don't know if GhostDB is there yet in terms of being able to provide that capability, but I'm more than happy for others to open issues and PRs if they feel they would be able to add this ability.

[joe-getcouragenow]

@jakegrog thanks for the response so quickly :)

I will watch the go sdk. BTW you might want to think about using GRPC in order to have a big percentage of the code for all languages be generated for you. Makes it much easier.

Here is a great example of that: https://github.com/liftbridge-io/liftbridge-api
It is also able to support subscriptions thats to the GRPC streaming capabilities.

Let me know how you feel about that.

---

Regarding the integration with other systems. Did you check the Link ? Its not a hug deal. Your system provides a API (GRPC or just HTTP REST like https://github.com/jakekgrog/GhostDB/blob/master/server/ghosthttp/server.go#L48 ) to update a Value in the KV store, and thats really it for CDC. Maybe i am missing something or maybe you are ? I mean did you understand what CDC is ?

[jakekgrog]

@joe-getcouragenow We're picking up development again and one thing we want to provide is the ability to integrate with other systems such as TiDB. We'll definitely be picking this once we do our rework (check the README update). :)