Consumer groups documentation

[fusion2004]

Coming to Karafka with only a very rudimentary understanding of Kafka, I was very confused at first on how to get the same topic to be processed by different consumers in the same application. (Our use case is wanting to reformat and deliver events to 3 different external services, 2 of which are being handled in the same Karafka app.)

I finally got the hint I needed in this FAQ: https://karafka.io/docs/FAQ/#what-are-consumer-groups-used-for
Unfortunately, it doesn't have a code example currently!

Do you think it would be beneficial to write an entire advanced doc on consumer groups, both in general for Kafka, as well as use in Karafka code and best practices recommendations? (for example: where do I put topic config when it appears multiple times in the routes?)

[mensfeld]

Yeah those are valid points.

This question (same topic multiple consumption) has been a frequent one. What do you think about following:

1. Update the FAQ with code example
2. Write advanced doc on consumer groups AND subscription groups (we can use this for CGs: https://mensfeld.pl/2017/11/kafka-on-rails-using-kafka-with-ruby-on-rails-part-1-kafka-basics-and-its-advantages/) and other docs for SGs (which are karafka concept)
3. Update other relevant docs as you mentioned

Thank you for pointing this out. To be honest there are sooo many things that need docs that I'm just unable to keep up 😓

[fusion2004]

Yeah, I think I could try to write that in a few days. IMHO tho: this and the other thing I posted are the only questions I've had trouble figuring out the answers to w/ Karafka. The documentation is generally fantastic!

[mensfeld]

I will be happy if you can add code example to the FAQ.

I'll tackle 2 and 3 after Karafka 2.4 is released (hopefully tomorrow).