API Redesign / Performance / Producer

This release is generally incompatible to 0.1 due to renaming of types and movement of functionality out of KafkaClient to Consumer and (the new) Producer APIs. If you are dependent on kafka-rust be sure to have a explicit dependency on the 0.1 version if you cannot upgrade right now.

Some highlights of this release are:

• Make KafkaClient API to accept references where possible (to avoid the need for clones at the caller side)
• Drop kafka::utils from public view; all of the types have been moved to kafka::client and some of them renamed to match their purpose
• Avoid cloning when fetching messages (huge performance win when fetching uncompressed messages)
• Correctly consume compressed messages
• Expose messages basically as a key/value pair (in both fetching and sending APIs)
• Allow client side to control the offset commitment in Consumer API
• Avoid Consumer to clone delivered messages; Consumer no longer implements Iterator
• Sending messages through KafkaClient no longer automatically distributes them to different partitions
• Introduce Producer API for sending messages to Kafka
• Separate out snappy code to snappy crate
• Make KafkaClient Sendable (to allow sending it after initialization to another thread)

Version correction

Line up git tag and project version

Compression, Stability, Performance

• Snappy/Gzip compression when sending messages - @jedisct1
• Handling partial trailing messages - @frew
• Allow sending multiple messages to the same topic-partition in one call - @jedisct1
• Internal re-factorings and small performance improvements - @jedisct1 @xitep
• Allow builds against custom libsnappy installations - @winding-lines
• Fallback offset for consumer - @xitep

Consumer

A new consumer struct for consumer messages from Kafka.

Offset Management

Commit Offset and Fetch Offset methods for a consumer group.

Multi methods

Fetch offset, fetch message and send message had been implemented for single arguments. This release provides methods for multiple arguments.

For eg:

fetch message can take either

1. topic, partition, offset OR
2. Vector of TopicPartitionOffset (a wrapper on above 3)

Similar changes for the other two methods too.

I have added methods for offset management but these are yet to be thoroughly tested.

Gzip Support

Added Gzip support.

Tests for snappy.rs, gzip.rs and codecs.rs

Pre-release

Better Error Handling and Documentation

Out the door

This version is just to get the lib out of the door.