0.12.1 (04.08.2017)

All issues and pull requests: 0.12.1 milestone

Features

(799) Offline clients

Add new endpoint and interface in Management which can be implemented to show
if data produced on given topic has been accessed recently in any offline storage
(like Hadoop). Read more in docs.

(692) Improved schema management

Avro schemas are now created/deleted in the same transaction as topic creation/deletion,
meaning that no topic is created if schema validation fails and vice versa.

Also new SchemaRepository interface method has been added which allows on validating
schema before trying to send it to Schema Registry (fail fast).

(794) Query topics by metrics

Bugfixes

(758) Use timestamp in seconds in ElasticSearch message tracking

Added new field in ElasticSearch message trace object which is used to order
messages in time. This should significantly increase the speed of fetching
data from ES.

(757) Rate limit schema registry calls in Consumers

There was no rate limit when trying to get schema from Schema Registry when
no cached schema matched the message. This could cause DoS attack on Schema Registry
for subscriptions with high traffic of malformed events.

(777) Subscription URI constraint fix

Fixed subscription URI constraints to match URI spec.

(787) Subscription validation fix

Subscriptions were validated at the wrong moment, which in some cases could lead
to ugly NullPointerException instead of validation message.

(780) Metrics block topic removal

0.12.0 (23.06.2017)

Features

(760) Added http2 client to consumers

(770) Added feature to restrict subscribing for particular topic

Enhancements

(775) Console notification box UX improvements

Notification box will resize according to body text, also error messages require user interaction in order to disappear.

(778) Creating consumer signals chains

From now on, consumers logs contain information about signals like their id or type.
This simplifies analysis of consumers history.

(781) Schema version aware deserialization is backward compatible

Which means that flag schemaVersionAwareSerializationEnabled can be set to true on the fly.
When the flag is enabled on a topic then consumed payload without schema version will be deserialized as well -
Hermes will try hard to adjust avro schema starting with the latest version.

Bugfixes

(774) Ensuring that consumer process exists for processed signal

Fixes NullPointerException which occurred when some signals (e.x. COMMIT) were processed for non existing consumer.

0.11.4 (15.05.2017)

Features

(764) Custom button on UI.

Allows to configure custom view near topic buttons area on hermes-console. Custom view can be set via configuration file, example:

{
    "topic": {
        "buttonsExtension": "<a class=\"btn btn-info {{topic.contentType === 'JSON' ? 'ng-show' : 'ng-hide'}}\" ng-href='http://migrator.example/topics/{{topic.name}}'>Migrate to AVRO</a>",
    }
}

0.11.3 (13.04.2017)

Features

(753) Filtering by headers

Added new filter type: header that allows on filtering messages by HTTP headers. Example of filter definition:

{"type": "header", "header": "My-Propagated-Header", "matcher": "^abc.*"}

Mind that by default no headers are propagated from Frontend to Consumers. To enable headers propagation, define and register
own HeadersPropagator via HermesFrontend.Builder#withHeadersPropagator.

(749) Handling avro/json content type

When converting messages from JSON to Avro Hermes uses json-avro-converter
to provide smooth experience, that does not require changing already produced JSONs (for instance to to support optional
fields).

However in some rare cases it might be desired to send JSON messages that are compatible with
standard Avro JSON encoding. To use vanilla JSON -> Avro converter and bypass json-avro-converter, send requests with avro/json content type.

Enhancements

(748) Topic authorization controls and status in Console

Console now has support for toggling auth on topics. This is an opt-in feature, enable by specifying:

{
    "topic": {
        "authEnabled": true,
    }
}

In Console config.json.

(710) Limit size of messages in preview

(751) Move Zookeeper cache update logs to DEBUG level

(750) Move Schema Registry cache refresh logs to DEBUG level

Bugfixes

(737) Updating subscription in hermes-console resets OAuth password

(734) Prevent manual setting of subscription state to PENDING

(743) Better defaults for max-rate algorithm

0.11.2 (15.03.2017)

Enhancements

(738) Payload content-type check and error handling

From now on, clients who send HTTP request without specified Content-Type on an Avro topic will receive proper error message.

(739) Added latency metrics for schema registry

Metrics are available in the following path:

schema.<schema-repo-type>.latency.read-schema

Bugfixes

(740) Invalid metrics names in zookeeper for topics with underscore in name

0.11.1 (7.03.2017)

Features

(733) Topic authorization

Added feature to control which system has permission to publish on particular topic.

(722) Frontend security context initialization

Added feature to configure SSL context.

(735) Configurable http keep_alive

(707) Throughput limit

Added feature to limit throughput in bytes/sec when publishing to particular topic.
Can be configured to work as simple threshold or dynamically calculated value.

(687) Throughput metric

Metric can be found under {producer|consumer}.{hostname}.throughput.{group}.{topic}

0.11.0 (15.02.2017)

Features

(693) Owners instead of support teams

Replaces group and subscription support team, contact and technical owner with a
single notion – owner. Topics and subscriptions have assigned owners, groups no longer
do, so everyone can create a topic in any group.

Migration guide

After deployment to hermes-management you need to run a migration task.
It will initialise topic and subscription owners by assigning what used to be related group and subscription support teams.
Perform with admin credentials:

POST /migrations/support-team-to-owner?source=Plaintext (or source=Crowd if you used Crowd support teams)

Enhancements

(721) Creator must be an owner of created topic or subscription

(714) Don't match when queried nested field doesn't exist instead of failing

(726) Pass Avro validation errors to users

Bugfixes

(717) NPE in Hermes frontend related to BlacklistZookeeperNotifyingCache

0.10.6 (07.02.2017)

Features

(611) Consumers rate negotiation

Max rate negotiation algorithm for balancing maximum delivery rate across subscription consumers.

Enhancements

(703) Update Curator dependency

(701) Updated migration guide for 0.10.5

(713) Admin scripts catalogue with initial migration script for 0.10.5

Bugfixes

(709) Fix docker-compose and docker setup

0.10.5 (31.01.2017)

Enhancements

(688) Selective algorithm healing

Improved durability of assignments during restarts and zookeeper flaps.
Reporting of assignments and running consumers has been improved and made consistent.
More reliable handling of consumer processes.

Migration guide:

To utilize these improvements it is required to stop all instances in hermes cluster, remove all nodes from {zookeeper.root}/consumers-workload/{kafka.cluster.name}/runtime and restart instances.

This adds a marker in selective algorithm's consumer assignments, which allows rebalancing with removing automatically created assignments.

Alternatively, to avoid switching off your cluster, a script updating assignments' zookeeper nodes' data to AUTO_ASSIGNED can be used. It should be also applied after all nodes run the new version, as previous run could shuffle assignments during deployment.

(698) Fix Dockerfile build

0.10.4 (23.01.2017)

Enhancements

(690) Update json-avro-converter to 0.2.5

(687) Added throughput metric

(684) Limit number of retries for inflight on Frontend graceful shutdown

Bugfixes

(694) Leaking file descriptors

Handling corner case in a race between ack and timeout task.

Because of it number of messages in backup storage was growing with a time.
Eventually, it lead to full backup-storage and further writes ended with an exception which was not caught.
This exception was the reason of file descriptor leak.

Mentioned corner case was fixed in this issue. Beside that, additional exception handling was added and backup-storage size
is from now on monitored.

(695) Decouple filtering rate limiting from backpressure based rate limiting

Filtered messages do not influence on a sending rate.

(686) Do not merge topic.contentType with default value

Fix in hermes-console.