2.10.0-2.4.0

New Features

• Added the support for Custom TLS Certificates. (PR #465)
• Option to configure max.incremental.fetch.session.cache.slots to increase partition scalability. (PR #461)

2.9.0-2.4.0

Updates

• Upgrade the base tech version of Apache Kafka to 2.4.0. See Kafka's Release Notes for 2.4.0 for details.
• Updated the SDK to version 0.57.3
  • #3215 is a major bug-fix since v0.57.0. Frameworks are recommended to upgrade to v0.57.3 and issue pod replace commands to exisiting deployments to mitigate the risks. Existing procedures for migrating a existing service to the quotated role should be followed.
• Updated the scheduler JRE to v11.

Important Notes

• Apache Kafka 2.4.0 introduces [KAFKA-7335] - Store clusterId locally to ensure broker joins the right cluster, which means that kafka cluster will store the clusterId locally so that it does not join the wrong zookeeper cluster accidentally. Therefore, if your kafka service is connected to default DC/OS zookeeper, changing the zookeeper path is not permitted. Please check here for more information.

2.8.1-2.3.1

Updates

• Upgrade the base tech version of Apache Kafka to 2.3.1. See Kafka's Release Notes for 2.3.1 for details.(#457)

Improvements

• Improve status check to verify topic successfully created in Zookeeper. In case of failure, the service returns 500 INTERNAL_SERVER_ERROR instead of 200 OK. (#458)

2.8.0-2.3.0

Updates

• Updated the SDK to version 0.57.0. For more information, see release notes for previous SDK releases:
  • 0.57.0
  • 0.56.3

New Features

• Added new CLI commands (PR #447):
  • dcos kafka topic config
  • dcos kafka topic delete-config
  • dcos kafka acl
• By upgrading the SDK, Kafka now comes with support for:
  • Quota enforcement
  • Node draining

2.7.0-2.3.0

Updates

• Upgrade the base dcos-commons SDK version to 0.56.2.
• Upgrade the base tech version of Apache Kafka to 2.3.0. See Kafka's Release Notes for 2.3.0 for details.
• Option to configure new listener config max.connections which limits the number of active connections on each listener.

New Features

• Added support for DC/OS Storage Service (DSS). See official DSS docs for more details.
• User can enable advanced service health checks. Option to choose between a simple port based check and an advanced producer-consumer check based on a custom heartbeat topic.

2.6.0-2.2.1

Updates

• Upgrade the base dcos-commons SDK version to 0.56.1.
• Upgrade the base tech version of Apache Kafka to 2.2.1. See Kafka's Release Notes for 2.2.1 for details.
• Oracle JDK is replaced by OpenJDK 8

New Features

• Autosuggestion available for Service Account and Secrets when launching the service from DC/OS UI
• Support for Secure JMX
• Added marathon service scheduler checks
• Service will fetch all required resources over HTTPS

2.4.0-1.1.1

Updates

• Upgrade the base dcos-commons SDK version to 0.55.2.
• Upgrade the base tech version of Apache Kafka to 1.1.1
• Upgrade Zookeeper Client version to 3.4.13

New Features

• Number of open file descriptors is now configurable via the RLIMIT_NOFILE_SOFT and RLIMIT_NOFILE_HARD configuration parameters.
• Timeouts for readiness checks are now configurable via the READINESS_CHECK_INTERVAL, READINESS_CHECK_DELAY and READINESS_CHECK_TIMEOUT configuration parameters.

2.5.0-2.1.0

In this release we have updated the base tech version of Apache Kafka to 2.1.0 and we have accomodated the new features that came along. Please pay attention the important notes.

Important Notes

• The inter_broker_protocol_version now defaults to the newer, 2.1. This has a few implications, as described below:

  • Kafka 1.1.0 supports inter_broker_protocol_version: 1.1 maximum, and by default it is set to 1.0.
  • Kafka 2.1.0 supports inter_broker_protocol_versions up to 2.1.
  • If you haven't specified a inter_broker_protocol_version in your options file, the new default will be used and changed to 2.1.

  The problem with this is that it will cause downtime - during the upgrade, some Kafka nodes will be on Kafka 1.1.0 using inter_broker_protocol_version 1.0 and others will be on Kafka 2.1.0 using protocol 2.1.

  To avoid any potential downtime caused by this, change the protocol version used when upgrading Kafka.

  • Set up CLI to connect to a soak cluster

  • Update your options_file.json with the following contents:

    {
        ...
        "kafka": {
            ...
            "inter_broker_protocol_version": "1.0"
            ...
        }
        ...
    }
    

  • And update your service like so:

    ~$ dcos package install --cli --yes kafka
    ~$ dcos kafka --name=data-services/kafka update start \
        --package-version=2.5.0-2.1.0 \
        --options=options_file.json
    

Updates

• Upgrade kafka base tech to version 2.1.0. See Kafka's Release Notes for 2.0 and for 2.1 more for details.
• Upgrade Zookeeper Client version to 3.4.13

1.1.20-0.10.1.0

Apache Kafka service using new yaml-based dcos-commons SDK (based on dcos-commons version 0.13.0)

Dcos-commons SDK uses a different format to store scheduler's state and configuration information in Zookeeper. This version automatically detects and upgrades old configuration/state from the previous version (dcos-kafka-service 1.1.19-0.10.1.0)

https://infinity-artifacts.s3.amazonaws.com/autodelete7d/kafka/20170329-200215-a8C1ExSJmdH8VNkU/stub-universe-kafka.zip

libmesos-bundle 1.9, cert installer, and test fixes

• Adds SSL certificate installer for use by scheduler in secure mode
• Switches to local libmesos instead of host libmesos