All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
-
Forward compatibility with Broadway v1.1
-
Handle messages without data
0.9.0 - 2023-11-06
-
Requires Elixir v1.11+.
-
Relax
nimble_options
dependency. -
Optional dependency
goth
must be v1.3+. -
Possible breaking change: The default token generator requires the
:goth
option to fetch tokens.If you had not yet upgraded to Goth v1.3+ please follow the upgrade guide. Then wherever you invoke
Broadway.start_link/2
, add the:goth
option:producer: [ module: {BroadwayCloudPubSub.Producer, goth: MyApp.Goth, subscription: "projects/<your-project-id>/subscriptions/<your-subscription-id>"} ]
If you had previously upgraded to Goth v1.3+ then wherever you invoke
Broadway.start_link/2
, you may have something like the following:producer: [ module: {BroadwayCloudPubSub.Producer, subscription: "projects/<your-project-id>/subscriptions/<your-subscription-id>", token_generator: {MyApp, :fetch_token, []}} ]
...where
MyApp.fetch_token/0
is similar to the following:defmodule MyApp def fetch_token do with {:ok, token} <- Goth.fetch(MyApp.Goth) do {:ok, token.token} end end end
You can remove your custom token generator function and replace your producer config with this:
producer: [ module: {BroadwayCloudPubSub.Producer, goth: MyApp.Goth, subscription: "projects/<your-project-id>/subscriptions/<your-subscription-id>"} ]
- The
:goth
option specifies the Goth server for the default token generator.
-
The
:scope
option has been removed. You may set custom scopes on your own Goth server via the:source
option.For example, if you want to use the scope
"https://www.googleapis.com/auth/pubsub"
, then wherever you start Goth add the:scopes
option to your authentication source:# The `:metadata` source type retrieves credentials from Google metadata servers. # Refer to the Goth documentation for more source options. source = {:metadata, scopes: ["https://www.googleapis.com/auth/pubsub"]} children = [ {Goth, name: MyApp.Goth, source: source} ]
0.8.0 - 2022-10-26
This version moves Cloud PubSub from Tesla to Finch, so read the notes below and upgrade with care.
-
Use
:finch
as the HTTP client and provide a:finch
producer option for a user-defined HTTP pool -
Make HTTP requests in a separate process for cleaner shutdown
-
Support multiple topologies from the same Brodway module
-
Add telemetry events around HTTP requests
-
Add
:deliveryAttempt
field to metadata
-
The
:pool_size
option has been removed. Define your own Finch pool and use the:finch
option instead.For example, if your
:pool_size
was 10, then add Finch to your application supervision tree (usually located inlib/my_app/application.ex
):children = [ {Finch, name: MyFinch, pools: %{:default => [size: 10]}} ]
...and wherever you invoke
Broadway.start_link/2
, replace this:producer: [ module: {BroadwayCloudPubSub.Producer, subscription: "projects/<your-project-id>/subscriptions/<your-subscription-id>", pool_size: 10} ]
...with this:
producer: [ module: {BroadwayCloudPubSub.Producer, subscription: "projects/<your-project-id>/subscriptions/<your-subscription-id>", finch: MyFinch} ]
0.7.1 - 2022-05-09
- Add
:receive_timeout
option
0.7.0 - 2021-08-30
- Require Broadway 1.0
0.6.3 - 2021-07-19
- Remove sensitive details from error log messages
- A new
:middleware
option to pass a list of custom Tesla middleware - Failures to ack are now automatically retried. Retries can be customised via the new
:retry
option
0.6.2 - 2021-02-24
- Fixed a bug causing malformed acknowledgement requests (#54)
0.6.1 - 2021-02-23
- Decreased maximum number of ackIds per request (#49)
0.6.0 - 2020-02-18
- Support for passing a tuple for the
scope
option
- Require Broadway v0.6.0
0.5.0 - 2019-11-06
-
Client options for connection pools (#37)
-
Support for configuring acknowledgement behavior (#36)
- Move acknowledger behaviour from
GoogleApiClient
intoClientAcknowledger
(#39)
0.4.0 - 2019-08-19
-
Move to Plataformatec GitHub organization and become an official Broadway connector
-
Rename behaviour
RestClient
toClient
(#23) -
Use hackney as the default adapter (#20)
-
Require Broadway 0.4.x and (optionally) Goth 1.x (#26)
-
Replace
:token_module
option with:token_generator
(#29) -
Hide
handle_receive_messages
function that was accidentally made public
0.3.0 - 2019-05-08
- BREAKING: The PubsubMessage struct now gets unpacked into the
%Broadway.Message{}
received in your pipeline. If you were usingmessage.data.data
before, you can now usemessage.data
. Additional properties from the PubsubMessage can be found in the message metadata, for instance:message.metadata.attributes
ormessage.metadata.messageId
. - Requires
:broadway ~> 0.3.0
0.1.3 - 2019-05-06
- Fixed
BroadwayCloudPubSub.GoogleApiClient
attempting to send an empty acknowledge request.
0.1.2 - 2019-04-11
- MIX_ENV for publishing releases to Hex.
0.1.1 - 2019-04-11
- This
CHANGELOG
file to hopefully serve as an evolving example of a standardized open source projectCHANGELOG
.
- Fixed CircleCI build for publishing docs to Hex.
0.1.0 - 2019-04-10
BroadwayCloudPubSub.Producer
- A GenStage producer that continuously receives messages from a Pub/Sub subscription acknowledges them after being successfully processed.BroadwayCloudPubSub.RestClient
- A generic behaviour to implement Pub/Sub clients using the REST API.BroadwayCloudPubSub.GoogleApiClient
- Default REST client used byBroadwayCloudPubSub.Producer
.BroadwayCloudPubSub.Token
- A generic behaviour to implement token authentication for Pub/Sub clients.BroadwayCloudPubSub.GothToken
- Default token provider used byBroadwayCloudPubSub.Producer
.