diff --git a/docs/advanced-endpoints/intro.mdx b/docs/advanced-endpoints/intro.mdx index 7243f47c..79d2bbb9 100644 --- a/docs/advanced-endpoints/intro.mdx +++ b/docs/advanced-endpoints/intro.mdx @@ -9,6 +9,8 @@ Svix supports multiple types of endpoints in addition to regular webhook endpoin - [Polling Endpoints](/advanced-endpoints/polling-endpoints) - [FIFO Endpoints](/advanced-endpoints/fifo-endpoints) - [Object Storage](/advanced-endpoints/object-storage) +- [SQS](/advanced-endpoints/sqs) +- [SNS](/advanced-endpoints/sns) ## Enabling Advanced Endpoint Types @@ -18,4 +20,4 @@ Advanced Endpoint Types can be enabled at the environment level in the [Svix Das When you enable Advanced Endpoint Types, your users will be able to create them in the [App Portal](/app-portal). -![Advanced Endpoint Types](../img/advanced-endpoints/object-storage-endpoints.png) +![Advanced Endpoint Types](../img/advanced-endpoints/advanced-endpoints.png) diff --git a/docs/advanced-endpoints/sns.mdx b/docs/advanced-endpoints/sns.mdx new file mode 100644 index 00000000..5a154ae6 --- /dev/null +++ b/docs/advanced-endpoints/sns.mdx @@ -0,0 +1,136 @@ +--- +title: SNS endpoints +--- + +# SNS (Simple Notification Service) endpoints + +SNS endpoints allow you to send webhook events to AWS SNS topics instead of HTTP endpoints. + +This is useful for publishing events to notification topics without your customers having to set up any listener endpoint or write any glue code. SNS endpoints are ideal for: + +- **Fan-out messaging**: Deliver events to multiple subscribers +- **Multi-protocol delivery**: Support email, SMS, HTTP, and other protocols +- **Reliability**: Leverage SNS's durability and retry mechanisms +- **Scalability**: Handle high-throughput event publishing + +## Configuration + +Create an SNS endpoint by providing your SNS topic ARN and AWS credentials. Events will be published to your topic as individual messages. + +```bash +curl -X 'POST' \ + 'https://api.eu.svix.com/api/v1/stream/{your_sink_id}/sink' \ + -H 'Authorization: Bearer AUTH_TOKEN' \ + -H 'Accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "config": { + "topicArn": "arn:aws:sns:us-east-1:123456789012:my-topic", + "region": "us-east-1", + "accessKeyId": "xxx", + "secretAccessKey": "xxx" + }, + "type": "sns" + }' +``` + +## Transformation + +SNS endpoints use transformations to format events for the target topic. Here's the transformation function for SNS: + +```javascript +/** + * @param input - The input object + * @param input.events - The array of events in the batch. The number of events in the batch is capped by the Sink's batch size. + * @param input.events[].payload - The message payload (string or JSON). + * @param input.events[].eventType - The message event type (string). + * + * @returns Object containing the response. + * @returns returns.messages - The array of SNS messages to send to the SNS topic. + * @returns returns.messages[].payload - The content of the message (string). + * @returns returns.messages[].subject - An optional subject of the message (string). + */ +function handler(input) { + const messages = input.events.map((event) => ({ + payload: event, + })); + + return { + messages, + }; +} +``` + +For example, if the endpoint receives the following events: + +```json +{ + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" +} +``` + +```json +{ + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" +} +``` + +Using the same example events, the default SNS transformation would produce: + +```json +{ + "messages": [ + { + "payload": { + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" + } + }, + { + "payload": { + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" + } + } + ] +} +``` + +To add a custom subject: +```javascript +function handler(input) { + const messages = input.events.map((event) => ({ + payload: event, + subject: `Webhook: ${event.eventType}`, + })); + + return { + messages, + }; +} +``` + +This would produce: + +```json +{ + "messages": [ + { + "payload": { + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" + }, + "subject": "Webhook: user.created" + }, + { + "payload": { + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" + }, + "subject": "Webhook: user.login" + } + ] +} +``` diff --git a/docs/advanced-endpoints/sqs.mdx b/docs/advanced-endpoints/sqs.mdx new file mode 100644 index 00000000..ca2355af --- /dev/null +++ b/docs/advanced-endpoints/sqs.mdx @@ -0,0 +1,126 @@ +--- +title: SQS endpoints +--- + +# SQS (Simple Queue Service) endpoints + +SQS endpoints allow you to send webhook events to AWS SQS queues instead of HTTP endpoints. + +This is useful for queuing payloads in a message queue without your customers having to set up any listener endpoint or write any glue code. SQS endpoints are ideal for: + +- **Asynchronous processing**: Decouple event delivery from processing +- **High throughput**: Handle large volumes of events with built-in batching +- **Reliability**: Leverage SQS's durability and retry mechanisms +- **Scalability**: Auto-scale consumers based on queue depth + +## Configuration + +Create an SQS endpoint by providing your SQS queue URL and AWS credentials. Events will be batched and sent to your queue according to your sink's batch configuration. + +```bash +curl -X 'POST' \ + 'https://api.eu.svix.com/api/v1/stream/{your_sink_id}/sink' \ + -H 'Authorization: Bearer AUTH_TOKEN' \ + -H 'Accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "config": { + "queueUrl": "https://sqs.us-east-1.amazonaws.com/123456789012/my-queue", + "region": "us-east-1", + "accessKeyId": "xxx", + "secretAccessKey": "xxx" + }, + "type": "sqs" + }' +``` + +## Transformation + +SQS endpoints use transformations to format events for the target queue. Here's the transformation function for SQS: + +```javascript +/** + * @param input - The input object + * @param input.events - The array of events in the batch. The number of events in the batch is capped by the Sink's batch size. + * @param input.events[].payload - The message payload (string or JSON). + * @param input.events[].eventType - The message event type (string). + * + * @returns Object containing the response. + * @returns returns.messages - The array of SQS messages to send to the SQS queue. + * @returns returns.messages[].payload - The payload of the message (string). + */ +function handler(input) { + const messages = input.events.map((event) => ({ + payload: event, + })); + + return { + messages, + }; +} +``` + +For example, if the endpoint receives the following events: + +```json +{ + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" +} +``` + +```json +{ + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" +} +``` + +The default transformation would produce these SQS messages: + +```json +{ + "messages": [ + { + "payload": { + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" + } + }, + { + "payload": { + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" + } + } + ] +} +``` + +To send only the payload data: +```javascript +function handler(input) { + const messages = input.events.map((event) => ({ + payload: event.payload, + })); + + return { + messages, + }; +} +``` + +This would produce: + +```json +{ + "messages": [ + { + "payload": "{\"email\": \"joe@enterprise.io\"}" + }, + { + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" + } + ] +} +``` diff --git a/docs/img/advanced-endpoints/advanced-endpoints.png b/docs/img/advanced-endpoints/advanced-endpoints.png new file mode 100644 index 00000000..9b14db9d Binary files /dev/null and b/docs/img/advanced-endpoints/advanced-endpoints.png differ diff --git a/docs/img/stream/sns-create.png b/docs/img/stream/sns-create.png new file mode 100644 index 00000000..68320baa Binary files /dev/null and b/docs/img/stream/sns-create.png differ diff --git a/docs/img/stream/sqs-create.png b/docs/img/stream/sqs-create.png new file mode 100644 index 00000000..e92de634 Binary files /dev/null and b/docs/img/stream/sqs-create.png differ diff --git a/docs/stream/sinks/introduction.mdx b/docs/stream/sinks/introduction.mdx index cb99b21c..38ce3189 100644 --- a/docs/stream/sinks/introduction.mdx +++ b/docs/stream/sinks/introduction.mdx @@ -11,6 +11,8 @@ The following sink types are supported by Svix Stream: * [HTTP Endpoints](./http) * [OpenTelemetry Trace Collector](./otel_trace) * [Amazon S3](./s3) +* [Amazon SQS](./sqs) +* [Amazon SNS](./sns) * [Google Cloud Storage](./gcs) * [Azure Blob Storage](./azure_blob) * [Polling Endpoint](./poller) @@ -149,4 +151,4 @@ If dispatch fails because the sink is misconfigured, or the retry schedule was e While a sink is disabled, events can still be written to the Stream without interruption. -In the UI, you can still alter the sink's configuration, and re-enable the sink once you're sure the error is resolved. Once a sink is re-enabled, it will resume dispatching events from where it left off. \ No newline at end of file +In the UI, you can still alter the sink's configuration, and re-enable the sink once you're sure the error is resolved. Once a sink is re-enabled, it will resume dispatching events from where it left off. diff --git a/docs/stream/sinks/sns.mdx b/docs/stream/sinks/sns.mdx new file mode 100644 index 00000000..a0a51267 --- /dev/null +++ b/docs/stream/sinks/sns.mdx @@ -0,0 +1,146 @@ +--- +title: Amazon SNS +--- + +# Amazon SNS + +Events can be sent to an Amazon SNS topic using the `amazonSns` sink type. + +Like all Sinks, SNS topics can be created in the Stream Portal... + +![sns-recent-events](../../img/stream/sns-create.png) + +... or [in the API](https://api.svix.com/docs#tag/Sink/operation/v1.stream.sink.create). + +```shell +curl -X 'POST' 'https://api.svix.com/api/v1/stream/strm_30XKA2tCdjHue2qLkTgc0/sink' \ + -H 'Authorization: Bearer AUTH_TOKEN' \ + -H 'Content-Type: application/json' \ + -d '{ + "type": "amazonSns", + "config": { + "topicArn": "arn:aws:sns:us-east-1:123456789012:my-topic", + "region": "us-east-1", + "accessKeyId": "xxx", + "secretAccessKey": "xxx" + }, + "uid": "unique-identifier", + "status": "enabled", + "batchSize": 100, + "maxWaitSecs": 300, + "eventTypes": [], + "metadata": {} +}' +``` + +Each event will be sent as individual messages to your SNS topic. + +# Transformations + +By default, all `amazonSns` Sinks come bundled with the following transformation code. + +```JavaScript +/** + * @param input - The input object + * @param input.events - The array of events in the batch. The number of events in the batch is capped by the Sink's batch size. + * @param input.events[].payload - The message payload (string or JSON). + * @param input.events[].eventType - The message event type (string). + * + * @returns Object containing the response. + * @returns returns.messages - The array of SNS messages to send to the SNS topic. + * @returns returns.messages[].payload - The content of the message (string). + * @returns returns.messages[].subject - An optional subject of the message (string). + */ +function handler(input) { + const messages = input.events.map((event) => ({ + payload: event, + })); + + return { + messages, + }; +} +``` + +`input.events` matches the events sent in [`create_events`](https://api.svix.com/docs#tag/Event/operation/v1.stream.events.create). + +`messages` contains the array of SNS messages that will be sent to the topic. Each message in the array will be sent as a separate SNS message with the specified payload. + +For example, if the following events are written to the stream: + +```shell +curl -X 'POST' \ + 'https://api.svix.com/api/v1/stream/{stream_id}/events' \ + -H 'Authorization: Bearer AUTH_TOKEN' \ + -H 'Accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "events": [ + { + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" + }, + { + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" + } + ] + }' +``` + +The default transformation code would result in two separate messages being sent to your SNS topic: + +**Message 1:** +```json +{ + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" +} +``` + +**Message 2:** +```json +{ + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" +} +``` + +If you want to add a custom subject to your SNS messages, you can customize the transformation: + +```JavaScript +function handler(input) { + const messages = input.events.map((event) => ({ + payload: event, + subject: `Webhook: ${event.eventType}`, + })); + + return { + messages, + }; +} +``` + +This would result in the following messages being sent to your SNS topic with subjects: + +**Message 1:** +```json +{ + "payload": { + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" + }, + "subject": "Webhook: user.created" +} +``` + +**Message 2:** +```json +{ + "payload": { + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" + }, + "subject": "Webhook: user.login" +} +``` diff --git a/docs/stream/sinks/sqs.mdx b/docs/stream/sinks/sqs.mdx new file mode 100644 index 00000000..2976a2d1 --- /dev/null +++ b/docs/stream/sinks/sqs.mdx @@ -0,0 +1,137 @@ +--- +title: Amazon SQS +--- + +# Amazon SQS + +Events can be sent to an Amazon SQS queue using the `amazonSqs` sink type. + +Like all Sinks, SQS queues can be created in the Stream Portal... + +![sqs-recent-events](../../img/stream/sqs-create.png) + +... or [in the API](https://api.svix.com/docs#tag/Sink/operation/v1.stream.sink.create). + +```shell +curl -X 'POST' 'https://api.svix.com/api/v1/stream/strm_30XKA2tCdjHue2qLkTgc0/sink' \ + -H 'Authorization: Bearer AUTH_TOKEN' \ + -H 'Content-Type: application/json' \ + -d '{ + "type": "amazonSqs", + "config": { + "queueUrl": "https://sqs.us-west-2.amazonaws.com/123456789012/my-queue", + "region": "us-west-2", + "accessKeyId": "xxx", + "secretAccessKey": "xxx" + }, + "uid": "unique-identifier", + "status": "enabled", + "batchSize": 100, + "maxWaitSecs": 300, + "eventTypes": [], + "metadata": {} +}' +``` + +Each event will be sent as individual messages to your SQS queue. + +# Transformations + +By default, all `amazonSqs` Sinks come bundled with the following transformation code. + +```JavaScript +/** + * @param input - The input object + * @param input.events - The array of events in the batch. The number of events in the batch is capped by the Sink's batch size. + * @param input.events[].payload - The message payload (string or JSON). + * @param input.events[].eventType - The message event type (string). + * + * @returns Object containing the response. + * @returns returns.messages - The array of SQS messages to send to the SQS queue. + * @returns returns.messages[].payload - The payload of the message (string). + */ +function handler(input) { + const messages = input.events.map((event) => ({ + payload: event, + })); + + return { + messages, + }; +} +``` + +`input.events` matches the events sent in [`create_events`](https://api.svix.com/docs#tag/Event/operation/v1.stream.events.create). + +`messages` contains the array of SQS messages that will be sent to the queue. Each message in the array will be sent as a separate SQS message with the specified payload. + +For example, if the following events are written to the stream: + +```shell +curl -X 'POST' \ + 'https://api.svix.com/api/v1/stream/{stream_id}/events' \ + -H 'Authorization: Bearer AUTH_TOKEN' \ + -H 'Accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "events": [ + { + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" + }, + { + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" + } + ] + }' +``` + +The default transformation code would result in two separate messages being sent to your SQS queue: + +**Message 1:** +```json +{ + "eventType": "user.created", + "payload": "{\"email\": \"joe@enterprise.io\"}" +} +``` + +**Message 2:** +```json +{ + "eventType": "user.login", + "payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}" +} +``` + +If you want to control the message format more precisely, you can customize the payload in each message. + +For example, the following transformation code would include both the payload and event type: + +```JavaScript +function handler(input) { + const messages = input.events.map((event) => ({ + payload: JSON.stringify({ + payload: event.payload, + eventType: event.eventType + }), + })); + + return { + messages, + }; +} +``` + +This would result in the following messages being sent to your SQS queue: + +**Message 1:** +```json +{"payload":{"email":"joe@enterprise.io"},"eventType":"user.created"} +``` + +**Message 2:** +```json +{"payload":{"id":12,"timestamp":"2025-07-21T14:23:17.861Z"},"eventType":"user.login"} +``` diff --git a/sidebars.js b/sidebars.js index 2433c707..d2343920 100644 --- a/sidebars.js +++ b/sidebars.js @@ -26,6 +26,8 @@ module.exports = { 'advanced-endpoints/polling-endpoints', 'advanced-endpoints/fifo-endpoints', 'advanced-endpoints/object-storage', + 'advanced-endpoints/sqs', + 'advanced-endpoints/sns', ] }, ], @@ -122,6 +124,8 @@ module.exports = { 'stream/sinks/http', 'stream/sinks/otel_trace', 'stream/sinks/s3', + 'stream/sinks/sqs', + 'stream/sinks/sns', 'stream/sinks/gcs', 'stream/sinks/azure_blob', 'stream/sinks/poller', @@ -143,4 +147,4 @@ module.exports = { ], }, ], -}; \ No newline at end of file +};