diff --git a/src/data/nav/chat.ts b/src/data/nav/chat.ts index 5cfbb7cf98..9a2e2d828f 100644 --- a/src/data/nav/chat.ts +++ b/src/data/nav/chat.ts @@ -105,6 +105,14 @@ export default { name: 'Hive (Dashboard)', link: '/docs/chat/moderation/direct/hive-dashboard', }, + { + name: 'Tisane', + link: '/docs/chat/moderation/direct/tisane', + }, + { + name: 'Bodyguard', + link: '/docs/chat/moderation/direct/bodyguard', + }, ], }, { diff --git a/src/pages/docs/chat/moderation/custom/index.mdx b/src/pages/docs/chat/moderation/custom/index.mdx index e76f494adf..63068726d0 100644 --- a/src/pages/docs/chat/moderation/custom/index.mdx +++ b/src/pages/docs/chat/moderation/custom/index.mdx @@ -13,15 +13,7 @@ Before publish moderation is where your moderation logic is invoked before the m ### Integration configuration -To fine-tune how Ably handles messages according to your use-case, you can configure before publish rule behavior using the following fields: - -| Field | Description | -| ----- | ----------- | -| Retry timeout | Maximum duration (in milliseconds) that an attempt to invoke the rule may take (including any retries). | -| Max retries | Maximum number of retries after the first attempt at invoking the rule. | -| Failed action | The action to take in the event that a rule fails to invoke. Options are reject the request or publish anyway. | -| Too many requests action | The action to take if your endpoint returns `429 Too Many Requests`, which may happen if your endpoint is overloaded. The options are to fail moderation, or retry. | -| Room filter (optional) | A regular expression to match to specific chat rooms. | +To fine-tune how Ably handles messages according to your use-case, see the [common configuration fields](/docs/chat/moderation#common-config) shared across all before-publish moderation rules. ### The API diff --git a/src/pages/docs/chat/moderation/direct/bodyguard.mdx b/src/pages/docs/chat/moderation/direct/bodyguard.mdx new file mode 100644 index 0000000000..9442a449c7 --- /dev/null +++ b/src/pages/docs/chat/moderation/direct/bodyguard.mdx @@ -0,0 +1,29 @@ +--- +title: Bodyguard +meta_description: "Detect and remove unwanted content in a Chat Room using Bodyguard AI." +--- + +[Bodyguard](https://bodyguard.ai/) is a powerful contextual analysis platform that can be used to moderate content in chat rooms. + +The Bodyguard integration can be applied to chat rooms so that you can use Bodyguard's content moderation capabilities to detect and handle inappropriate content before it's published to other users. + +## Integration setup + +Configure the integration in your [Ably dashboard](https://ably.com/accounts/any/apps/any/integrations) or using the [Control API](/docs/account/control-api). + +The following fields are specific to Bodyguard configuration: + +| Field | Description | +| ----- | ----------- | +| Bodyguard API key | The API key for your Bodyguard account. | +| Channel ID | The ID of your Bodyguard channel where moderation rules are configured. | +| Default Language (optional) | The default language to use for content analysis. This will be used as a fallback in case automatic language detection fails. | +| Model URL (optional) | A custom URL if using a custom moderation model. | + +For additional configuration options shared across all before-publish moderation rules, see the [common configuration fields](/docs/chat/moderation#common-config). + +Messages will be rejected if Bodyguard's analysis returns a `REMOVE` recommended action based on the moderation rules configured in your Bodyguard channel. + +## Handling rejections + +Messages are rejected when they fail Bodyguard's analysis. Bodyguard returns a REMOVE action in these instances and the messages will not be published to your channel. The publish request will be rejected. Moderation rejections will use the error code `42213`. diff --git a/src/pages/docs/chat/moderation/direct/hive-model-only.mdx b/src/pages/docs/chat/moderation/direct/hive-model-only.mdx index 96067ca932..8309cf541e 100644 --- a/src/pages/docs/chat/moderation/direct/hive-model-only.mdx +++ b/src/pages/docs/chat/moderation/direct/hive-model-only.mdx @@ -18,11 +18,8 @@ The following are the fields specific to Hive (model only) configuration: | Hive API key | The API key for your Hive account. | | Thresholds | A map of text [text moderation classes](https://docs.thehive.ai/reference/text-moderation) to [severity](https://docs.thehive.ai/docs/detailed-class-descriptions-text-moderation). When moderating text, any message deemed to be at or above a specified threshold will be rejected and not published to the chat room. | | Model URL (optional) | A custom URL if using a custom moderation model. | -| Retry timeout | Maximum duration (in milliseconds) that an attempt to invoke the rule may take (including any retries). The possible range is 0 - 5000ms. | -| Max retries | Maximum number of retries after the first attempt at invoking the rule. | -| Failed action | The action to take in the event that a rule fails to invoke. Options are reject the request or publish anyway. | -| Too many requests action | The action to take in the event that Hive returns a 429 (Too Many Requests Response). Options are to fail rule invocation, or retry. | -| Room filter (optional) | A regular expression to match to specific chat rooms. | + +For additional configuration options shared across all before-publish moderation rules, see the [common configuration fields](/docs/chat/moderation#common-config). ## Text length diff --git a/src/pages/docs/chat/moderation/direct/tisane.mdx b/src/pages/docs/chat/moderation/direct/tisane.mdx new file mode 100644 index 0000000000..bc60d38f07 --- /dev/null +++ b/src/pages/docs/chat/moderation/direct/tisane.mdx @@ -0,0 +1,29 @@ +--- +title: Tisane +meta_description: "Detect and remove unwanted content in a Chat Room using Tisane AI." +--- + +[Tisane](https://tisane.ai/) is a powerful Natural Language Understanding (NLU) platform that can be used to moderate content in chat rooms. + +The Tisane integration can be applied to chat rooms so that you can use Tisane's text moderation capabilities to detect and handle inappropriate content before it's published to other users. + +## Integration setup + +Configure the integration in your [Ably dashboard](https://ably.com/accounts/any/apps/any/integrations) or using the [Control API](/docs/account/control-api). + +The following are the fields specific to Tisane configuration: + +| Field | Description | +| ----- | ----------- | +| Tisane API key | The API key for your Tisane account. | +| Thresholds | A map of [text moderation categories](https://docs.tisane.ai/apis/tisane-api-response-guide#supported-types) to severity levels (`low`, `medium`, `high`, `extreme`). When moderating text, any message deemed to be at or above a specified threshold will be rejected and not published to the chat room. | +| Default Language | The language to use for content analysis. | +| Model URL (optional) | A custom URL if using a custom moderation model. | + +For additional configuration options shared across all before-publish moderation rules, see the [common configuration fields](/docs/chat/moderation#common-config). + +## Handling rejections + +If a message fails moderation the message will not be published and the publish request will be rejected. + +Moderation rejections will use error code `42213`. diff --git a/src/pages/docs/chat/moderation/index.mdx b/src/pages/docs/chat/moderation/index.mdx index d4d5c7f82e..ff0e288dc3 100644 --- a/src/pages/docs/chat/moderation/index.mdx +++ b/src/pages/docs/chat/moderation/index.mdx @@ -19,6 +19,18 @@ When using before publish moderation, a message is reviewed by an automated mode This approach provides additional safety guarantees, but may come at the cost of a small amount of latency, as messages must be vetted prior to being published. +#### Common configuration fields + +All before-publish moderation rules share the following configuration fields: + +| Field | Description | +| ----- | ----------- | +| Retry timeout | Maximum duration (in milliseconds) that an attempt to invoke the rule may take (including any retries). The possible range is 0 - 5000ms. | +| Max retries | Maximum number of retries after the first attempt at invoking the rule. | +| Failed action | The action to take in the event that a rule fails to invoke. Options are reject the request or publish anyway. | +| Too many requests action | The action to take in the event that the moderation provider returns a 429 (Too Many Requests Response). Options are to fail rule invocation, or retry. | +| Room filter (optional) | A regular expression to match to specific chat rooms. | + ### After publish When using after publish moderation, a message is published as normal, but is forwarded to a moderation engine after the fact. This enables you to avoid the latency penalty of vetting content prior to publish, at the expense of bad content being visible in the chat room (at least briefly). Many automated moderation solutions are able to process and delete offending messages within a few seconds of publication. @@ -37,12 +49,26 @@ Alternatively, you might have a custom solution you wish to integrate with, or a [Hive](https://hivemoderation.com) provide automated content moderation solutions. The first of these is the [model only](/docs/chat/moderation/direct/hive-model-only) solution, which provides access to a powerful ML model that takes content and categorises it against various criteria, for example, violence or hate speech. For each classification, it also provides an indication of the severity of the infraction. Using this information, you can determine what level of classification is appropriate for your chat room and filter / reject content accordingly. Hive offer free credits to allow you to experiment with this solution. -The second solution is the [dashboard](/docs/chat/moderation/direct/hive-dashboard). This is an all-in-one moderation tool, that allows you to combine automated workflows using ML models as well as human review and decisions to control the content in your chat room. +The second solution is the [dashboard](/docs/chat/moderation/direct/hive-dashboard). This is an all-in-one moderation tool, that allows you to combine automated workflows using ML models as well as human review and decisions to control the content in your chat room. Ably is able to integrate your chat rooms directly directly with both of these solutions, to allow you to get up and running with Hive moderation with minimal code required. +### Tisane + +Tisane provides automated content moderation through their [API solution](https://tisane.ai/). The platform uses Natural Language Understanding (NLU) to analyze user-generated content and categorize it against various criteria for content moderation. Each analysis includes information about the type and location of potentially problematic content, enabling you to determine appropriate moderation actions for your chat room. + +Tisane also offers an [on-premises solution](https://tisane.ai/) for organizations that need to deploy the moderation engine within their own infrastructure. Both options can be integrated with Ably to provide automated content moderation for your chat rooms. + +### Bodyguard + +Bodyguard provides content moderation through their [API solution](https://bamboo.bodyguard.ai/api). The service performs contextual analysis of messages to detect and categorize inappropriate content based on configurable criteria. Each analysis provides detailed information about the content, enabling you to implement appropriate moderation actions in your chat room. + +The platform is accessed through a REST API and is integrated with Ably to provide automated content moderation for your chat rooms. + ## What's next? * [Add moderation with Hive (model only)](/docs/chat/moderation/direct/hive-model-only) * [Add moderation with Hive Moderation Dashboard](/docs/chat/moderation/direct/hive-dashboard) +* [Add moderation with Tisane](/docs/chat/moderation/direct/tisane) +* [Add moderation with Bodyguard](/docs/chat/moderation/direct/bodyguard) * [Build custom moderation with the Moderation API](/docs/chat/moderation/custom)