Sentinel: a dockerized rare-class-affinity signal for detecting grooming patterns

[sahilx13]

We've built and tested an integration for Sentinel: a
local, self-hosted scoring service that detects rare harmful text patterns (e.g. grooming) using
contrastive example banks and wired it up as a first-class Coop signal. Sharing here per the
contribution norms before opening the PR against this repo.

What it does

A new built-in signal, SENTINEL_RARE_CLASS_AFFINITY, scores string content against Sentinel's
/score endpoint. A few things made this more interesting than a typical "call an external API"
signal:

• No org credentials. Sentinel runs as a local HTTP service (via Docker Compose), so the
  signal is disabled gracefully (with a helpful message) whenever Sentinel is unreachable or its
  banks aren't loaded yet, no API key configuration needed.
• Thread-aware scoring. Sentinel's aggregate score is a skewness over a set of per-message
  scores, not a simple per-message average: a single message essentially never produces a
  meaningful score on its own (skewness of a sample smaller than 5 is defined as 0). So the
  signal pulls prior messages from the same thread (via ItemInvestigationService) and scores the
  whole recent window together, which is what actually surfaces patterns that build up over a
  conversation.

Architecture

sequenceDiagram
    participant Client
    participant SubmitContent
    participant Scylla as ItemInvestigation (Scylla)
    participant RuleEngine
    participant SentinelSignal
    participant SentinelAPI

    Client->>SubmitContent: POST content
    SubmitContent->>Scylla: insertItem (best-effort, pre-rules)
    SubmitContent->>RuleEngine: runEnabledRules
    RuleEngine->>SentinelSignal: evaluate with thread runtime args
    SentinelSignal->>Scylla: getThreadSubmissionsByTime
    SentinelSignal->>SentinelAPI: POST /score (primary + thread texts)
    SentinelSignal-->>RuleEngine: rare_class_affinity_score

Loading

Content is written to Scylla before rule evaluation runs (best-effort, non-blocking) so
thread-aware signals have context available on the very first rule evaluation of a new message,
not just retroactively.

What's included

• SentinelService: a small HTTP client (/health, /banks/status, /score), fully unit tested
• SentinelRareClassAffinitySignal: the signal itself, including disabled-state handling and
  thread-context assembly (with a regression-tested fix for a double-counting edge case where the
  triggering submission could get echoed back into its own thread context)
• A non-configurable SENTINEL tile on the integrations dashboard
• SENTINEL_API_URL env var + local Docker Compose overlay for running Sentinel alongside Coop

Try it locally

# 1. Point Coop at a local Sentinel instance
echo 'SENTINEL_API_URL=http://localhost:8000' >> server/.env

# 2. Run Sentinel via the local overlay compose file
docker compose -f docker-compose.yaml -f docker-compose.sentinel.yaml up -d sentinel

# 3. Start Coop as usual, then create a rule using
#    "Sentinel Rare Class Affinity" on a string content field

Status

Validated end-to-end against our fork (UMass-Rescue/coop-public#1,
24 files changed, unit tests passing, manually verified with threaded test data against a live
Sentinel instance). Opening a PR against roostorg/coop shortly, flagging here first in case
there's early feedback on the approach.

Open questions

• Generalizability. Is a self-hosted/local-only integration (no API key, no SaaS dependency) a
  pattern you'd want to see more of, or is Sentinel specific enough to grooming detection that it
  doesn't generalize?
• Deployment model. Sentinel currently ships as an opt-in overlay
  (docker-compose.sentinel.yaml) rather than a required service in the default
  docker-compose.yaml, so orgs that don't use it pay zero extra resource/complexity cost. Is that
  the right default, or should it eventually be a standard sidecar like the other backing services?
• Thread-context primitive is time-bounded, not identity-bounded.
  ItemInvestigationService.getThreadSubmissionsByTime returns everything before "now" in a time
  window — it doesn't know which submission is "the one currently triggering a rule." That's what
  let this signal double-count its own triggering message before the fix here. Is that surprising
  for other thread-aware signal authors, or worth hardening at the primitive level rather than in
  each signal?
• Silent-failure writes to Scylla. submitContent.ts swallows
  ItemInvestigationService.insertItem errors so a Scylla hiccup never blocks content submission:
  reasonable for uptime, but we hit a real case locally where a schema drift (missing migration)
  failed every single write with zero visible error, and thread context silently stayed empty.
  Worth at least a metric/log line for swallowed failures so this class of issue is diagnosable in
  production?
• Fixed 5-observation floor. Sentinel's skewness-based aggregation is 0 by design until it
  has 5+ messages of context: a deliberate anti-noise measure, but it also means the signal is
  silent for the first few messages of every new thread. Should that floor be tunable per rule/org,
  or is a fixed floor the safer default?

[taobojlen]

Thanks for this; Sentinel looks very cool!

A few early thoughts:

• I don't think that Coop should own the deployment of Sentinel (or other classifiers). Deployment will be very org- and scale-specific, for example, some orgs might want to deploy to GPUs or specialized inference providers. Coop should define the API but the service itself could live in a separate repo, maybe even in the Sentinel one.
• Did you look into Coop's integration plugins? I wonder if that'd be a better fit than a new built-in signal.
  • To make this work, we'd have to extend our plugin API to give all signals a standard context that includes thread items.

In general, there are lots of great classifiers out there, and many adopters will want to use a mix of them. I think Coop should have generic support for classifiers rather than hard-code specific ones! @julietshen and @cassidyjames I wonder what you think -- we already have hard-coded integrations for e.g. Zentropi/OpenAI content mod but I'd argue this should all be generic, letting adopters send in scores from arbitrary classifiers and letting Coop handle it all. Is the plugin system a newer thing? Do we use it elsewhere?

[cassidyjames]

@taobojlen yeah, the plugin system was added relatively recently; the idea being that we shouldn't ship all possible integrations upstream in Coop itself, but should enable adopters to bring their own plugins whether that's for their own internal systems or for external third-party integrations. I'd love to explore more what a sort of "plugin store" would look like, as long as it doesn't imply that ROOST endorses all of the plugins available, and as long as there's some way to ensure versioning/compatibility is handled well.

[sahilx13]

Thanks for the feedback, really useful! I went through the Plugin example at @roostorg/coop-integration-example and also looked into whether making Sentinel a plugin is currently a possibility.

1. Deployment ownership

The PR going to roostorg/coop doesn't add a Dockerfile, a compose service, or any Sentinel source code. It only adds three things: an HTTP client that calls whatever SENTINEL_API_URL you configure, the signal logic itself, and the type wiring for the SENTINEL integration. Coop has no opinion on how or where Sentinel actually runs, it just needs a URL to call.

I do have a docker-compose file for running Sentinel locally while testing, but it's deliberately excluded from the PR.

2. Plugin vs. built-in, the actual gap

Plugins are really cool and I definitely should have looked into them more. After doing some investigation, here's a specific limitation that matters for Sentinel. It comes down to two different things:

• Knowing which thread a message belongs to. This part is already generic. Every signal, built-in or plugin, gets a thread identifier passed to it automatically.
• Actually fetching that thread's message history. This part is not available to plugins. Only built-in signals can reach the internal service that turns a thread identifier into real message content. Plugins have no equivalent way to do this.

So a plugin can tell which conversation a message came from, but it can't pull up the rest of that conversation to look at.

This matters a lot for Sentinel, because reading the conversation is the entire point of it. Sentinel doesn't just score one message, it looks at a run of messages in a thread and looks for a shifting pattern across them. Scoring a single message in isolation is already covered by the OpenAI/Zentropi signals, so there'd be little reason to add Sentinel if it could only do that. Without a way to pull thread history, a plugin version of this signal would quietly fall back to single-message scoring, which defeats the reason it exists.

So right now, built-in isn't a stylistic choice, it's the only option that lets this signal actually work as intended unless we provide plugins more power.

3. Generic classifier support

Agreed this is the right direction long-term, and it's not really about Sentinel specifically. OpenAI and Zentropi are hardcoded the exact same way Sentinel is now. Given the gap above though, Sentinel can't move to the plugin path as-is no matter what we decide on the bigger question. Someone would first need to give plugins a way to read thread history, the same way built-in signals already can. That's really its own design question (how much data access plugins should get) and feels separate from whether Sentinel itself belongs in Coop.

Curious whether @julietshen and @cassidyjames have thoughts on whether extending plugin access to thread data is worth pursuing, and if so, how big a change that ends up being. If we agree to pursuing this path, I can open up another discussion related to it.

[juanmrad]

The sentinel repo linked doesn't seem to be public so I can't take a look 😅 .

As far as built-in vs plugin. We can definitively expand more on the capabilities of plugins, but IMHO I think it is ok for sentinel to be built-in as an open source project. I would move from a ENV variable to a plugin variable instead as I would like us to move out of having env variable for signals and be more configurable on the org level.

As far as it being optional I do think that we do not want to have this be a core feature as adopters can make the decision if they want this integrated and how the integration will play out. As well as thresholds and even configuring the time window to look at for threads.

[gbiss]

Hi all! Sorry to come late to the discussion. Talking with Sahil, we're thinking the following. 1) It doesn't seem straightforward to integrate Sentinel as a plugin, unless we really stretch the definition of "plugin", and 2) if Sentinel runs as a built-in integration under Coop, then it makes the most sense to us that the Sentinel service (optionally) launch with the other Coop services using docker compose.

On point 1. The reason why it's hard to run Sentinel as a plugin is that several native changes to Coop are already necessary in order to get it working. For example, Sentinel requires sequences of strings from a conversation in order to formulate a rare class affinity score. For this reason, Sahil needed to add code to track conversation "threads" in Coop. And several primitives needed to be added to various drop-down fields in the UI. The changes were not extensive, but the point is that Coop code needs to have some awareness of the types of objects Sentinel uses.

On point 2. The reason we prefer to have Sentinel startup with the rest of the Coop services is that most users would likely find it cumbersome to have to set up Sentinel separately in its current form. Like @juanmrad suggested, we think that launching Sentinel through Coop's docker compose should be optional. This is easy to implement using docker profiles, which would disable Sentinel by default and enable it with an additional command line switch.

Is the Coop team ok with a built-in integration as discussed in point 1 as well as launching Sentinel as part of Coop like discussed in point 2?