How Skywatch uses Osprey

[scarnecchia]

@julietshen had requested that I draft something on how Skywatch uses Osprey, so I've put together a quick summary and shared it here. In addition, I've linked the various sidecar modules that we use, along with the Claude Skills and MCP server.

Skywatch and Osprey

Background

Skywatch Blue is one of the largest community labelers operating on the ATProto network. Since its inception in February 2024, it has grown from a personal project into a robust third-party tool for users to shape their timelines, and for infrastructure administrators to monitor their systems for problematic patterns. We began with an initial automod implementation in Bash, moved to a TypeScript rewrite in December 2025, and in early March 2026, migrated to the Osprey rules engine.

Skywatch processes between 3,000 and 5,000 user reports monthly and emits 115,000 to 150,000 labels across the same period. The previous system, while functional, was entirely event driven: it could apply labels based on regular expressions as content arrived via the firehose, but lacked both post hoc analytical capability and near real-time pattern detection. It was not modular in the manner that Osprey's model infrastructure affords, meaning that rules targeting new lexicons were costly to introduce, and temporal logic (account age gates, incrementing windows) was particularly challenging to implement.

Osprey

Osprey addressed these limitations directly. Its model infrastructure enabled more granular rule composition, which in turn reduced the overall false positive rate. More consequentially, the ability to query execution results and data stored in ClickHouse created a feedback loop: emerging patterns can be identified through analysis, translated into new rules, and deployed within the same operational cycle. This capacity for rapid iteration from observation to enforcement may prove to be Osprey's most significant contribution to the way that Skywatch operates.

We have often been able to identify and deploy rules labelling scam and spam networks affecting our users as fast as, or faster than, Bluesky itself. Skywatch's primary operational interface with Osprey is Claude, via the Skywatch MCP and an associated set of skills, which provide rapid access to the underlying database and to Ozone for trend observation, analysis, and rule development and deployment.

This analytical capacity has also enabled Skywatch to function as a signal provider to third-party PDS hosts, alerting operators when accounts hosted on their infrastructure begin exhibiting suspicious behaviour such as high-volume spam emission.

Analytic Sidecars

Beyond the core rules engine, the Osprey architecture has supported the development of several sidecar programmes that perform batch analyses. These remain analytical rather than enforcement tools, as the signals they produce are noisy, but they have proved valuable for identifying trends and informing new rule creation:

• PDS Signup Anomaly: employs Poisson regression to detect PDS hosts creating new accounts at volumes exceeding their historical baseline
• URL Co-sharing: identifies clusters of accounts that repeatedly share the same URLs within a single day, using Leiden community detection
• Quote Post Co-sharing: applies the same Leiden-based clustering to accounts that quote-post identical target URIs within the same day
• Account Entropy: computes Shannon entropy over account posting timestamps to detect automated patterns, where low entropy (regular timing) suggests likely automation
• Quote Overdispersion: detects anomalous quote-post concentration targeting specific posts using Poisson and normal-approximation statistics
• URL Overdispersion: applies equivalent statistical methods to identify anomalous domain-sharing patterns

These tools are available on GitHub under an Apache 2.0 licence.

The Future

As ATProto grows, Skywatch intends to grow with it, and Osprey is a key part of this work. The ability to implement rule sets that monitor additional lexicons makes it straightforward to offer services to other builders and infrastructure hosts within the broader community. We are also looking to integrate additional tools offered by Roost, such as HMA, in order to offer third-party infrastructure operators perceptual hashing as a trust and safety signal.

• Giulia @ Skywatch

[cassidyjames]

This is so great! 🙌🏻 I've added the skills and sidecars to our ROOST playground page. :)