Office Hours: May 27, 2026

[andrewmchang]

Our next office hours are happening Wednesday, May 27, 2026 at 1630–1700 UTC!

Learn how T&S peers are using AI for safety workflows, ask for support in implementing open safety models, and connect directly with RMC model partners. We'll also discuss how you can contribute to ongoing projects that the RMC is working on!

• Discord event
• Google Meet
• #roost-model-community

Proposed Agenda

Introduction / Updates

• Recap of what the ROOST Model Community is and the purpose of office hours
• New RMC model: Zentropi's CoPE-B-A4B
• Policy Packs!
• Community Updates: Benchmarks and more!

Q&A / Show and Tell

• What models have caught your eye lately?
• What questions do you have about the RMC? What resources would be most beneficial?
• What progress have you made in testing, using, or evaluating different open safety models?
• What use cases are you most interested in these models for? What would you like to see more of?

Admin:

• Feedback! How can the RMC be more helpful to you? What would you like to see more of?
• Contributing to upcoming RMC projects and engaging with the RMC (sharing experiments, feedback)

What's Next:

• Introduce yourself in GitHub Discussions
• Start a new discussion! Whether it's an implementation question, research findings, or model specific feedback

[andrewmchang]

Zentropi CoPE-B Announcement

• Launched today! Part of the ROOST Model Community. Open weights, Apache 2.0 license!
• Zentropi will be joining the next RMC office hours on June 10th to share more!!
• Juliet did some benchmarking
• Compared GPT-OSS-Safeguard, CoPE-A, CoPE-B:
• CoPE-B has specific format it’s looking for (similar to Safeguard using Harmony format); CoPE-B is a chat format
• This is one eval from one person; would appreciate validation from others as well
• Shorter prompts don’t work as well; models prefer more detail and examples
• Super long prompt: beyond CoPE-A’s context window, but worked for CoPE-B
• Feels like there could be a shift with policy-writers; rather than requiring legal background, it could enable more engineers and others to craft policies to be used with these models
• BYOP models are more pluralistic! They can support more nuanced policies and platforms; e.g. specific allowed/disallowed nudity, specifically-allowed roleplay scenarios

Policy Packs

• https://github.com/roostorg/model-community/tree/main/resources/policy-packs
• CCF policies are an incredible resource

Benchmarks and Evals

• Astroware: Constitutional approach. Synthetic ways of creating data. Strength of approach is defining constitution, then generate more representative, wide breadth dataset.
• Defined 27 harm based constitutions + more harmless constitutions
• Benchmarks are outdated – can we create a dataset that is more representative?
• Create a 10K slice of training data from constitutions across threat surfaces (e.g. prompt injections), language transforms.
• This will be the attack + transformation + harmless inverse.
• As long as data is labeled properly: we have a base set of data, apply transforms, widen it. So: normal data + transformed.
• EG, Perverted Justice dataset -> relabeled and refined
• Most synthetic data is detectable as llm generated
• Challenge: avoid homogeneity in the dataset (tone, composition, etc)
• Taking prompts from realdata - LLMsys etc – use this framing to generate an example