[Luosuu]

Summary

• forward GPT-OSS per-head s_aux logits to FA4 as learnable_sink
• cast sink parameters to the query dtype required by the FA4 kernel
• reject FA3 and other unsupported vexact attention backends instead of silently dropping sinks
• add a paged-KV FA4 regression covering learnable sinks together with sliding-window attention
• patch Transformers GPT-OSS experts to call VeOmni's GPT-OSS-specific interleaved gate/up quack kernel
• require the same fused_quack MoE implementation on the VeOmni actor and vexact rollout

Why

Transformers' GptOssAttention passes each layer's learned sink parameter as s_aux. The vexact FlashAttention wrapper previously accepted it through **kwargs but discarded it, changing the softmax denominator and producing incorrect GPT-OSS attention outputs.

The VeOmni actor already uses its GPT-OSS-specific interleaved gate/up quack kernel. Vexact now keeps the common Transformers AutoModelForCausalLM loading path and applies a model-specific GPT-OSS experts patch, matching the actor arithmetic without adding GPT-OSS branches to the common loader.

Validation

• local targeted sink tests: 9 passed, 3 skipped
• local GPT-OSS model/config and existing MoE conversion tests: 13 passed
• tiny GPT-OSS regression confirms the common Transformers model class and pipeline plan are retained while experts dispatch through the fused quack patch
• targeted pre-commit hooks: passed
• 8×H100 worker (SM90) targeted FA4 sink suite: 10 passed
• the GPU regression compares FA4 paged-KV output against a PyTorch reference where the sink is an extra softmax logit excluded from value aggregation

GPT-OSS rollout requires attn_impl=fa-invariant-cute and moe_implementation=fused_quack. Model configuration reports a clear error for unsupported choices.

The full GPT-OSS VeRL one-step E2E has a strict training/rollout_probs_diff_max == 0.0 assertion prepared, but the latest 8×H100 launch is pending workspace group quota availability.