Add diarization scoring CLI and improve DER reporting

[ipmedenn]

Important

The Update branch button must only be pressed in very rare occasions.
An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.

What does this PR do?

Adds a standalone CLI for diarization scoring and improves DER and speaker-count reporting.

The scoring script accepts compound RTTM files, directories containing per-recording RTTMs, and NeMo diarization manifests. It reports per-recording and cumulative diarization metrics using NeMo's existing DER implementation.

Collection: ASR / Speaker Diarization

Changelog

• Add scripts/speaker_tasks/score_diarization.py.
• Support scoring RTTM files, RTTM directories, and JSON/JSONL diarization manifests.
• Support file/directory input combinations for reference and hypothesis data.
• Treat reference recordings missing from the hypothesis as empty hypotheses.
• Reject hypothesis recording IDs that are not present in the reference.
• Use manifest offset and duration when calculating speaker-count metrics.
• Add speaker-count mean absolute error alongside speaker-count accuracy.
• Log complete recording IDs for speaker-count mismatches.
• Improve the per-recording DER report with aligned columns and percentages for false alarm, missed speech, confusion, and total DER.
• Add unit tests for the scoring CLI and updated DER reporting.

Usage

Score two compound RTTM files:

python scripts/speaker_tasks/score_diarization.py \
  -r /path/to/reference.rttm \
  -h /path/to/hypothesis.rttm \
  -c 0.25

The reference and hypothesis may each be:

• A compound RTTM file
• A directory containing per-recording RTTM files
• A JSON/JSONL diarization manifest

When both inputs are directories, RTTM filenames must match exactly.

The scorer includes overlapping speech regions (ignore_overlap=False).

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation? CLI help and docstrings are included; no additional documentation is required.
• Does the PR affect components that are optional to install? No new optional dependencies or imports are introduced.

PR Type:

• New Feature
• Bugfix
• Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[tango4j]

/ok to test 4687127

[tango4j]

pytest.parametrize() decorator for unit test.

[tango4j]

/ok to test 80b2d34

[tango4j]

Tests are fixed with parameterization. Approving. Still have some unrelated TTS CICD error.

[tango4j]

tests now have parameterizations. LGTM