Skip to content

Conversation

@pcsid
Copy link
Collaborator

@pcsid pcsid commented Dec 19, 2025

…l in need of full model scoring. Fixed small inconsistency bug in config by changing judge_properties to judge_settings.

📌 Description

🔗 Related Issue(s)

🛠️ Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality including new tasks)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor / Code cleanup
  • Maintenance / Chore / Task
  • Other (please describe):

✅ How Has This Been Tested?

  • Unit tests
  • Integration tests
  • Manual testing

Test Results / Screenshots (if applicable):

📸 Screenshots / Demos

📋 Checklist

  • Code follows project style guidelines
  • Tests have been added/updated (if applicable)
  • Documentation has been updated (if applicable)
  • Linked relevant issue(s)
  • Self-reviewed my code

🙌 Additional Notes

  1. Added three new tasks - please test on full dataset, don't have access to cluster so can't test that many samples myself.
  • Phonemes tasks not working as expected as audio clips are too short(0.5-1 seconds), do model often fails to recognize audio.
  • Stuttering task worked fine, but only tested on few samples
  • Noise detection task worked fine, but only tested on few samples
  1. Small bug fix - reads judge_settings instead of judge_properties from the config - maintains similarity with other documentation and codebase pattern(judge_settings for top level, judge_properties when creating judge object itself)

…l in need of full model scoring. Fixed small inconsistency bug in config by changing judge_properties to judge_settings.
@pcsid pcsid self-assigned this Dec 19, 2025
@pcsid pcsid added bug Something isn't working enhancement New feature or request labels Dec 19, 2025
Copy link
Collaborator

@nhhoang96 nhhoang96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure to include the minimum results to compare against the existing reported results from current literature. Update in the PR Screenshots/ Demos section.

Performance results on phoneme and noise_detection tasks are low. Please inspect and submit output samples to verify the expected behaviors of the added tasks.

@@ -0,0 +1,19 @@
task_name: noise_detection
dataset_path: NoiseDetection_LJSpeech_MUSAN-Gaussian
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure the dataset_path is the correct path from HF

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants