-
Notifications
You must be signed in to change notification settings - Fork 23
Description
Context
Mostro Mobile is a Flutter application with critical security requirements for Bitcoin P2P trading. High-quality tests are essential for ensuring the safety of user funds and the reliability of the trading platform.
What is Mutation Testing?
Mutation testing is a technique to measure the quality and effectiveness of our test suite. It works by:
- Creating mutants: The tool makes small, controlled changes (mutations) to the source code
Example: Change == to !=, + to -, true to false
-
Running the test suite: Tests are executed against each mutated version
-
Measuring survival rate:
✅ Mutant killed: Test failed → Good! Tests detected the artificial bug
❌ Mutant survived: Test passed → Bad! Tests did not catch the change
Mutation Score
Score = (Mutants killed / Total mutants) × 100
-
80%: Excellent test coverage
- 50-80%: Acceptable, room for improvement
- < 50%: Needs work
Why Mutation Testing for Mostro Mobile?
- Security-critical code: Trading and key management code needs rigorous verification
- Detect weak tests: Find tests that "cover" code but do not actually verify correctness
- Force better assertions: Encourages specific, strict assertions instead of generic ones
- Find edge cases: Surviving mutants often reveal untested boundary conditions
- CI integration: Can fail builds if mutation score drops below threshold
- Documentation: Living documentation of what behavior is actually tested
Implementation Plan
Phase 1: Setup
-
Install
mutation_testpackagedart pub add --dev mutation_test
-
Configure mutation testing
Createmutation_test.yaml:name: Mostro Mobile targets: lib/** exclude: - "**/*.g.dart" - "**/generated/**" - "**/*.freezed.dart"
Phase 2: CI Integration
Add to .github/workflows/flutter.yml:
- name: Run tests
run: flutter test
- name: Mutation Testing
run: dart run mutation_test mutation_test.yaml
continue-on-error: truePhase 3: Gradual Improvement
- Measure current baseline mutation score
- Set threshold to current score (do not decrease)
- Identify and fix surviving mutants in critical areas:
- Key management (
lib/services/key_derivator.dart) - Nost service (
lib/services/nostr/) - Trade logic (
lib/features/trade/)
- Key management (
- Increase threshold by 5% each sprint
- Target: 80% mutation score for security-critical modules
Acceptance Criteria
- Add mutation_test to dev_dependencies in pubspec.yaml
- Create mutation_test.yaml configuration
- Add mutation testing step to CI workflow
- Generate baseline mutation report
- Document mutation score interpretation in docs/MUTATION_TESTING.md
- Set initial threshold and enforce in CI (non-blocking initially)
Related
- Similar issue in Choke project: grunch/choke#37
- Mostro uses NIP-59 gift wrap, BIP-32 HD keys, and Lightning Network - all areas where mutation testing adds significant value