Skip to content

Latest commit

 

History

History
323 lines (251 loc) · 12.1 KB

TESTING_PLAN.md

File metadata and controls

323 lines (251 loc) · 12.1 KB

Comprehensive Testing Plan

This document outlines the complete testing strategy for all RoslynTools features.

Testing Layers

  1. Unit Tests (xUnit) - Test individual methods and classes
  2. Executable Tests (PowerShell) - Test skill executables end-to-end
  3. Agent Scenario Tests - Test Claude Code agent usage patterns
  4. Integration Tests - Test GlobalServer + WorkspaceManager interactions

Unit Test Requirements by Component

1. WorkspaceManager (Server Core)

File: tests/CyberdyneDevelopment.RoslynTools.Server.Tests/WorkspaceManagerTests.cs

Tests Needed:

  • LoadSolutionLoadsAllProjects() - Verify all projects loaded
  • LoadSolutionHandlesInvalidPath() - Should return error for invalid path
  • LoadSolutionHandlesMissingFile() - Should return error for non-existent file
  • GetDiagnosticsReturnsCompilerErrors() - CS errors present
  • GetDiagnosticsReturnsAnalyzerDiagnostics() - CA/SA warnings present
  • GetDiagnosticsFiltersHiddenDiagnostics() - Hidden diagnostics excluded
  • CommitToDiskWritesNewFiles() - New files written to disk
  • CommitToDiskUpdatesExistingFiles() - Modified files updated
  • CommitToDiskHandlesConflicts() - Three-way merge works
  • RollbackDiscardsChanges() - All changes discarded
  • GenerateCodeCreatesValidFile() - New file added to workspace
  • GenerateCodeRejectsSyntaxErrors() - Invalid syntax rejected
  • GenerateCodeAllowsSemanticErrors() - Missing return type allowed

2. TypeFinder

File: tests/CyberdyneDevelopment.RoslynTools.TypeFinder.Tests/TypeFinderTests.cs

Tests Needed:

  • FindTypeByExactName() - Find "Calculator"
  • FindTypeByPattern() - Find ".*Service"
  • FindTypeReturnsNamespace() - Correct namespace returned
  • FindTypeReturnsFilePath() - Correct file path returned
  • FindTypeReturnsLineNumber() - Correct line number returned
  • FindTypeFindsClasses() - Classes found
  • FindTypeFindsInterfaces() - Interfaces found
  • FindTypeFindsRecords() - Records found
  • FindTypeFindsStructs() - Structs found
  • FindTypeFindsEnums() - Enums found
  • FindTypeReturnsBaseType() - Base class returned
  • FindTypeReturnsInterfaces() - Implemented interfaces returned
  • FindTypeHandlesGenericTypes() - Generic types handled
  • FindTypeReturnsEmptyForNonExistent() - Returns empty list, not error

3. ComplexityAnalyzer

File: tests/CyberdyneDevelopment.RoslynTools.ComplexityAnalyzer.Tests/ComplexityAnalyzerTests.cs

Tests Needed:

  • CalculateComplexityForSimpleMethod() - Complexity = 1
  • CalculateComplexityForIfStatement() - +1 for if
  • CalculateComplexityForForLoop() - +1 for loop
  • CalculateComplexityForWhileLoop() - +1 for while
  • CalculateComplexityForSwitchCase() - +1 per case
  • CalculateComplexityForTernary() - +1 for ?:
  • CalculateComplexityForLogicalAnd() - +1 for &&
  • CalculateComplexityForLogicalOr() - +1 for ||
  • CalculateComplexityForNullCoalescing() - +1 for ??
  • PrioritizesCriticalComplexity() - Complexity > 20 = CRITICAL
  • PrioritizesHighComplexity() - Complexity 11-20 = HIGH
  • PrioritizesMediumComplexity() - Complexity 6-10 = MEDIUM
  • PrioritizesLowComplexity() - Complexity 1-5 = LOW
  • IdentifiesMethodsNeedingTests() - shouldTest = true for complexity > 1

4. BuildFixer

File: tests/CyberdyneDevelopment.RoslynTools.BuildFixer.Tests/BuildFixerTests.cs

Tests Needed:

  • DetectsSyntaxErrors() - CS1xxx errors found
  • DetectsSemanticErrors() - CS0xxx errors found
  • DetectsAnalyzerWarnings() - CA/SA warnings found
  • ProvidesSuggestedFixes() - Fix suggestions returned
  • PrioritizesErrorsByFrequency() - Most common errors first
  • FiltersErrorsByPattern() - Pattern filter works
  • ReturnsEmptyForCleanCode() - No errors for valid code

5. NamespaceExplorer

File: tests/CyberdyneDevelopment.RoslynTools.NamespaceExplorer.Tests/NamespaceExplorerTests.cs

Tests Needed:

  • ExploreNamespaceReturnsTypes() - All types in namespace returned
  • ExploreNamespaceReturnsChildNamespaces() - Nested namespaces returned
  • ExploreNamespaceShowsTypeKind() - class/interface/etc shown
  • ExploreNamespaceShowsAccessibility() - public/internal shown
  • ExploreNamespaceHandlesEmptyNamespace() - Empty namespace OK

6. MemberIdentifier

File: tests/CyberdyneDevelopment.RoslynTools.MemberIdentifier.Tests/MemberIdentifierTests.cs

Tests Needed:

  • IdentifiesMethodMembers() - Methods returned
  • IdentifiesPropertyMembers() - Properties returned
  • IdentifiesFieldMembers() - Fields returned
  • IdentifiesEventMembers() - Events returned
  • ReturnsMethodSignatures() - Full signatures with params
  • ReturnsAccessibility() - public/private/protected/internal
  • ReturnsStaticFlag() - isStatic correct
  • ReturnsAbstractFlag() - isAbstract correct
  • ReturnsVirtualFlag() - isVirtual correct
  • ReturnsOverrideFlag() - isOverride correct
  • ReturnsErrorForNonExistentType() - Error for missing type

7. TypeCollisionDetector

File: tests/CyberdyneDevelopment.RoslynTools.TypeCollisionDetector.Tests/TypeCollisionDetectorTests.cs

Tests Needed:

  • DetectsCollisionsInDifferentNamespaces() - Same name, different NS
  • ReturnsAllOccurrences() - All locations returned
  • ReturnsEmptyForNoCollisions() - Empty list for unique names
  • FiltersCollisionsByPattern() - Pattern filter works

8. CodeReader

File: tests/CyberdyneDevelopment.RoslynTools.CodeReader.Tests/CodeReaderTests.cs

Tests Needed:

  • StripsXmlDocumentation() - /// comments removed
  • PreservesCodeStructure() - Code logic intact
  • PreservesTodoComments() - TODO preserved
  • PreservesHackComments() - HACK preserved
  • PreservesNoteComments() - NOTE preserved
  • DoesNotModifyDiskFiles() - Original files unchanged
  • StripsInMemoryOnly() - Workspace modified, not disk

9. CodeWriter

File: tests/CyberdyneDevelopment.RoslynTools.CodeWriter.Tests/CodeWriterTests.cs

Tests Needed:

  • CreatesNewFile() - File created in workspace
  • RejectsSyntaxErrors() - Invalid syntax rejected
  • AllowsSemanticErrors() - Missing return type allowed
  • AppliesNamespaceWrapping() - Namespace added if missing
  • ReturnsDocumentPath() - Correct path returned
  • ReturnsEmptyDiagnosticsForValidCode() - No errors for valid code
  • DoesNotWriteToDisk() - File in memory only

10. DocGenerator

File: tests/CyberdyneDevelopment.RoslynTools.DocGenerator.Tests/DocGeneratorTests.cs

Tests Needed:

  • GeneratesXmlDocForMethod() - , ,
  • GeneratesXmlDocForClass() -
  • GeneratesXmlDocForProperty() - ,
  • SkipsAlreadyDocumented() - No duplicate docs
  • InfersDocFromNaming() - "GetUser" → "Gets user"
  • HandlesGenericTypes() -

11. CodeEditor

File: tests/CyberdyneDevelopment.RoslynTools.CodeEditor.Tests/CodeEditorTests.cs

Tests Needed:

  • AppliesFormatting() - Code formatted
  • AppliesTransformation() - Specified transformation applied
  • ReturnsModifiedCode() - Changes returned
  • HandlesInvalidTransformation() - Error for unknown transformation

12. Refactor (Critical - many test cases)

File: tests/CyberdyneDevelopment.RoslynTools.Refactor.Tests/RefactorTests.cs

Tests Needed:

  • RenameClassUpdatesDeclaration() - Class name changed
  • RenameClassUpdatesReferences() - All usages updated
  • RenameClassUpdatesFileName() - File renamed (optional)
  • RenameMethodUpdatesDeclaration() - Method name changed
  • RenameMethodUpdatesInvocations() - All calls updated
  • RenamePropertyUpdatesDeclaration() - Property name changed
  • RenamePropertyUpdatesAccess() - All property access updated
  • RenameFieldUpdatesDeclaration() - Field name changed
  • RenameFieldUpdatesReferences() - All field refs updated
  • RenameParameterUpdatesSignature() - Param name in signature
  • RenameParameterUpdatesUsage() - Param usage in method body
  • RenameNamespaceUpdatesDeclarations() - All namespace decls
  • RenameNamespaceUpdatesUsingDirectives() - Using statements
  • RenameNamespaceUpdatesQualifiedNames() - Fully qualified names
  • RefactorPreservesAccessibility() - public/private maintained
  • RefactorPreservesSignatures() - Parameters/return types intact
  • RefactorHandlesConflicts() - Error if name exists

13. MarkUntestable

File: tests/CyberdyneDevelopment.RoslynTools.MarkUntestable.Tests/MarkUntestableTests.cs

Tests Needed:

  • MarksLowComplexityMethods() - Complexity <= threshold
  • AddsExcludeFromCodeCoverageAttribute() - Attribute added
  • SkipsHighComplexityMethods() - Complexity > threshold not marked
  • SkipsAlreadyMarkedMethods() - No duplicate attributes

14. CoverageAnalyzer

File: tests/CyberdyneDevelopment.RoslynTools.CoverageAnalyzer.Tests/CoverageAnalyzerTests.cs

Tests Needed:

  • ParsesCoberturaXml() - Coverage file parsed
  • CalculatesLineCoverage() - Line coverage %
  • CalculatesBranchCoverage() - Branch coverage %
  • IdentifiesUncoveredLines() - Missing coverage highlighted
  • GeneratesHtmlReport() - HTML report created

Executable Test Requirements

Script: BuildScripts/test-all-skills.ps1

Tests each skill executable with:

  1. Valid inputs (should succeed with isSuccess: true)
  2. Invalid inputs (should fail gracefully with proper error)
  3. Missing arguments (should show usage/error)
  4. Edge cases (empty strings, special characters, etc.)

Coverage:

  • All 15 skills tested
  • Both success and failure paths
  • Error message validation
  • JSON response structure validation

Agent Scenario Tests

Script: AgentScenarioTests.md

Real-world scenarios where Claude Code agent uses skills:

Scenario 1: Fix Build Errors

  1. User: "My code won't build, can you fix it?"
  2. Agent should:
    • Use build-fixer to identify errors
    • Use type-finder to locate problematic code
    • Use code-editor to fix issues
    • Use build-fixer again to verify fixes
    • Use commit to save changes

Scenario 2: Add New Feature

  1. User: "Add a StringUtils class with a Reverse method"
  2. Agent should:
    • Use code-writer to create class
    • Use member-identifier to verify created
    • Use build-fixer to check for errors
    • Use commit to save

Scenario 3: Refactor Code

  1. User: "Rename Calculator class to MathCalculator"
  2. Agent should:
    • Use type-finder to locate class
    • Use refactor to rename
    • Use type-finder to verify old name gone
    • Use build-fixer to check no broken references
    • Use commit to save

Scenario 4: Improve Test Coverage

  1. User: "What methods need tests?"
  2. Agent should:
    • Use complexity-analyzer to identify complex methods
    • Use coverage-analyzer to see current coverage
    • Suggest which methods need tests
    • Use mark-untestable for trivial code

Scenario 5: Understand Codebase

  1. User: "How is the authentication system organized?"
  2. Agent should:
    • Use namespace-explorer to see structure
    • Use type-finder to locate auth-related classes
    • Use member-identifier to see class APIs
    • Explain architecture to user

Test Execution

Run Unit Tests

dotnet test CyberdyneDevelopment.RoslynTools.sln

Run Executable Tests

pwsh BuildScripts/test-all-skills.ps1

Run Agent Tests

# Manual execution following AgentScenarioTests.md
# Or automated via Claude Code testing framework

Test Coverage Goals

  • Unit Tests: 80%+ code coverage
  • Executable Tests: 100% of skills tested with valid/invalid inputs
  • Agent Tests: All 5 core scenarios passing
  • Integration Tests: All workspace operations tested

Current Status

  • Unit tests written for all components
  • Executable test script created
  • Agent scenario tests created
  • All tests passing
  • Coverage goals met