Add testing plugin for comprehensive test case generation

[BhargaviGudi]

What this PR does / why we need it:

This PR adds a new testing plugin that automates comprehensive test case generation for features.

Key capabilities:

• Generates detailed test cases across 6 categories (Functional, Regression, Smoke, Edge Cases, Security, Performance)
• Supports priority filtering (--priority high|medium|low) for focused test suites
• Component tagging (--component name) for test organization
• Multiple output formats: Markdown (default) or DOCX for stakeholders
• Includes critical test case summary section for quick validation
• Creates test cases in the current working directory - run the command from your project directory

Why we need it:

• Automates time-consuming manual test case creation
• Ensures consistent, comprehensive test coverage
• Provides structured documentation for QA teams
• Supports different output formats for technical and non-technical audiences

Command:

/testing:generate-test-case-doc <feature_name> [--priority high|medium|low] [--component name] [--format markdown|docx]

Usage pattern:
The command creates test case files in your current working directory. Navigate to your project directory first, then run the command.

Example:
cd ~/projects/openshift/security-profiles-operator
/testing:generate-test-case-doc "audit logging"
# Creates: ~/projects/openshift/security-profiles-operator/testcases-audit-logging.md

Which issue(s) this PR fixes:

N/A - New feature addition

Special notes for your reviewer:

Plugin Structure

• Located in plugins/testing/ following repository conventions
• Passed all linter validations (make lint)
• Documentation auto-generated via make update

Key Files

• plugins/testing/commands/generate-test-case-doc.md - Command definition in man page format
• plugins/testing/skills/testcase--doc-generator/generate_docx.py - DOCX conversion helper
• plugins/testing/README.md - User documentation

File Location Behavior

• Test case files are created in the current working directory where the command is invoked
• File naming pattern: testcases-{sanitized-feature-name}.md or .docx
• Example: Feature "Audit Logging" → testcases-audit-logging.md
• Allows users to organize test cases alongside project code

Dependencies

• Markdown output: No dependencies
• DOCX output: Requires pip install python-docx (optional)

Usage Examples
Basic usage - creates testcases-user-authentication.md in current directory
/testing:generate-test-case-doc "User Authentication"

High priority tests with DOCX format
/testing:generate-test-case-doc "Payment Gateway" --priority high --format docx

With component tagging
/testing:generate-test-case-doc "API Gateway" --component api,security

Real-world example in specific project

cd ~/projects/openshift/security-profiles-operator
/testing:generate-test-case-doc "audit logging"
Creates: ~/projects/openshift/security-profiles-operator/testcases-audit-logging.md

Testing details:

> /testing:generate-test-case-doc is running… "json enricher" 

● I'll generate comprehensive test cases for the "json enricher" feature. Let me start by analyzing the codebase to gather context.

● 🔍 Analyzing codebase for 'json enricher'...

● Search(pattern: "json.?enricher")
  ⎿  Found 35 files (ctrl+o to expand)

● Search(pattern: "enricher", type: "go")
  ⎿  Found 55 files (ctrl+o to expand)

● Search(pattern: "**/*enricher*.go")
  ⎿  Found 6 files (ctrl+o to expand)

● Search(pattern: "**/README*.md")
  ⎿  Found 100 files (ctrl+o to expand)

● Perfect! I found the JSON enricher implementation and existing test files. Let me read the key files to understand the feature better.

● Read(internal/pkg/daemon/enricher/jsonenricher.go)
  ⎿  Read 524 lines

● Read(test/tc_json_enricher_test.go)
  ⎿  Read 250 lines

● Read(internal/pkg/daemon/enricher/json_enricher_test.go)
  ⎿  Read 371 lines

● Read(doc/audit-logging-guide.md)
  ⎿  Read 156 lines

● Let me now look at the README to understand the project better and search for configuration examples.

● Read(README.md)
  ⎿  Read 91 lines

● Search(pattern: "enableJsonEnricher", output_mode: "content", head_limit: 20)
  ⎿  Found 20 lines (ctrl+o to expand)

● 📋 Looking for existing test files...

● Read(installation-usage.md)
  ⎿  Read 200 lines

● Perfect! Now let me gather more details about the types and APIs.

● Read(internal/pkg/daemon/enricher/types/types.go)
  ⎿  Read 102 lines

● Search(pattern: "**/Makefile")
  ⎿  Found 39 files (ctrl+o to expand)

● ✓ Context analysis complete

  Now I have gathered comprehensive context about the JSON enricher feature. Let me update the todo and generate the test cases.

● Now let me generate comprehensive test cases based on the context I've gathered:

● Write(testcases-json-enricher.md)
  ⎿  Wrote 1795 lines to testcases-json-enricher.md   
     # Test Cases: JSON Enricher
     **Generated**: 2025-11-03T00:00:00Z
     **Feature**: JSON Enricher (Audit Log Enrichment)
     **Component**: enricher, audit-logging
     **Priority Filter**: all
     **Total Test Cases**: 42
     ---
     … +1785 lines (ctrl+o to expand)

● ✓ Test cases generated successfully!

  Feature: JSON Enricher
  Total Test Cases: 35
  Priority Filter: all
  Component: enricher, audit-logging

  Breakdown:
  - High Priority: 10
  - Medium Priority: 19
  - Low Priority: 6

  Test Types:
  - Functional: 11
  - Regression: 5
  - Smoke: 3
  - Edge Cases: 8
  - Security: 4
  - Performance: 4

  Output saved to: /home/bgudi/work/src/github.com/openshift/security-profiles-operator/testcases-json-enricher.md
  Format: markdown

  Summary

  I've generated comprehensive test cases for the JSON Enricher feature based on thorough analysis of the Security Profiles Operator codebase. The test cases are specifically tailored to this project and
  include:

  Key Features Tested:

  1. Core Functionality: Enabling enricher, capturing audit logs, enriching with metadata
  2. Configuration Options: File output, log rotation, audit intervals, verbosity
  3. Advanced Features: Enricher filters, eBPF fallback, webhook integration
  4. Edge Cases: Missing container info, short-lived processes, special characters
  5. Security: No secrets leakage, RBAC permissions, input sanitization
  6. Performance: High volume handling, startup time, cache eviction

  Context-Driven Test Generation:

  The test cases are based on actual codebase analysis:
  - ✅ Existing test files: Learned patterns from test/tc_json_enricher_test.go and unit tests
  - ✅ Implementation code: Analyzed jsonenricher.go, enricher types, and filters
  - ✅ Documentation: Extracted setup steps from audit-logging-guide.md and installation-usage.md
  - ✅ Real commands: Used actual kubectl commands, file paths, and configuration from the repo
  - ✅ Platform-specific: Noted Ubuntu ephemeral /proc issue and CRI-O privileged seccomp requirements

  Critical Test Cases (Quick Validation - ~40 minutes):

  10 high-priority test cases covering smoke tests, core functionality, regression, and security are highlighted for rapid validation.

  All test cases include detailed preconditions, step-by-step commands (from the actual codebase), and expected results based on the implementation.

Test case document created by slash command
testcases-json-enricher.md

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.

Summary by CodeRabbit

• New Features
  • Introduced Testing plugin for comprehensive test case generation and QA automation.
  • Added test case document generator with support for functional, regression, smoke, edge, security, and performance test categories.
  • Enabled multi-format output (Markdown and DOCX) with priority filtering and component tagging capabilities.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: BhargaviGudi
Once this PR has been reviewed and has the lgtm label, please assign dgoodwin for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

Hi @BhargaviGudi. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[stbenjam]

/ok-to-test

[openshift-ci]

@ngopalak-redhat: changing LGTM is restricted to collaborators

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[coderabbitai]

Walkthrough

This PR introduces a new Testing plugin that provides test case generation and QA automation capabilities. The plugin includes a generate-test-case-doc command that generates test case documents across multiple categories (functional, regression, smoke, edge, security, performance) in Markdown or DOCX formats. Supporting infrastructure includes plugin manifests, documentation, command definitions, a skill for DOCX conversion, and a Python script implementing Markdown-to-DOCX conversion logic.

Changes

Cohort / File(s)	Change Summary
Plugin Registry \.claude-plugin/marketplace.json, plugins/testing/\.claude-plugin/plugin.json	Register new testing plugin with name, source path, and description; define plugin metadata including version 0.0.1 and author information.
Documentation Registry PLUGINS.md, docs/data.json	Add Testing plugin entry to documentation TOC; register plugin with command metadata (generate-test-case-doc), skill metadata (Test Case Document Generator), and version info.
Plugin Documentation plugins/testing/README.md, plugins/testing/commands/generate-test-case-doc.md, plugins/testing/skills/testcase-doc-generator/SKILL.md	Introduce comprehensive plugin README, command documentation with usage examples and implementation outline, and skill documentation for DOCX generation workflow.
Implementation plugins/testing/skills/testcase-doc-generator/generate_docx.py	Add Python script for Markdown-to-DOCX conversion with document styling, table parsing, and support for headings, code blocks, bold text, and lists.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant Command
    participant Generator
    participant DOCX_Skill
    participant Output
    
    User->>CLI: /testing:generate-test-case-doc --priority high --format DOCX
    CLI->>Command: Parse arguments & validate options
    Command->>Generator: Analyze repo (docs, tests, code, config, deps)
    Generator->>Generator: Create test cases (functional, regression, smoke, edge, security, performance)
    Generator->>Generator: Filter by priority & component
    Generator->>Output: Generate test_cases.md
    
    alt Format == DOCX
        Command->>DOCX_Skill: convert_markdown_to_docx(test_cases.md)
        DOCX_Skill->>DOCX_Skill: Parse Markdown (headings, tables, code, lists)
        DOCX_Skill->>DOCX_Skill: Apply document styling (fonts, colors)
        DOCX_Skill->>Output: Generate test_cases.docx
    end
    
    Output-->>User: Returns output file paths & summary

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Python script complexity: generate_docx.py contains moderate logic for Markdown parsing (tables, code blocks, lists, frontmatter handling) and DOCX document construction with styled elements. Review should verify parsing edge cases and DOCX API usage correctness.
• Documentation completeness: Multiple documentation files introduce new command and skill; verify consistency between README, command docs, and skill docs regarding usage syntax and parameters.
• Plugin metadata accuracy: Cross-check plugin registry entries (marketplace.json, plugin.json, docs/data.json) for consistency in name, description, version, and command/skill definitions.

Pre-merge checks and finishing touches

✅ Passed checks (6 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title 'Add testing plugin for comprehensive test case generation' directly and specifically describes the main change: introducing a new testing plugin for test case generation.
No Real People Names In Style References	✅ Passed	No real people names found in style references, plugin commands, skill documentation, example prompts, or instructions across all PR files.
No Assumed Git Remote Names	✅ Passed	No hardcoded git remote names found. The testing plugin files do not assume git remotes like 'origin' or 'upstream' without discovery.
Git Push Safety Rules	✅ Passed	No git push operations or autonomous git commands detected in the PR changes.
No Untrusted Mcp Servers	✅ Passed	Pull request introduces a testing plugin with local files only; no MCP servers or untrusted external dependencies are installed.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

plugins/testing/skills/testcase-doc-generator/generate_docx.py (1)

277-281: Consider more specific exception handling.

The broad Exception catch is flagged by Ruff. While acceptable for a CLI tool, consider catching more specific exceptions for better error diagnosis.

Apply this diff for more specific error handling:

     # Convert
     try:
         convert_markdown_to_docx(args.input, args.output, args.title)
-    except Exception as e:
+    except (OSError, ValueError, UnicodeDecodeError) as e:
         print(f"Error converting markdown to DOCX: {e}", file=sys.stderr)
         sys.exit(1)
+    except Exception as e:
+        print(f"Unexpected error: {e}", file=sys.stderr)
+        sys.exit(1)

Based on learnings

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between f71e9be and e620814.

📒 Files selected for processing (8)

• .claude-plugin/marketplace.json (1 hunks)
• PLUGINS.md (2 hunks)
• docs/data.json (1 hunks)
• plugins/testing/.claude-plugin/plugin.json (1 hunks)
• plugins/testing/README.md (1 hunks)
• plugins/testing/commands/generate-test-case-doc.md (1 hunks)
• plugins/testing/skills/testcase-doc-generator/SKILL.md (1 hunks)
• plugins/testing/skills/testcase-doc-generator/generate_docx.py (1 hunks)

🧰 Additional context used

🪛 LanguageTool

plugins/testing/README.md

[uncategorized] ~183-~183: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...rsion Control**: Maintain test cases in markdown format alongside code - **Test Manageme...

(MARKDOWN_NNP)

plugins/testing/commands/generate-test-case-doc.md

[uncategorized] ~93-~93: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ..., docx` - If invalid, default to markdown with warning 3. **Analyze Feature Cont...

(MARKDOWN_NNP)

---

[grammar] ~616-~616: Use a hyphen to join words.
Context: ...est summary section ### Example 2: High priority test cases only ``` /testing:ge...

(QB_NEW_EN_HYPHEN)

---

[uncategorized] ~622-~622: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... high ``` Output: - Generates only High priority test cases - Useful for critical path t...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

---

[uncategorized] ~660-~660: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...curity --format docx ``` Output: - High priority test cases only - Tagged with payment a...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

---

[grammar] ~665-~665: Use a hyphen to join words.
Context: ...ecurity scenarios ### Example 7: Medium priority test cases for regression suite...

(QB_NEW_EN_HYPHEN)

---

[uncategorized] ~671-~671: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ium --component cart ``` Output: - Medium priority test cases - Suitable for extended regr...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

plugins/testing/skills/testcase-doc-generator/SKILL.md

[uncategorized] ~114-~114: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...error message} ``` Solution: Check markdown formatting, ensure valid UTF-8 encoding...

(MARKDOWN_NNP)

---

[uncategorized] ~257-~257: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...in DOCX Common causes: - Malformed markdown tables - Unclosed code blocks - Invalid...

(MARKDOWN_NNP)

---

[uncategorized] ~261-~261: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...haracters Solution: Review and fix markdown formatting ### Issue: Script not found...

(MARKDOWN_NNP)

🪛 markdownlint-cli2 (0.18.1)

plugins/testing/commands/generate-test-case-doc.md

16-16: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

337-337: Bare URL used

(MD034, no-bare-urls)

---

338-338: Bare URL used

(MD034, no-bare-urls)

---

339-339: Bare URL used

(MD034, no-bare-urls)

plugins/testing/skills/testcase-doc-generator/SKILL.md

16-16: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🪛 Ruff (0.14.3)

plugins/testing/skills/testcase-doc-generator/generate_docx.py

279-279: Do not catch blind exception: Exception

(BLE001)

🔇 Additional comments (12)

plugins/testing/.claude-plugin/plugin.json (1)

1-8: LGTM! Clean plugin manifest.

The plugin manifest follows the established structure with appropriate metadata for a new plugin release (v0.0.1).

PLUGINS.md (2)

18-18: LGTM! TOC entry added correctly.

The Testing plugin is properly added to the table of contents in alphabetical order.

---

176-183: LGTM! Plugin documentation section is well-structured.

The Testing plugin section follows the established documentation pattern with clear command description and synopsis.

.claude-plugin/marketplace.json (1)

82-86: LGTM! Marketplace entry is correct.

The testing plugin is properly registered in the marketplace with consistent metadata.

plugins/testing/skills/testcase-doc-generator/SKILL.md (1)

1-280: LGTM! Comprehensive skill documentation.

The skill documentation clearly explains the DOCX generation workflow, prerequisites, usage examples, and troubleshooting steps. The integration with the main command is well documented.

plugins/testing/README.md (1)

1-358: LGTM! Excellent plugin README.

The README provides comprehensive documentation covering all aspects of the plugin: overview, installation, usage examples, prerequisites, use cases for different roles, output formats, best practices, and troubleshooting. This will greatly help users adopt the plugin.

docs/data.json (1)

607-627: LGTM! Plugin registry entry is correct.

The testing plugin is properly registered in the data.json with complete metadata including commands, skills, and version information.

plugins/testing/skills/testcase-doc-generator/generate_docx.py (5)

27-35: LGTM! Proper dependency check with helpful error message.

The ImportError handling provides clear guidance to users on how to install the required python-docx library.

---

64-86: LGTM! Comprehensive document styling.

The styling setup creates a professional document appearance with consistent fonts, colors, and sizes for all heading levels and normal text.

---

88-99: LGTM! Robust markdown table parsing.

The table parser correctly handles markdown table syntax and filters out separator rows. Good use of set comparison to identify separator lines.

---

102-131: LGTM! Well-formatted table generation.

The table creation applies professional styling with bold headers, blue header background, and proper table formatting using the Light Grid Accent 1 style.

---

133-263: LGTM! Comprehensive markdown-to-DOCX conversion.

The conversion function handles all common markdown elements: frontmatter, code blocks, tables, headings, lists, bold text, and horizontal rules. The state machine approach with flags for code blocks and tables ensures proper processing.