Implement automatic retry and better error recovery

## Problem

Users frequently get stuck when operations fail, with no automatic recovery or clear guidance on next steps. Error messages are generic and don't provide actionable solutions.

---

## Current Issues

- **No automatic retry mechanisms** for failed selectors (timing/race conditions fail without recovery)
- **Inconsistent error response formats** across tools (12+ message format variations)
- **Generic error messages** without context or actionable suggestions
- **No rollback capabilities** for failed multi-step operations
- ⚠️ **PARTIAL:** Cache invalidation infrastructure exists (implemented Aug 29, 2025 in commit `13c75e8`) but is NOT automatically called on operation failures

**Root Cause:** Current implementation has no retry logic - if a selector fails once due to timing, it fails permanently. The fallback system tries different selectors but never retries the same selector.

---

## Retry Strategy Specification

### Configuration
```typescript
interface RetryConfig {
  maxRetries: 3;              // Total attempts (initial + 2 retries)
  baseDelay: 100;             // 100ms initial delay
  backoffMultiplier: 2;       // Exponential backoff
  timeoutPerAttempt: 5000;    // 5s timeout per attempt
  retryableErrors: string[];  // Which errors trigger retry
}

const RETRY_CONFIG = {
  browser_click: { maxRetries: 3, baseDelay: 100, timeout: 5000 },
  browser_type: { maxRetries: 3, baseDelay: 100, timeout: 5000 },
  browser_navigate: { maxRetries: 2, baseDelay: 500, timeout: 10000 },
  executeTestStep: { maxRetries: 2, baseDelay: 200, timeout: 8000 }
};
```

### Retry Flow
1. **Initial attempt:** Try operation with selector
2. **On failure:** Classify error (transient vs permanent)
3. **If transient:** Invalidate cached selector
4. **Wait:** Exponential backoff delay (100ms → 200ms → 400ms)
5. **Retry:** Try next fallback selector from cache (25+ strategies)
6. **Repeat:** Up to maxRetries attempts
7. **Final failure:** Return standardized error with suggestions

### Retryable Operations
- `browser_click`, `browser_type`, `browser_navigate`
- `executeTestStep` (in test scenarios)
- All operations using `wrapSelectorOperation`

---

## Error Code Taxonomy

### Retryable Errors (Transient)
```typescript
enum RetryableError {
  SELECTOR_NOT_FOUND = 'SELECTOR_NOT_FOUND',       // Element selector didn't match
  ELEMENT_NOT_VISIBLE = 'ELEMENT_NOT_VISIBLE',     // Element exists but not visible
  ELEMENT_DETACHED = 'ELEMENT_DETACHED',           // Element removed from DOM
  NETWORK_TIMEOUT = 'NETWORK_TIMEOUT',             // Network request timed out
  NAVIGATION_FAILED = 'NAVIGATION_FAILED',         // Page navigation failed
  CONTEXT_DESTROYED = 'CONTEXT_DESTROYED',         // Browser context lost
}
```

**Retry Strategy:** 3 attempts with exponential backoff, try fallback selectors

### Non-Retryable Errors (Permanent)
```typescript
enum PermanentError {
  INVALID_SELECTOR = 'INVALID_SELECTOR',           // Malformed selector syntax
  SECURITY_VIOLATION = 'SECURITY_VIOLATION',       // Input validation failed
  INVALID_INPUT = 'INVALID_INPUT',                 // Invalid parameter values
  PERMISSION_DENIED = 'PERMISSION_DENIED',         // Browser permission blocked
}
```

**Retry Strategy:** Fail immediately with user guidance

### Special Handling
```typescript
enum SpecialError {
  BROWSER_CRASH = 'BROWSER_CRASH',                 // Restart browser, retry operation
  CACHE_CORRUPTION = 'CACHE_CORRUPTION',           // Invalidate cache, retry with fresh lookup
}
```

### Error Classification
```typescript
interface ErrorClassification {
  errorCode: ErrorCode;
  isTransient: boolean;        // Can retry help?
  maxRetries: number;          // How many attempts?
  requiresManual: boolean;     // Needs user intervention?
  canUseFallback: boolean;     // Try alternative selectors?
  shouldInvalidate: boolean;   // Remove from cache?
}

const ERROR_CATEGORIES: Record<ErrorCode, ErrorClassification> = {
  SELECTOR_NOT_FOUND: {
    isTransient: true,
    maxRetries: 3,
    requiresManual: false,
    canUseFallback: true,
    shouldInvalidate: true
  },
  SECURITY_VIOLATION: {
    isTransient: false,
    maxRetries: 0,
    requiresManual: true,
    canUseFallback: false,
    shouldInvalidate: false
  },
  // ... etc
};
```

---

## Standard Error Response Interface

```typescript
interface StandardErrorResponse {
  content: [{
    type: "text",
    text: string  // Human-readable error message
  }],
  isError: true,
  metadata: {
    errorCode: ErrorCode,           // E.g., "SELECTOR_NOT_FOUND"
    errorCategory: 'selector' | 'network' | 'validation' | 'timeout' | 'browser',
    operation: string,              // E.g., "browser_click", "browser_navigate"
    context: {
      url?: string,
      selector?: string,
      retryCount?: number,
      maxRetries?: number,
      attemptDuration?: number,     // How long each attempt took
      failedAttempts?: Array<{
        selector: string,
        error: string,
        duration: number
      }>
    },
    suggestion: {
      action: 'retry' | 'fallback' | 'manual' | 'abort',
      message: string,              // Actionable guidance
      alternativeSelectors?: string[]  // From cache fallback system
    },
    recovery?: {
      attempted: boolean,
      strategy?: 'cache_fallback' | 'retry_backoff' | 'browser_restart',
      result?: 'success' | 'failure',
      recoveryDuration?: number
    }
  }
}
```

### Example Error Response
```json
{
  "content": [{
    "type": "text",
    "text": "Click failed after 3 attempts: Element not found: button:has-text('Delete')"
  }],
  "isError": true,
  "metadata": {
    "errorCode": "SELECTOR_NOT_FOUND",
    "errorCategory": "selector",
    "operation": "browser_click",
    "context": {
      "url": "http://localhost:3000/todos",
      "selector": "button:has-text('Delete')",
      "retryCount": 3,
      "maxRetries": 3,
      "failedAttempts": [
        { "selector": "button:has-text('Delete')", "error": "Not found", "duration": 105 },
        { "selector": "*:has-text('Delete')", "error": "Not found", "duration": 210 },
        { "selector": "[role=button]:has-text('Delete')", "error": "Not found", "duration": 415 }
      ]
    },
    "suggestion": {
      "action": "manual",
      "message": "Element may be dynamically loaded. Try waiting for element or checking DOM structure.",
      "alternativeSelectors": [
        "button >> text='Delete'",
        "#delete-button",
        ".btn-delete"
      ]
    },
    "recovery": {
      "attempted": true,
      "strategy": "cache_fallback",
      "result": "failure",
      "recoveryDuration": 730
    }
  }
}
```

---

## Implementation Phases

### Phase 1: Error Standardization (Foundation) - 3-4 days
**Priority:** P0 (Required for all other phases)

**Deliverables:**
- [ ] Create `src/utils/error-formatter.ts` with `StandardErrorResponse` interface
- [ ] Define complete `ErrorCode` enum with all error types
- [ ] Implement `classifyError()` function with transient/permanent logic
- [ ] Add error metrics tracking to `PerformanceMonitor`
- [ ] Update all 26+ MCP tools to use standardized errors

**Files to Create:**
- `src/utils/error-formatter.ts` (new file, ~150 lines)
- `src/types/error-types.ts` (new file, ~100 lines)

**Files to Modify:**
- `src/mcp/server.ts` (update all tool error handlers)
- `src/core/performance-monitor.ts:43` (add error tracking metrics)

**Acceptance Criteria:**
- [ ] All MCP tools return `StandardErrorResponse` format
- [ ] Error responses include errorCode, context, and suggestions
- [ ] Error classification logic tested with 15+ scenarios

---

### Phase 2: Retry with Exponential Backoff - 4-5 days
**Priority:** P1 (High impact on user experience)

**Deliverables:**
- [ ] Create `src/core/retry-strategy.ts` with configurable retry logic
- [ ] Implement exponential backoff (100ms → 200ms → 400ms)
- [ ] Add retry logic to `wrapSelectorOperation` (tiered-cache.ts)
- [ ] Add retry to `executeTestStep` (server.ts:856)
- [ ] Integrate with cache fallback system (25+ strategies)

**Retry Implementation:**
```typescript
// src/core/retry-strategy.ts
export class RetryStrategy {
  async executeWithRetry<T>(
    operation: () => Promise<T>,
    config: RetryConfig,
    context: OperationContext
  ): Promise<T> {
    let lastError: Error;
    
    for (let attempt = 0; attempt < config.maxRetries; attempt++) {
      try {
        return await operation();
      } catch (error) {
        lastError = error;
        
        // Classify error
        const classification = classifyError(error);
        if (!classification.isTransient) {
          throw error;  // Don't retry permanent errors
        }
        
        // Invalidate cache if needed
        if (classification.shouldInvalidate) {
          await this.invalidateCache(context);
        }
        
        // Wait with exponential backoff
        if (attempt < config.maxRetries - 1) {
          const delay = config.baseDelay * Math.pow(config.backoffMultiplier, attempt);
          await new Promise(resolve => setTimeout(resolve, delay));
        }
      }
    }
    
    throw lastError;  // All retries failed
  }
}
```

**Files to Create:**
- `src/core/retry-strategy.ts` (new file, ~200 lines)

**Files to Modify:**
- `src/core/tiered-cache.ts:243` (add retry to wrapSelectorOperation)
- `src/mcp/server.ts:974` (add retry to browser_click)
- `src/mcp/server.ts:1041` (add retry to browser_type)
- `src/mcp/server.ts:911` (add retry to browser_navigate)
- `src/mcp/server.ts:856` (add retry to executeTestStep)

**Acceptance Criteria:**
- [ ] 3 retry attempts with exponential backoff (100ms, 200ms, 400ms)
- [ ] Fallback selectors tried between retries
- [ ] Success rate >95% for transient failures
- [ ] Retry metrics tracked (attempts, success rate per error type)

---

### Phase 3: Cache Invalidation Wiring - 2-3 days
**Priority:** P1 (Prevents cache corruption)

**Deliverables:**
- [ ] Wire `invalidateSelector()` calls in all failure paths
- [ ] Add automatic invalidation in `browser_click` failures
- [ ] Add automatic invalidation in `browser_type` failures
- [ ] Add automatic invalidation in `executeTestStep` failures
- [ ] Add metrics for invalidation frequency and impact

**Implementation Locations:**
```typescript
// src/mcp/server.ts:1007 - browser_click tool
try {
  const result = await enhancedCache.wrapSelectorOperationEnhanced(...);
  return { success: true, result };
} catch (error) {
  // NEW: Auto-invalidation on failure
  await enhancedCache.invalidateFailedSelector(selector, url);
  
  // NEW: Retry logic (from Phase 2)
  if (shouldRetry(error)) {
    return await retryOperation(...);
  }
  
  // NEW: Standardized error (from Phase 1)
  return formatStandardError(error, context);
}
```

**Files to Modify:**
- `src/core/enhanced-cache-integration.ts` (expose invalidation method publicly)
- `src/mcp/server.ts:1007` (browser_click invalidation)
- `src/mcp/server.ts:1073` (browser_type invalidation)
- `src/mcp/server.ts:825` (test step invalidation)

**Acceptance Criteria:**
- [ ] Failed selectors automatically removed from cache
- [ ] Cache integrity maintained at 100% (no bad selectors cached)
- [ ] No repeated failures from cached bad selectors
- [ ] Invalidation metrics tracked and reportable

---

### Phase 4: Operation Rollback (Optional) - 5-6 days
**Priority:** P2 (Nice to have, complex implementation)

**Deliverables:**
- [ ] Create `src/core/operation-rollback.ts` with rollback strategies
- [ ] Implement state snapshots for multi-step operations
- [ ] Add rollback to test scenario execution
- [ ] Add rollback to form fill operations
- [ ] Add rollback metrics and reporting

**Rollback Strategies:**
```typescript
interface RollbackStrategy {
  operation: 'test_scenario' | 'form_fill' | 'multi_step';
  stateSnapshots: Array<{
    url: string;
    formData?: Record<string, any>;
    timestamp: number;
  }>;
  
  async rollback(): Promise<void> {
    // For test scenarios: Navigate back to start URL
    // For form fills: Refresh page or clear inputs
    // For multi-step: Restore to last known good state
  }
}
```

**Scope:**
- **Test Scenarios:** Navigate back to start URL, clear execution records
- **Form Fills:** Refresh page or clear inputs
- **Multi-step Operations:** Restore to last known good state

**Files to Create:**
- `src/core/operation-rollback.ts` (new file, ~250 lines)
- `src/core/operation-snapshot.ts` (new file, ~150 lines)

**Files to Modify:**
- `src/core/test-scenario-cache.ts:367` (add rollback to executeTestSteps)
- `src/mcp/server.ts:653` (add rollback to browser_fill_form)

**Acceptance Criteria:**
- [ ] Failed multi-step operations can rollback to start state
- [ ] User doesn't need manual cleanup after failures
- [ ] Rollback success rate >90%
- [ ] Rollback metrics tracked

---

## Success Metrics & Measurement

### Baseline Metrics (Current State)
```typescript
interface ErrorRecoveryMetrics {
  totalOperations: number;
  failures: number;
  manualInterventions: number;  // User had to fix manually
  averageRecoveryTime: number;  // 0ms (no automatic recovery)
  stuckScenarios: number;       // Operations requiring manual intervention
}
```

### Target Metrics (After Implementation)
```typescript
interface TargetMetrics {
  automaticRecoveries: number;      // Target: >80% of failures
  manualInterventions: number;      // Target: <20% of failures
  averageRecoveryTime: number;      // Target: <30 seconds
  cacheIntegrity: number;           // Target: 100% (no bad selectors)
  retrySuccessRate: number;         // Target: >95% for transient errors
}
```

### Tracking Implementation
```typescript
// Add to src/core/performance-monitor.ts
interface ErrorMetrics {
  byErrorCode: Record<ErrorCode, {
    occurrences: number;
    recoveries: number;
    avgRetryAttempts: number;
    avgRecoveryTime: number;
  }>;
  
  overallRecoveryRate: number;      // % of errors automatically recovered
  manualInterventionRate: number;   // % requiring user action
}
```

### Reporting
```bash
# View error recovery metrics
npx claude-playwright cache debug --error-metrics

# Expected output:
Error Recovery Statistics:
- Total Operations: 1,250
- Automatic Recoveries: 1,050 (84%)  ✓ Target: >80%
- Manual Interventions: 200 (16%)    ✓ Target: <20%
- Avg Recovery Time: 28s             ✓ Target: <30s
- Cache Integrity: 100%              ✓ Target: 100%

By Error Type:
- SELECTOR_NOT_FOUND: 450 → 430 recovered (95.6%)
- NETWORK_TIMEOUT: 120 → 118 recovered (98.3%)
- ELEMENT_NOT_VISIBLE: 80 → 72 recovered (90.0%)
```

---

## Testing & Validation

### Phase 1 Tests (Error Standardization)
```bash
# Unit tests for error formatting
npx tsx tests/unit/error-formatter.test.ts

# Test scenarios:
- All error codes have proper classification
- Error responses include all required fields
- Suggestions are actionable and specific
- Context includes relevant debug information
```

### Phase 2 Tests (Retry Logic)
```bash
# Integration tests for retry behavior
npx tsx tests/integration/retry-strategy.test.ts

# Test scenarios:
- Retry with exponential backoff works correctly
- Transient errors retry, permanent errors don't
- Fallback selectors tried between retries
- Max retry limit respected
- Retry metrics tracked accurately
```

### Phase 3 Tests (Cache Invalidation)
```bash
# Cache integrity tests
npx tsx tests/integration/cache-invalidation.test.ts

# Test scenarios:
- Failed selectors removed from cache
- Cache hit rate maintained after invalidation
- No bad selectors remain in cache after failures
- Invalidation doesn't affect valid selectors
```

### Phase 4 Tests (Rollback - Optional)
```bash
# Rollback functionality tests
npx tsx tests/integration/operation-rollback.test.ts

# Test scenarios:
- Test scenario rollback restores initial state
- Form fill rollback clears inputs
- Multi-step rollback works correctly
- Rollback doesn't affect unrelated operations
```

---

## Acceptance Criteria

### Phase 1 (Error Standardization)
- [ ] All MCP tools return `StandardErrorResponse` format
- [ ] Error responses include: errorCode, context, suggestion, recovery
- [ ] 30+ error codes defined and categorized
- [ ] Error classification logic tested with 15+ scenarios

### Phase 2 (Retry Logic)
- [ ] Automatic retry with fallback selectors for failed clicks/types
- [ ] Exponential backoff implemented (100ms → 200ms → 400ms)
- [ ] Success rate >95% for transient failures
- [ ] Retry metrics tracked and reportable

### Phase 3 (Cache Invalidation)
- [ ] Failed selectors automatically invalidated from cache
- [ ] Cache integrity maintained at 100%
- [ ] Invalidation integrated in all operation failure paths
- [ ] No repeated failures from cached bad selectors

### Phase 4 (Rollback - Optional)
- [ ] Rollback mechanism for failed multi-step operations
- [ ] Test scenario rollback implemented
- [ ] Form fill rollback implemented
- [ ] Rollback success rate >90%

### Overall Success (All Phases)
- [ ] **80% reduction in "operation stuck" scenarios** (primary goal)
- [ ] Average recovery time <30 seconds
- [ ] Error messages include actionable next steps
- [ ] User experience significantly improved

---

## Estimated Effort

| Phase | Priority | Complexity | Effort | Dependencies |
|-------|----------|-----------|--------|--------------|
| Phase 1: Error Standardization | P0 | Medium | 3-4 days | None |
| Phase 2: Retry Logic | P1 | High | 4-5 days | Phase 1 |
| Phase 3: Cache Invalidation | P1 | Low | 2-3 days | Phase 1 (independent from Phase 2) |
| Phase 4: Rollback (Optional) | P2 | High | 5-6 days | Phases 1-3 |
| **Total (Required)** | | | **9-12 days** | Phases 1-3 |
| **Total (Complete)** | | | **14-18 days** | All phases |

**Recommended Approach:** Implement Phases 1-3 (required), evaluate Phase 4 based on user feedback.

---

## Priority

**High** - Directly impacts user productivity and tool reliability. 80% reduction in stuck scenarios is critical for production usage.

Related to #22

Phase	Priority	Complexity	Effort	Dependencies
Phase 1: Error Standardization	P0	Medium	3-4 days	None
Phase 2: Retry Logic	P1	High	4-5 days	Phase 1
Phase 3: Cache Invalidation	P1	Low	2-3 days	Phase 1 (independent from Phase 2)
Phase 4: Rollback (Optional)	P2	High	5-6 days	Phases 1-3
Total (Required)			9-12 days	Phases 1-3
Total (Complete)			14-18 days	All phases

Implement automatic retry and better error recovery #25

Description

Problem

Current Issues

Retry Strategy Specification

Configuration

Retry Flow

Retryable Operations

Error Code Taxonomy

Retryable Errors (Transient)

Non-Retryable Errors (Permanent)

Special Handling

Error Classification

Standard Error Response Interface

Example Error Response

Implementation Phases

Phase 1: Error Standardization (Foundation) - 3-4 days

Phase 2: Retry with Exponential Backoff - 4-5 days

Phase 3: Cache Invalidation Wiring - 2-3 days

Phase 4: Operation Rollback (Optional) - 5-6 days

Success Metrics & Measurement

Baseline Metrics (Current State)

Target Metrics (After Implementation)

Tracking Implementation

Reporting

Testing & Validation

Phase 1 Tests (Error Standardization)

Phase 2 Tests (Retry Logic)

Phase 3 Tests (Cache Invalidation)

Phase 4 Tests (Rollback - Optional)

Acceptance Criteria

Phase 1 (Error Standardization)

Phase 2 (Retry Logic)

Phase 3 (Cache Invalidation)

Phase 4 (Rollback - Optional)

Overall Success (All Phases)

Estimated Effort

Priority

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions