Adaptive Rate Limiting System Implementation

[AllyMurray]

Summary

This PR implements a sophisticated adaptive rate limiting system that dynamically allocates API capacity between user requests and background operations based on real-time activity patterns. This intelligent system maximizes API utilization while ensuring optimal user experience.

🚀 Key Features

Intelligent Capacity Management

• Peak Efficiency: During inactive periods (e.g., nighttime), background processes get 100% of API capacity (200/200 requests per hour)
• User Priority: During high activity, users receive up to 90% of capacity while background requests are paused
• Smooth Transitions: Gradual scaling between allocation strategies prevents abrupt changes
• Trend Awareness: Increasing user activity triggers immediate background pause to preserve capacity

Real-Time Adaptation

• 30-Second Recalculation: Capacity is recalculated every 30 seconds based on recent activity
• 15-Minute Monitoring Window: Activity patterns are analyzed over a rolling 15-minute window
• Sustained Inactivity Detection: After 30 minutes of zero user activity, background gets full capacity
• Automatic Recovery: System automatically adjusts when user activity resumes

Zero-Configuration Operation

• Smart Defaults: Works perfectly out-of-the-box with carefully tuned default settings
• Optional Customization: Advanced users can fine-tune thresholds and behaviors
• Backward Compatibility: Existing applications continue working without changes

🎯 Real-World Impact

Night Time Operation (Maximum Background Throughput)

Time: 2:00 AM (after 30+ minutes of zero activity)
Allocation: Background gets 200/200 requests per hour (100% capacity!)
Result: Maximum possible background throughput - 100% API utilization

Peak Hours (User Priority Mode)

Time: 10:00 AM (15 user requests in last 15 minutes, increasing trend)
Allocation: Users get 180/200 requests, background paused
Result: Users experience zero delays, system adapts instantly to demand

Mixed Workload (Balanced Mode)

Time: 8:00 AM (5 user requests in last 15 minutes, stable trend)
Allocation: Users get 100/200 requests, background gets 100/200 requests
Result: Both user and background operations proceed efficiently

📋 Implementation Details

Four Dynamic Allocation Strategies

1. High Activity Strategy (≥10 requests/15min)

   • User Capacity: Up to 180 requests/hour (90% of limit)
   • Background: Paused during increasing trends
   • Use Case: Peak usage periods

2. Moderate Activity Strategy (3-9 requests/15min)

   • User Capacity: 40-70% with dynamic scaling
   • Background: Remaining capacity
   • Use Case: Normal balanced usage

3. Zero Activity Strategy (0 requests/15min)

   • Initial: 5 user reserved, 195 background
   • Sustained (30+ min): 0 user reserved, 200 background (100%!)
   • Use Case: Nighttime, background-heavy processing

4. Low Activity Strategy (1-2 requests/15min)

   • User Capacity: 30% base allocation (60 requests/hour)
   • Background: 70% allocation (140 requests/hour)
   • Use Case: Light user activity

Architecture Components

• Priority-Based Request Classification: 'user' vs 'background' priority
• Activity Monitoring: Real-time tracking with configurable windows
• Dynamic Capacity Allocation: Four adaptive strategies
• Trend Detection: Analysis of activity trends
• Store Implementations: Both in-memory and SQLite-backed storage

🔧 Technical Changes

HTTP Client Integration

• Added priority parameter to HttpClient.get() method
• Implemented type safety with adaptive rate limit store detection
• Updated rate limiting calls to pass priority information throughout the pipeline

Resource API Updates

• Updated BaseResource.retrieve() and fetchPage() to extract and pass priority
• All Comic Vine resources now support priority options
• Maintained backward compatibility - priority parameters are optional

Store Implementations

In-Memory Store:

• AdaptiveRateLimitStore with configurable adaptive behavior
• Zero I/O overhead, automatic cleanup of old requests
• Perfect for single-process applications

SQLite Store:

• SqliteAdaptiveRateLimitStore with cross-process coordination
• Persistent storage with automatic schema migration
• Optimized indexes for efficient priority-based queries

Type Safety

• Runtime detection of adaptive features prevents errors with basic stores
• Comprehensive TypeScript support with proper type guards
• Full backward compatibility with existing RateLimitStore interface

📚 Usage Examples

Zero Configuration (Recommended)

import ComicVine from '@comic-vine/client';
import { AdaptiveRateLimitStore } from '@comic-vine/in-memory-store';

const client = new ComicVine({
  apiKey: 'your-api-key',
  stores: {
    rateLimit: new AdaptiveRateLimitStore(), // Uses smart defaults
  },
});

// User requests get priority automatically
const userIssue = await client.issue.retrieve(123, { priority: 'user' });

// Background requests are managed efficiently
const backgroundData = await client.issue.list({ 
  priority: 'background',
  limit: 100 
});

Custom Configuration

const client = new ComicVine({
  apiKey: 'your-api-key',
  stores: {
    rateLimit: new AdaptiveRateLimitStore({
      adaptiveConfig: {
        highActivityThreshold: 15,        // More tolerant of user activity
        sustainedInactivityThresholdMs: 45 * 60 * 1000, // Wait longer for full scale-up
        minUserReserved: 10,              // Guarantee more user requests
      },
    }),
  },
});

Status Monitoring

const status = await rateLimitStore.getStatus('issues');
console.log({
  userReserved: status.adaptive.userReserved,      // Capacity for users
  backgroundMax: status.adaptive.backgroundMax,    // Capacity for background
  backgroundPaused: status.adaptive.backgroundPaused, // Background pause status
  recentActivity: status.adaptive.recentUserActivity, // Recent user activity
  reason: status.adaptive.reason                   // Human-readable reason
});

🧪 Testing

Test Coverage

• 277 total tests across all packages
• 100% passing build with comprehensive test coverage
• Edge case handling for initial state, background-only activity, and sustained inactivity
• Integration tests for complete request pipeline
• Performance tests for minimal overhead verification

Test Categories

• Unit tests for each adaptive strategy scenario
• Integration tests for end-to-end priority behavior
• Edge case tests for concurrent requests and rapid priority changes
• Backward compatibility tests for existing functionality

📖 Documentation

Comprehensive README Updates

• Root README: Added adaptive rate limiting overview with benefits and quick start
• Client Package: Detailed API documentation with TypeScript examples
• In-Memory Store: Complete AdaptiveRateLimitStore configuration and features
• SQLite Store: Cross-process coordination and persistence capabilities

Documentation Features

• Real-world scenarios and use cases
• Migration guidance for existing users
• Performance considerations and best practices
• TypeScript examples with proper type annotations
• Zero-configuration quick start and advanced customization

🔄 Migration Path

Seamless Upgrade

// Before: Basic rate limiting
import { InMemoryRateLimitStore } from '@comic-vine/in-memory-store';

const client = new ComicVine({
  stores: { rateLimit: new InMemoryRateLimitStore() },
});

// After: Adaptive rate limiting (drop-in replacement)
import { AdaptiveRateLimitStore } from '@comic-vine/in-memory-store';

const client = new ComicVine({
  stores: { rateLimit: new AdaptiveRateLimitStore() },
});

Gradual Enhancement

1. Phase 1: Replace rate limit store with adaptive version
2. Phase 2: Add priority to user-facing requests
3. Phase 3: Add priority to background/batch operations
4. Phase 4: Customize configuration based on observed patterns

⚡ Performance Impact

Memory Usage

• In-Memory Store: ~1-2KB per monitored resource
• SQLite Store: ~1KB in-memory + database storage
• Automatic Cleanup: Expired request timestamps removed automatically

CPU Overhead

• Capacity Calculation: Every 30 seconds per resource
• Activity Tracking: Minimal overhead per request
• Trend Analysis: Simple mathematical operations

Network Benefits

• Zero Additional Requests: All calculations based on existing patterns
• Improved Efficiency: Better API utilization reduces total requests needed

🎉 Benefits Summary

For Application Developers

• Better User Experience: Interactive requests get priority during peak usage
• Maximum Efficiency: Background processes utilize full API capacity during off-hours
• Zero Configuration: Intelligent defaults work perfectly out-of-the-box
• Easy Migration: Drop-in replacement for existing rate limiting

For Production Deployments

• Cross-Process Coordination: SQLite store enables multi-instance deployments
• Persistent State: System remembers activity patterns across restarts
• Monitoring Integration: Detailed status information for observability
• Scalable Architecture: Handles high-throughput applications efficiently

🔍 Code Quality

• TypeScript First: Full type safety throughout the implementation
• Extensive Testing: 277 tests with 100% pass rate
• Backward Compatibility: Zero breaking changes for existing users
• Performance Optimized: Minimal overhead with intelligent caching
• Production Ready: Comprehensive error handling and edge case coverage

---

This adaptive rate limiting system represents a significant advancement in API client efficiency, providing intelligent resource management that adapts to real-world usage patterns while maintaining full backward compatibility and requiring minimal configuration.