Skip to content

LOGGING.md

Latest commit

 

History

History
634 lines (490 loc) · 20.8 KB

LOGGING.md

File metadata and controls

634 lines (490 loc) · 20.8 KB

DEXBot2 Logging System - Complete Documentation

Table of Contents

  1. Quick Start
  2. Architecture Overview
  3. Configuration Guide
  4. Logging Categories
  5. Display Features
  6. Change Detection System
  7. Implementation Details
  8. Deduplication Strategy
  9. Troubleshooting & FAQ

Quick Start

Run Bot (No Config Changes Needed)

The logging system works out of the box with sensible defaults:

npm start  # Works with smart logging enabled

Optional: Tune Logging

Edit profiles/general.settings.json to customize:

{
  "LOG_LEVEL": "info",
  "LOGGING_CONFIG": {
    "categories": {
      "fundChanges": { "enabled": false },
      "orderStateChanges": { "enabled": true },
      "fillEvents": { "enabled": true },
      "boundaryEvents": { "enabled": true }
    }
  }
}

Then restart the bot:

npm start

Run Tests

npm test  # All 25 tests should pass

Architecture Overview

✅ Project Status: COMPLETE

All 7 implementation phases finished, legacy code removed, all tests passing (25/25).

Design: Hybrid Centralized Architecture

The logging system uses a semi-centralized (hybrid) architecture:

Centralized:

  • ✅ Logger Class (modules/order/logger.js - 467 lines) - Single source of logging logic
  • ✅ LoggerState (modules/order/logger_state.js - 141 lines) - Smart change detection
  • ✅ Configuration (modules/constants.js) - Central config management + override loading from general.settings.json
  • ✅ Data Flow - All calls → Logger → console.log

Distributed:

  • ⚠️ Log Call Sites - 161+ calls across 8 modules (strategy, manager, dexbot_class, accounting, sync_engine, startup_reconcile, utils, runner)

Why This Design?

Aspect Benefit
Centralized logic Easy to maintain, test, update logging behavior globally
Distributed call sites Modules know WHEN and WHAT to log (context-aware)
Configuration centralized Single control point for output without code changes
Smart change detection Prevents log spam (40-50% output reduction)
Clean separation Logger handles HOW, modules handle WHEN/WHAT, config decides WHETHER

System Architecture Diagram

┌─────────────────────────────────────────────────────────────┐
│                        APPLICATION                          │
├─────────────────────────────────────────────────────────────┤
│  Module A          Module B          Module C          ...   │
│  (strategy.js)     (manager.js)      (dexbot_class)          │
│  21 log calls      20 log calls      45 log calls            │
│        │                 │                  │                │
│        └─────────────────┴──────────────────┘                │
│                          │ (distributed)                     │
└──────────────────────────┼────────────────────────────────────┘
                           │
                    logger.log(msg, level)
                           │
                ┌──────────▼────────────┐
                │   Logger Class        │  ← CENTRALIZED
                │  (logger.js)          │
                │                       │
                │ • Check config        │
                │ • Check level         │
                │ • Format message      │
                │ • Run change detect   │
                │ • Output to stdout    │
                └──────────────┬─────────┘
                               │
                           console.log
                               │
                               ↓
                         PM2 → File Rotation

Configuration Guide

Configuration Hierarchy

Defaults (from constants.js) are overridden by profiles/general.settings.json:

constants.js DEFAULTS
├─ LOG_LEVEL: "debug"
├─ LOGGING_CONFIG with 6 categories
└─ All enabled by default

           ↓ OVERRIDDEN BY

general.settings.json (USER)
└─ LOGGING_CONFIG:
   ├─ fundChanges: enabled=false
   ├─ orderStateChanges: enabled=true
   └─ ...

           ↓ MERGED INTO

Logger Instance (ACTIVE)
├─ fundChanges: enabled=false  (from general.settings)
├─ orderStateChanges: enabled=true  (from general.settings)
├─ fillEvents: enabled=true  (from defaults)
└─ ...

Configuration Examples

Production (Minimal - 90%+ reduction)

{
  "LOG_LEVEL": "warn",
  "LOGGING_CONFIG": {
    "categories": {
      "fundChanges": { "enabled": false },
      "orderStateChanges": { "enabled": false },
      "fillEvents": { "enabled": false },
      "boundaryEvents": { "enabled": false }
    }
  }
}

Standard (Normal - 40-50% reduction)

{
  "LOG_LEVEL": "info",
  "LOGGING_CONFIG": {
    "categories": {
      "fundChanges": { "enabled": false },
      "orderStateChanges": { "enabled": true },
      "fillEvents": { "enabled": true },
      "boundaryEvents": { "enabled": true }
    }
  }
}

Debug (Full Verbosity - no reduction)

{
  "LOG_LEVEL": "debug",
  "LOGGING_CONFIG": {
    "changeTracking": { "enabled": true },
    "categories": {
      "fundChanges": { "enabled": true },
      "orderStateChanges": { "enabled": true },
      "fillEvents": { "enabled": true },
      "boundaryEvents": { "enabled": true },
      "errorWarnings": { "enabled": true },
      "edgeCases": { "enabled": true }
    },
    "display": {
      "fundStatus": { "enabled": true, "showDetailed": true },
      "gridDiagnostics": { "enabled": false },
      "statusSummary": { "enabled": false }
    }
  }
}

Logging Categories

6 Independent Categories

Each category can be enabled/disabled independently:

Category Level Default Purpose Use Case
fundChanges debug enabled Fund state changes Track fund balance updates in detail
orderStateChanges info enabled Order operations Monitor order placement, cancellation, state transitions
fillEvents info enabled Fill processing Track when orders are filled and how funds are updated
boundaryEvents info enabled Boundary recovery/shifts Monitor grid boundary adjustments and recovery
errorWarnings warn/error enabled All errors/warnings Critical issues and warnings only
edgeCases warn/error enabled Edge condition handling Unusual conditions that don't cause errors

Batch Processing & Recovery Categories

Log messages for fill batching and recovery system:

Log Type Module Example Purpose
[FILL-BATCH] dexbot_class.js [FILL-BATCH] Popping 3 fills (queue depth: 8) Diagnostic visibility into batch sizing decisions
[FILL-BATCH] dexbot_class.js [FILL-BATCH] Processing batch with 3 fills... Tracks batch start before rebalance pipeline
[RECOVERY] accounting.js [RECOVERY] Attempting recovery (attempt 2/5) Monitors recovery retry system (count+time-based)
[RECOVERY] accounting.js [RECOVERY] Recovery succeeded, resetting state Confirms successful recovery and reset
[RECOVERY-RESET] accounting.js [RECOVERY-RESET] Periodic 10min sync, resetting retry counter Shows periodic reset points
[ORPHAN-FILL] dexbot_class.js [ORPHAN-FILL] Skipping double-credit for stale-cleaned order 12345 Orphan-fill deduplication guard
[HARD-ABORT] dexbot_class.js [HARD-ABORT] Illegal state during batch processing with 12 ops Batch hard-abort with operation count telemetry
[COOLDOWN] dexbot_class.js [COOLDOWN] Arming maintenance cooldown after hard-abort Confirms cooldown consistency
[STALE-CANCEL] dexbot_class.js [STALE-CANCEL] Single-op batch stale recovery (fast-path) Fast-path recovery for single operations
[REMAINDER] grid.js [REMAINDER] Tracking per-slot allocations (actual: 450/500) Unallocated remainder accuracy tracking

Configuration for Batch Processing Logs

Enable batch processing diagnostics (useful during development/troubleshooting):

{
  "LOG_LEVEL": "debug",
  "LOGGING_CONFIG": {
    "categories": {
      "fillEvents": { "enabled": true },
      "errorWarnings": { "enabled": true }
    }
  }
}

Batch Processing Log Examples:

[FILL-BATCH] Measuring queue depth: 8 fills awaiting
[FILL-BATCH] Stress tier: [8,3] → batch size 3
[FILL-BATCH] Popping 3 fills from queue (5 remaining)
[FILL-BATCH] Processing batch: fill1@100.5, fill2@100.6, fill3@100.7
[FILL-BATCH] Crediting 45000 BTS proceeds (batched)
[FILL-BATCH] Rebalance: placed 4 orders, rotated 2 orders
[FILL-BATCH] Batch broadcast completed, persisting grid

[RECOVERY] Recovery attempt 1/5 triggered
[RECOVERY] Retrying with 60s minimum interval
[RECOVERY] Recovery succeeded, resetting state
[RECOVERY-RESET] Periodic 10min sync reset recovery counter

[ORPHAN-FILL] Received fill for order 12345
[ORPHAN-FILL] Checking stale-cleaned map...
[ORPHAN-FILL] Skipping double-credit for stale-cleaned order 12345

[HARD-ABORT] Batch execution failed: "Limit order 999 does not exist"
[HARD-ABORT] Cleaning up stale order 999
[HARD-ABORT] Illegal state during batch processing with 12 ops
[HARD-ABORT] Arming maintenance cooldown (50 cycles)

Configuration Example

Disable verbose fund logs but keep everything else:

{
  "LOGGING_CONFIG": {
    "categories": {
      "fundChanges": { "enabled": false }
    }
  }
}

Display Features

3 Optional On-Demand Displays

Display features are gated by configuration and can be forced via method parameters:

Feature Default Method Purpose
gridDiagnostics off logGridDiagnostics(mgr, ctx, true) Show active/spread/partial order summary
fundStatus off logFundsStatus(mgr, ctx, true) Show detailed fund breakdown
statusSummary off displayStatus(mgr, true) Show comprehensive account/order status

Enable Display Features

{
  "LOGGING_CONFIG": {
    "display": {
      "gridDiagnostics": { "enabled": true, "showOnDemandOnly": true },
      "fundStatus": { "enabled": true, "showDetailed": true },
      "statusSummary": { "enabled": true }
    }
  }
}

Change Detection System

How Smart Change Detection Works

The LoggerState module implements a smart change detection system to prevent log spam:

  1. Tracks State: Maintains history of previous values per logging category
  2. Detects Changes: Compares new state vs previous state using deep diff
  3. Applies Thresholds: Ignores minor changes (fund: 8 decimals, price: 4 decimals)
  4. Skips Redundant Logs: If no change detected, log is skipped
  5. Reduces Output: 40-50% fewer logs with standard config

Example: Fund Changes

// First call with fundChanges
logger.log('[FUND] available: 100.12345678', 'debug')  // ✓ LOGGED (first time)

// Same values, called again
logger.log('[FUND] available: 100.12345678', 'debug')  // ✗ SKIPPED (no change)

// Slight change (within precision threshold)
logger.log('[FUND] available: 100.12345679', 'debug')  // ✗ SKIPPED (< 0.00000001)

// Significant change
logger.log('[FUND] available: 100.50000000', 'debug')  // ✓ LOGGED (change detected)

Precision Thresholds

{
  "changeTracking": {
    "enabled": true,
    "ignoreMinor": {
      "fundPrecision": 8,      // Ignore changes < 0.00000001
      "pricePrecision": 4      // Ignore changes < 0.0001
    }
  }
}

Output Reduction Impact

Config Profile Output Reduction Lines/Cycle
Production 90%+ ~10 lines
Standard 40-50% ~100-150 lines
Debug 0% ~200+ lines

Implementation Details

Phase 1: Core Implementation (Steps 1-3)

Step 1: Configuration Infrastructure ✅

  • Added LOGGING_CONFIG to modules/constants.js (77 lines)
  • Implemented deep merge loading from general.settings.json
  • 6 categories with independent enable/disable
  • 3 optional display features
  • Configuration frozen/immutable after load

Step 2: State Tracking Module ✅

  • Created modules/order/logger_state.js (141 lines)
  • Change detection via deep diff algorithm
  • Change history tracking (last 100 per category)
  • Threshold-based filtering (fund: 8 decimals, price: 4 decimals)
  • Methods: detectChanges(), isSignificantChange(), recordChange(), reset()

Step 3: Logger Refactoring ✅

  • Updated modules/order/logger.js (467 lines, -50 lines legacy code)
  • Integrated LoggerState for smart change detection
  • Configuration-driven behavior
  • Config gating for display features
  • 100% backward compatible signatures

Phase 2: Integration (Steps 4-5)

Step 4: Call Site Verification ✅

  • Analyzed 161 logger.log() calls → All compatible
  • Analyzed 5 logFundsStatus() calls → All compatible
  • Analyzed 2 logGridDiagnostics() calls → All compatible
  • Analyzed 2 displayStatus() calls → All compatible
  • Result: Zero code changes needed in call sites

Step 5: Deduplication & Cleanup ✅

  • Removed 50 lines of unused legacy code from logger.js
  • Multi-level deduplication: configuration filtering + change detection + code cleanup

Phase 3: Testing & Documentation (Steps 6-7)

Step 6: Testing & Validation ✅

  • Fixed test_logger.js to use forceDetailed/forceOutput parameters
  • All 25 automated tests passing (100%)
  • Comprehensive coverage of all logging paths

Step 7: Comprehensive Documentation ✅

  • Multiple guides covering architecture, migration, implementation

Files Modified Summary

Created Files

  • modules/order/logger_state.js (141 lines) - State tracking & change detection
  • docs/LOGGING.md - This comprehensive guide

Modified Files

modules/constants.js

  • Added: LOGGING_CONFIG defaults (77 lines)
  • Added: Deep merge override loading (15 lines)
  • Added: LOGGING_CONFIG to exports and freeze
  • Impact: Configuration now centralized and user-configurable

modules/order/logger.js

  • Updated: Constructor to load config (30 lines modified)
  • Added: LoggerState integration (5 lines)
  • Modified: logFundsStatus() for change detection (50 lines)
  • Modified: logGridDiagnostics() to check config (5 lines)
  • Modified: displayStatus() to check config (5 lines)
  • Removed: logSnapshot() method (4 lines) - never called
  • Removed: logSnapshotComparison() method (50 lines) - never called
  • Net change: +28 lines modified, -50 lines removed

modules/order/sync_engine.js

  • Added: Comments indicating moved concern (3 lines)
  • Net change: -37 lines

Unchanged Files (100% Compatible)

  • modules/order/strategy.js - 21 logger.log() calls
  • modules/order/manager.js - 20 logger.log() calls
  • modules/dexbot_class.js - 45 logger.log() calls
  • modules/order/accounting.js - 8 logger.log() calls
  • modules/order/startup_reconcile.js - 37 logger.log() calls
  • modules/order/utils.js - 8 logger.log() calls
  • modules/order/runner.js - 1 logger.log() call
  • All test files - All compatible

Deduplication Strategy

Three-Level Deduplication

The logging system prevents redundancy at three levels:

Level 1: Configuration-Based Filtering

Users disable entire logging categories they don't need:

{
  "categories": {
    "fundChanges": { "enabled": false },  // Disable this category
    "orderStateChanges": { "enabled": true }
  }
}

Effect: Eliminates entire categories from output

Level 2: Smart Change Detection

LoggerState skips logging if values haven't changed:

// Repeated calls with same values
logger.log('[FUND] amount: 100.50', 'debug')  // ✓ Logged first time
logger.log('[FUND] amount: 100.50', 'debug')  // ✗ Skipped (no change)

Effect: 40-50% reduction in log lines

Level 3: Code-Level Cleanup

Removed truly obsolete code:

  • logSnapshot() method - never called
  • logSnapshotComparison() method - never called

Effect: Cleaner codebase, removed 48 lines of dead code

Deduplication Results

Strategy Impact Example
Config filtering Per-category Disable fundChanges → no fund logs
Change detection Per-message Same value = no log
Code cleanup Architectural Removed unused methods

Troubleshooting & FAQ

Q: Do I need to change my code?

A: No. Everything is backward compatible. All existing logger.log() calls work unchanged.

Q: How much output reduction can I expect?

A:

  • Standard config: 40-50% reduction
  • Production config: 90%+ reduction
  • Debug config: 0% reduction (all logs enabled)

Q: Can I use the old logging behavior?

A: Yes. Set LOG_LEVEL to 'debug' and enable all categories:

{
  "LOG_LEVEL": "debug",
  "LOGGING_CONFIG": {
    "categories": {
      "fundChanges": { "enabled": true },
      "orderStateChanges": { "enabled": true },
      "fillEvents": { "enabled": true },
      "boundaryEvents": { "enabled": true }
    }
  }
}

Q: Is there a way to revert?

A: Git has full history. But revert is not needed - new system is fully compatible.

Q: What about PM2 file rotation?

A: Works better now with ~50% fewer lines written per cycle. File rotation performance improves.

Q: Can I customize logging per bot?

A: Currently via general.settings.json (global). Per-bot configuration is a future enhancement.

Q: How do I see the change detection history?

A: LoggerState maintains last 100 state changes per category. This is internal for deduplication.

Q: What if I want to force a log even if value didn't change?

A: Use the forceOutput parameter when calling display methods:

manager.logger.logFundsStatus(manager, ctx, true)  // Force output even if unchanged
manager.logger.displayStatus(manager, true)        // Force comprehensive status

Q: How is the configuration loaded?

A:

  1. Defaults are set in modules/constants.js
  2. If profiles/general.settings.json exists, it's read
  3. User settings are deep merged with defaults
  4. Merged config is frozen (immutable)

Q: Can I have different configs for different bots?

A: Not yet. All bots use the same global config in profiles/general.settings.json. Per-bot config is planned for future.

Q: What happens if I put invalid JSON in general.settings.json?

A: The app warns with [WARN] Failed to load local settings and uses defaults instead.

Q: Are there any performance implications?

A: Change detection adds minimal overhead:

  • Deep diff is only on values that change
  • Precision thresholds prevent excessive comparisons
  • Overall: negligible performance impact, significant output reduction

Summary

✅ Status: PRODUCTION READY

The logging system has been completely refactored with:

  • ✅ Smart hybrid centralized architecture
  • ✅ Configuration-driven behavior (no code changes needed)
  • ✅ 40-50% output reduction (standard), up to 90% (production)
  • ✅ 100% backward compatibility
  • ✅ 25/25 tests passing
  • ✅ 48 lines of legacy code removed
  • ✅ Comprehensive documentation

Key Metrics

Metric Value
New files created 2 (logger_state.js, LOGGING.md)
Files modified 3 (constants.js, logger.js, sync_engine.js)
Lines added ~600 (config, state tracking)
Lines removed 48 (legacy code)
Breaking changes 0 (100% backward compatible)
Call site changes needed 0 (all compatible)
Test files passing 25/25 (100%)
Output reduction 40-50% (standard), 90%+ (production)

Deployment Checklist

  • ✅ Configuration infrastructure in place
  • ✅ State tracking module created
  • ✅ Logger refactored and tested
  • ✅ Call sites verified (backward compatible)
  • ✅ Redundant code removed
  • ✅ All 25 automated tests passing
  • ✅ Zero breaking changes
  • ✅ Production-ready

Quick Deployment

  1. No code changes needed - Drop-in replacement
  2. Works out of the box - Smart defaults enabled
  3. Optional tuning - Configure via profiles/general.settings.json
  4. Test thoroughly - All tests pass before deployment

No code changes needed. Deploy with confidence.