CeWinFileCacheFS

A WinFsp-based hybrid cache filesystem for Compiler Explorer, designed to efficiently serve multiple MSVC compiler versions by caching frequently-used files locally while falling back to network shares for less-critical files.

Features

• Hybrid Caching: Intelligent caching of compiler executables and headers with LRU eviction
• Async Download Manager: Multi-threaded download system with configurable worker pools
• Prometheus Metrics: Comprehensive metrics collection for monitoring cache performance
• Always-Cached Directory Tree: Fast directory navigation with complete metadata caching
• Memory Cache Manager: High-performance in-memory caching with configurable size limits
• JSON Configuration: Flexible configuration with support for multiple compiler versions
• Cross-Platform Development: Windows target with macOS development and testing support

Prerequisites

For Production (Windows)

• Windows 10/11 or Windows Server 2016+
• WinFsp installed
• Visual Studio 2019+ or compatible C++ compiler with C++20 support
• CMake 3.20+

For Development (macOS/Linux)

• C++20 compatible compiler (GCC 10+, Clang 12+)
• CMake 3.20+
• prometheus-cpp (automatically downloaded via FetchContent)

Building

Native Windows Build (Recommended)

1. Clone this repository with submodules:

git clone --recursive https://github.com/your-org/ce-win-file-cache.git
cd ce-win-file-cache
git submodule update --init --recursive

2. Install WinFsp if not already installed

3. Run the native Windows build:

build-msvc.bat

This will:

• Set up the MSVC environment automatically
• Build with optimizations enabled
• Copy required DLLs and config files to the output directory

Building from WSL with MSVC

If developing from WSL, you can cross-compile using MSVC:

1. Important: Source must be on a Windows drive (e.g., /mnt/d/ or /mnt/c/)
2. Run the WSL build script:

./build-msvc.sh

Alternative Build Methods

Manual CMake Build

# Set up MSVC environment first (run from Developer Command Prompt)
mkdir build && cd build
cmake .. -G "Visual Studio 17 2022" -A x64
cmake --build . --config Release

Wine Build (Experimental)

For cross-platform development or CI on Linux:

./build-wine.sh
wine build-wine/bin/CeWinFileCacheFS.exe --help

macOS/Linux Development Build

For cross-platform development and testing:

# Build with integrated static analysis
./build-macos.sh

# Run comprehensive test suite
./run_all_tests.sh

# Available options
./run_all_tests.sh --help     # Show usage information  
./run_all_tests.sh --clean    # Clean build before testing
./run_all_tests.sh --quick    # Skip CMake configuration

The build-macos.sh script includes integrated clang-tidy static analysis (if available) to ensure code quality and catch potential issues during development.

The test runner builds and executes 12 comprehensive test programs:

• Cache operations and performance validation
• Async download manager with stress testing
• Prometheus metrics collection and validation
• Directory tree caching and navigation
• Configuration loading and validation
• Edge case handling and error scenarios
• Glob pattern matching with comprehensive unit tests
• JSON configuration parsing and validation

Testing the Build

After building, test the installation:

On Windows:

# Test config parsing
.\build-msvc\bin\CeWinFileCacheFS.exe --test-config

# Test path resolution
.\build-msvc\bin\CeWinFileCacheFS.exe --test-paths

# Test network mapping
.\build-msvc\bin\CeWinFileCacheFS.exe --test-network

# Run all tests
.\build-msvc\bin\CeWinFileCacheFS.exe --test

On macOS/Linux:

# Run comprehensive test suite
./run_all_tests.sh

Configuration

Create a compilers.json file (see included example):

{
  "global": {
    "total_cache_size_mb": 8192,
    "eviction_policy": "lru",
    "cache_directory": "D:\\CompilerCache",
    "download_threads": 6
  },
  "metrics": {
    "enabled": true,
    "bind_address": "127.0.0.1",
    "port": 8080,
    "endpoint_path": "/metrics"
  },
  "compilers": {
    "msvc-14.40": {
      "network_path": "\\\\127.0.0.1\\efs\\compilers\\msvc\\14.40.33807-14.40.33811.0",
      "cache_size_mb": 2048,
      "cache_always": [
        "bin/Hostx64/x64/*.exe",
        "bin/Hostx64/x64/*.dll",
        "include/**/*.h",
        "lib/x64/*.lib"
      ],
      "prefetch_patterns": [
        "include/**/*.h",
        "include/**/*.hpp"
      ]
    },
    "windows-kits-10": {
      "network_path": "\\\\127.0.0.1\\efs\\compilers\\windows-kits-10",
      "cache_size_mb": 1024,
      "cache_always": [
        "Include/**/*.h",
        "Lib/**/*.lib",
        "bin/**/*.exe"
      ],
      "prefetch_patterns": [
        "Include/**/*.h"
      ]
    },
    "ninja": {
      "network_path": "\\\\127.0.0.1\\efs\\compilers\\ninja",
      "cache_size_mb": 64,
      "cache_always": [
        "*.exe"
      ],
      "prefetch_patterns": []
    }
  }
}

Quick Start

For detailed installation and configuration instructions, see the Installation & Usage Guide.

Basic Usage

1. Install WinFsp from GitHub Releases
2. Configure your compilers in compilers.json (see example below)
3. Mount the filesystem:

   CeWinFileCacheFS.exe --config compilers.json --mount M:

4. Access your compilers through the cached mount point (e.g., M:\msvc-14.40\bin\cl.exe)

Example: Network Share to Cached Path

Network Location	Cached Path
Z:\compilers\msvc\14.40\bin\cl.exe	M:\msvc-14.40\bin\cl.exe
\\server\tools\ninja.exe	M:\tools\ninja.exe

Files accessed through the cached paths are automatically cached for faster subsequent access.

Command Line Options

Runtime Options

• -c, --config FILE: Configuration file (default: compilers.json)
• -m, --mount POINT: Mount point (default: M:)
• -u, --volume-prefix: Volume prefix for UNC paths
• -d, --debug [LEVEL]: Enable debug logging
• -h, --help: Show help message

Testing Options (No WinFsp required)

• -t, --test: Run all test modes without mounting
• --test-config: Test configuration file parsing only
• --test-paths: Test virtual path to network path resolution
• --test-network: Test network path mapping validation

Example Usage

# Test configuration before mounting
CeWinFileCacheFS.exe --test

# Mount to drive M: with default config
CeWinFileCacheFS.exe

# Mount to specific directory with custom config
CeWinFileCacheFS.exe --config my-compilers.json --mount C:\\compilers

# Enable debug logging
CeWinFileCacheFS.exe --debug --mount M:

# Test specific functionality
CeWinFileCacheFS.exe --test-paths --config compilers.json

Architecture

File States

• VIRTUAL: File exists in metadata only
• CACHED: File is stored locally
• PLACEHOLDER: Metadata exists, content fetched on demand
• FETCHING: Currently downloading from network
• NETWORK_ONLY: Always fetch from network

Cache Policies

• ALWAYS_CACHE: Permanently cache these files
• ON_DEMAND: Cache after first access
• NEVER_CACHE: Always fetch from network

Directory Structure

/
├── msvc-14.40/
│   ├── bin/Hostx64/x64/    # Executables (always cached)
│   ├── include/            # Headers (on-demand caching)
│   └── lib/x64/           # Libraries (on-demand caching)
├── windows-kits-10/
│   ├── Include/           # Windows SDK headers
│   ├── Lib/              # Windows SDK libraries
│   └── bin/              # SDK tools
└── ninja/
    └── ninja.exe         # Build system executable

Performance Considerations

• Executables (*.exe, *.dll) are cached on first access
• Headers are cached based on usage patterns
• Cache eviction uses LRU algorithm
• Background prefetching for common files
• Network timeouts handled gracefully

Troubleshooting

Common Issues

1. Mount fails: Ensure WinFsp is installed and you have admin privileges
2. Network access denied: Check credentials and network share permissions
3. Cache full: Increase total_cache_size_mb or clear cache directory

Debug Logging

Enable debug logging for troubleshooting:

CeWinFileCacheFS.exe --debug 1 --mount M:

Cache Management

Clear cache manually:

rmdir /s /q C:\CompilerCache

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

Continuous Integration

The project includes comprehensive CI/CD workflows:

• Linux CI: Runs comprehensive test suite on every push/PR with multiple Ubuntu versions and compilers
• Windows CI: Full WinFsp integration testing with MSVC builds on Windows runners
• Code Quality: Static analysis, warning checks, and documentation validation

All tests run on their target platforms - Linux tests cover core logic without WinFsp dependencies, while Windows tests validate the complete WinFsp integration.

Documentation

Comprehensive documentation is available in the docs/ directory:

• Installation & Usage Guide - Complete setup and configuration guide
• Architecture Overview - System design, components, and visual diagrams
• Windows CI Pipeline - Complete CI/CD implementation details
• Testing Guide - Test framework and validation procedures
• Caching Design - Cache algorithms and policies
• Async Download Flow - Multi-threaded download system
• Development Setup - Remote development configuration
• TODO List - Implementation roadmap and pending items

Monitoring and Metrics

The system provides comprehensive Prometheus metrics for monitoring cache performance:

Available Metrics

• Cache metrics: Hit/miss rates, cache size, eviction counts
• Download metrics: Queue depth, completion rates, failure reasons, duration histograms
• Filesystem metrics: Operation counts and file open duration
• Network metrics: Operation success rates and latency

Accessing Metrics

When metrics are enabled in configuration:

# View metrics in Prometheus format
curl http://127.0.0.1:8080/metrics

Metrics include dynamic labels for detailed analysis:

• Cache hits/misses by operation type
• Download failures by specific reason
• Network operations by success/failure status

Current Implementation Status

✅ Completed Components

• Memory Cache Manager: Full LRU caching with metrics integration
• Async Download Manager: Multi-threaded downloads with comprehensive testing
• Directory Tree Caching: Always-cached directory structure for fast navigation
• Prometheus Metrics: Complete metrics collection with dynamic labels
• JSON Configuration: Full configuration parsing and validation
• Glob Pattern Matching: Proper glob matching for file patterns (*, **, ?)
• Test Infrastructure: Comprehensive test suite with automated runner
• Windows CI/CD Pipeline: Complete MSVC build and WinFsp integration testing
• Cross-Platform Development: Linux development with Windows production target
• Security Hardening: Comprehensive memory safety analysis and fixes
• Thread Safety: Per-node locking for concurrent directory operations
• Static Analysis Integration: clang-tidy checks integrated into build process

📝 Remaining Work

• WinFsp Filesystem Operations: Complete file read/write operations
• Production Deployment: Enhanced logging, error recovery, and monitoring
• Performance Optimization: Profile and optimize cache algorithms

Roadmap

• WinFsp filesystem driver integration
• Write support for compiler outputs
• File integrity verification
• Compression for cached files
• GUI configuration tool
• Production logging and monitoring
• Docker container support