From b02fc8d0fa5d8e8c6d0073c783ad2a1db77edc05 Mon Sep 17 00:00:00 2001 From: k3scat Date: Sat, 18 Jul 2026 12:57:07 -0400 Subject: [PATCH 1/4] docs: show key_blocking in the provider example configs The credential quarantine setting shipped with the starter config in nya/config.yaml and is listed in the README feature table, but none of the provider examples mentioned it, so anyone copying one as a starting point had no sign the option exists. Add the block to the openai, gemini, and novelai examples, mirroring the starter config's values (disabled, 403, 300s) rather than asserting per-provider status codes. Left disabled deliberately: quarantining on a wrongly chosen status code removes keys from rotation, which should be an explicit choice. Verified: all five configs validate against nya/schema.json and load through ConfigManager, which reports key_blocking for every API. --- configs/gemini.yaml | 4 ++++ configs/novelai.yaml | 4 ++++ configs/openai.yaml | 4 ++++ 3 files changed, 12 insertions(+) diff --git a/configs/gemini.yaml b/configs/gemini.yaml index faa0bb9..a3cfef5 100644 --- a/configs/gemini.yaml +++ b/configs/gemini.yaml @@ -49,6 +49,10 @@ default_settings: retry_after_seconds: 1 retry_request_methods: [ POST, GET, PUT, DELETE, PATCH, OPTIONS ] retry_status_codes: [ 429, 500, 502, 503, 504 ] + key_blocking: + enabled: false # Set to true to take a key out of rotation after the statuses below + status_codes: [ 403 ] + duration_seconds: 300 # How long a quarantined key stays unavailable timeouts: request_timeout_seconds: 300 diff --git a/configs/novelai.yaml b/configs/novelai.yaml index 216f99d..a5b0aa9 100644 --- a/configs/novelai.yaml +++ b/configs/novelai.yaml @@ -49,6 +49,10 @@ default_settings: retry_after_seconds: 1 retry_request_methods: [ POST, GET, PUT, DELETE, PATCH, OPTIONS ] retry_status_codes: [ 429, 500, 502, 503, 504 ] + key_blocking: + enabled: false # Set to true to take a key out of rotation after the statuses below + status_codes: [ 403 ] + duration_seconds: 300 # How long a quarantined key stays unavailable timeouts: request_timeout_seconds: 300 diff --git a/configs/openai.yaml b/configs/openai.yaml index a0dacab..2117ea1 100644 --- a/configs/openai.yaml +++ b/configs/openai.yaml @@ -49,6 +49,10 @@ default_settings: retry_after_seconds: 1 retry_request_methods: [ POST, GET, PUT, DELETE, PATCH, OPTIONS ] retry_status_codes: [ 429, 500, 502, 503, 504 ] + key_blocking: + enabled: false # Set to true to take a key out of rotation after the statuses below + status_codes: [ 403 ] + duration_seconds: 300 # How long a quarantined key stays unavailable timeouts: request_timeout_seconds: 300 From ddede03265c6dabd28312cf24c6328b4062af330 Mon Sep 17 00:00:00 2001 From: k3scat Date: Sat, 18 Jul 2026 14:37:06 -0400 Subject: [PATCH 2/4] fix: apply config changes reliably and keep quotas across restarts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Editing configuration through the UI did not take effect. Three separate faults, each of which alone was enough to break it: 1. Docker could not save at all. Nacho writes config by renaming a temporary file over the target, and that rename fails with EBUSY against a bind-mounted file: StorageError: Cannot save /app/config.yaml: [Errno 16] Resource busy: '/app/.config.yaml.qk9ed65s.tmp' -> '/app/config.yaml' The documented compose file also mounted it read-only. Mount the directory, read-write, and run as a user that can write it. 2. The image shipped --no-reload, which makes trigger_reload a no-op. Restarting is how a config change is applied, so this stranded every edit until the container was restarted by hand. 3. Restarting reset the state that enforces quotas. Rate-limit windows and key cool-downs live only in memory, so each edit handed out a fresh burst allowance and released every quarantined key — also true of any ordinary deploy or crash. Persist limiter windows and cool-downs across restarts. Limiters are rebuilt lazily so the rate always comes from the current configuration and only the consumed window carries over; a limit raised while the process was down therefore applies immediately. Entries are keyed by a hash of the limiter name because that name embeds the upstream credential and client IPs, and the file is written 0600 via a temporary file and rename. A missing, corrupt, or version-mismatched file is ignored: cold counters are a degradation, refusing to start is an outage. Also debounce reload triggers. A single save emits several change events and every restart drops in-flight requests, including streaming responses. The state file lives beside the configuration, taken from the config manager rather than CONFIG_PATH — the path can arrive as a constructor argument, and falling back to the working directory made two instances started from one directory share quotas. That bug surfaced as e2e tests leaking rate-limit state between runs. Not persisted: queued requests. Each owns an asyncio.Future tied to a client socket that does not survive the restart, so replaying one would spend upstream quota on a response nobody is waiting for. Shutdown already fails them so clients can retry. Verified end to end in Docker: a UI save now returns 200, persists to the host, triggers "WatchFiles detected changes in 'watch.txt'. Reloading...", and writes .nya_state.json on the way out. 28 new tests, checked by mutation: disabling restoration fails 4 of them, and removing the name hashing fails the credential-leak test. --- .gitignore | 3 + Dockerfile | 5 +- docker-compose.yml | 16 +- nya/common/constants.py | 4 + nya/core/control.py | 47 +++- nya/server/app.py | 74 ++++++ nya/services/limit.py | 58 ++++- nya/services/state.py | 114 +++++++++ tests/unit/test_server_app.py | 13 ++ tests/unit/test_state_persistence.py | 336 +++++++++++++++++++++++++++ 10 files changed, 665 insertions(+), 5 deletions(-) create mode 100644 nya/services/state.py create mode 100644 tests/unit/test_state_persistence.py diff --git a/.gitignore b/.gitignore index 46cb3c3..7d2a6da 100644 --- a/.gitignore +++ b/.gitignore @@ -179,3 +179,6 @@ watch.txt config.yaml # Serena MCP tool state .serena/ + +# Runtime state (rate-limit windows, key cool-downs) persisted across restarts +.nya_state.json diff --git a/Dockerfile b/Dockerfile index b43b7e8..a7fd4c2 100644 --- a/Dockerfile +++ b/Dockerfile @@ -38,4 +38,7 @@ HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \ CMD wget -q -O - http://127.0.0.1:8080/health || exit 1 ENTRYPOINT ["nyaproxy"] -CMD ["--config", "/app/config.yaml", "--host", "0.0.0.0", "--no-reload"] +# Hot-reload stays on: restarting is how a configuration change is applied, so +# --no-reload silently strands every edit made through the config UI. Rate-limit +# windows and key cool-downs survive the restart (see nya/services/state.py). +CMD ["--config", "/app/config.yaml", "--host", "0.0.0.0"] diff --git a/docker-compose.yml b/docker-compose.yml index 66de399..a3fd41f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -6,5 +6,17 @@ services: ports: - "127.0.0.1:8080:8080" volumes: - # Create config.yaml first and replace every placeholder credential. - - ./config.yaml:/app/config.yaml:ro + # Mount the DIRECTORY, not config.yaml itself. Saving from the config UI + # writes a temporary file and renames it over the target; that rename + # fails with EBUSY against a bind-mounted file, so every save through the + # UI returns a 500. Read-write for the same reason. + # + # Put your config.yaml in ./data/ before starting, and replace every + # placeholder credential. NyaProxy also writes .nya_state.json here so + # rate-limit windows and key cool-downs survive the restart that applies + # a configuration change. + - ./data:/app + # The image runs as uid 100, which cannot write a directory owned by you. + # This runs the container as your own user instead; the alternative is + # `sudo chown -R 100:101 ./data`. + user: "${UID:-1000}:${GID:-1000}" diff --git a/nya/common/constants.py b/nya/common/constants.py index 14ebb81..b566363 100644 --- a/nya/common/constants.py +++ b/nya/common/constants.py @@ -55,3 +55,7 @@ } WATCH_FILE = "watch.txt" + +# A single save in the config UI can emit several change events. Restarts drop +# in-flight requests, so a burst of edits is collapsed into one reload. +RELOAD_DEBOUNCE_SECONDS = 2.0 diff --git a/nya/core/control.py b/nya/core/control.py index 9c06057..7c577f7 100644 --- a/nya/core/control.py +++ b/nya/core/control.py @@ -4,11 +4,12 @@ import asyncio import time -from typing import TYPE_CHECKING, Dict, Optional, Tuple, Union +from typing import TYPE_CHECKING, Any, Dict, Optional, Tuple, Union from ..common.exceptions import APIKeyNotConfiguredError from ..services.lb import LoadBalancer from ..services.limit import RateLimiter +from ..services.state import state_key if TYPE_CHECKING: from ..config.manager import ConfigManager @@ -40,6 +41,9 @@ def __init__(self, config: "ConfigManager"): self.rate_limiters: Dict[str, RateLimiter] = {} self._limiter_prune_interval_seconds = 60.0 self._last_limiter_prune = 0.0 + # Limiter state carried over from a previous process, applied lazily + # as each limiter is recreated. See import_state. + self._restorable_state: Dict[str, Any] = {} self._lock = asyncio.Lock() @@ -69,12 +73,53 @@ def get_or_create_limiter(self, name: str, rate_limit) -> RateLimiter: self._prune_idle_limiters() limiter = self.rate_limiters.get(name) if not limiter: + # The rate always comes from the current configuration; only the + # consumed window and cool-down are carried over from a previous + # run, so a limit edited across a restart takes effect. limiter = RateLimiter(rate_limit=rate_limit) + restored = self._restorable_state.pop(state_key(name), None) + if restored: + limiter.restore_state(restored) self.rate_limiters[name] = limiter else: limiter.touch() return limiter + def export_state(self) -> Dict[str, Any]: + """ + Snapshot limiter state for the next process. + + Restarting is how NyaProxy applies configuration changes, so without + this every edit would hand out a fresh burst allowance and release + every quarantined key. + + Entries are keyed by a hash of the limiter name, because the name + itself contains the upstream credential or a client IP. + """ + limiters = {} + for name, limiter in self.rate_limiters.items(): + state = limiter.export_state() + if state: + limiters[state_key(name)] = state + return {"rate_limiters": limiters} + + def import_state(self, state: Dict[str, Any]) -> int: + """ + Stage limiter state from a previous run. + + Nothing is applied here: limiters are rebuilt lazily on first use so + they pick up the current configuration, and the staged entry is + applied at that point. Entries for limiters that are never touched + again are simply dropped. + """ + staged = state.get("rate_limiters") or {} + if not isinstance(staged, dict): + return 0 + self._restorable_state = { + name: value for name, value in staged.items() if isinstance(value, dict) + } + return len(self._restorable_state) + def _prune_idle_limiters(self) -> None: """ Remove stale per-client limiters to avoid unbounded growth. diff --git a/nya/server/app.py b/nya/server/app.py index 76769d1..0a6272a 100644 --- a/nya/server/app.py +++ b/nya/server/app.py @@ -7,6 +7,8 @@ import contextlib import os import sys +import time +from pathlib import Path import uvicorn from fastapi import FastAPI, Request @@ -20,6 +22,7 @@ DEFAULT_HOST, DEFAULT_PORT, DEFAULT_SCHEMA_NAME, + RELOAD_DEBOUNCE_SECONDS, WATCH_FILE, ) from ..common.models import ProxyRequest @@ -27,6 +30,7 @@ from ..core.proxy import NyaProxyCore from ..dashboard.api import DashboardAPI from ..services.metrics import PROMETHEUS_CONTENT_TYPE, MetricsCollector +from ..services.state import load_state, resolve_state_path, save_state from .auth import AuthManager, AuthMiddleware logger.remove() @@ -237,6 +241,9 @@ async def init_nya_services(self): self.init_metrics_collector() self.init_core() + # Rate-limit windows and key cool-downs must survive the restart + # that applies a configuration change. + self.restore_runtime_state() # Mount sub-applications for NyaProxy if available self.init_config_ui() @@ -378,6 +385,52 @@ def setup_proxy_routes(self): async def proxy_request(request: Request): return await self.generic_proxy_request(request) + def _state_path(self) -> Path: + """ + Location of the file holding state that outlives the process. + + The configured path is taken from the config manager rather than the + environment: the path can also arrive as a constructor argument, and + falling back to the working directory would make two instances + started from the same directory share one another's quotas. + """ + config_path = getattr(self.config, "config_path", None) or os.environ.get( + "CONFIG_PATH" + ) + return resolve_state_path(config_path) + + def restore_runtime_state(self) -> int: + """ + Re-apply rate-limit windows and key cool-downs from the previous run. + + Returns the number of staged limiter entries. Failure here is never + fatal: cold counters are a degradation, refusing to start is an + outage. + """ + if not self.core or not getattr(self.core, "control", None): + return 0 + try: + state = load_state(self._state_path()) + if not state: + return 0 + staged = self.core.control.import_state(state) + if staged: + logger.info(f"Restored rate-limit state for {staged} limiters") + return staged + except Exception as e: + logger.warning(f"Could not restore runtime state: {e}") + return 0 + + def persist_runtime_state(self) -> bool: + """Write rate-limit windows and key cool-downs for the next run.""" + if not self.core or not getattr(self.core, "control", None): + return False + try: + return save_state(self._state_path(), self.core.control.export_state()) + except Exception as e: + logger.warning(f"Could not persist runtime state: {e}") + return False + async def shutdown(self): """ Clean up resources on shutdown. @@ -385,6 +438,10 @@ async def shutdown(self): logger.info("Shutting down NyaProxy") + # Persist before tearing anything down, so a restart triggered by a + # config change does not reset quotas or release quarantined keys. + self.persist_runtime_state() + # Close proxy handler client if self.core and hasattr(self.core, "request_executor"): try: @@ -449,16 +506,33 @@ def create_app(): return nya_proxy_app.app +_last_reload_trigger = 0.0 + + def trigger_reload(**kwargs): """ Trigger a reload of the application by touching the file uvicorn watches. + + Restarting is how a configuration change is applied, so this is on the + path of every edit made through the config UI. """ + global _last_reload_trigger + if os.environ.get("DISABLE_HOT_RELOAD"): logger.warning( "Configuration changed but hot-reload is disabled; " "restart NyaProxy to apply the new settings" ) return + + # A UI save can emit several change events in quick succession, and each + # restart drops in-flight requests. Collapse a burst into one reload. + now = time.monotonic() + if now - _last_reload_trigger < RELOAD_DEBOUNCE_SECONDS: + logger.debug("Configuration changed again; reload already pending") + return + _last_reload_trigger = now + logger.info("Configuration changed, triggering reload...") with open(WATCH_FILE, "a") as f: f.write("reload\n") diff --git a/nya/services/limit.py b/nya/services/limit.py index 6bc5e3e..8965b93 100644 --- a/nya/services/limit.py +++ b/nya/services/limit.py @@ -5,7 +5,7 @@ import re import time from collections import deque -from typing import Deque, Optional, Tuple +from typing import Any, Deque, Dict, Optional, Tuple from ..common.exceptions import ConfigurationError @@ -183,3 +183,59 @@ def clear(self) -> None: self.touch() self.request_timestamps = deque() self.blocked_until = 0.0 + + def export_state(self) -> Optional[Dict[str, Any]]: + """ + Snapshot the state worth surviving a restart, or ``None`` if there is + nothing to keep. + + Only the consumed window and an active cool-down matter. ``locked`` + is deliberately excluded: it tracks in-flight concurrency for a + process that is going away, so restoring it would strand a key. + """ + now = time.time() + if self.window_seconds: + timestamps = [ + t for t in self.request_timestamps if now - t < self.window_seconds + ] + else: + timestamps = [] + blocked_until = self.blocked_until if self.blocked_until > now else 0.0 + if not timestamps and not blocked_until: + return None + return { + "rate_limit": self.rate_limit, + "timestamps": timestamps, + "blocked_until": blocked_until, + } + + def restore_state(self, state: Dict[str, Any]) -> None: + """ + Re-apply a snapshot from ``export_state``. + + The window is re-filtered against *this* limiter's configuration, so a + rate limit edited while the process was down still applies. Timestamps + in the future are dropped: a backwards clock change or a hand-edited + state file must not be able to hold a limiter shut indefinitely. + """ + now = time.time() + timestamps = [] + for raw in state.get("timestamps") or []: + try: + value = float(raw) + except (TypeError, ValueError): + continue + if value > now: + continue + if self.window_seconds and now - value >= self.window_seconds: + continue + timestamps.append(value) + + self.request_timestamps = deque(sorted(timestamps)) + + try: + blocked_until = float(state.get("blocked_until") or 0.0) + except (TypeError, ValueError): + blocked_until = 0.0 + self.blocked_until = blocked_until if blocked_until > now else 0.0 + self.touch() diff --git a/nya/services/state.py b/nya/services/state.py new file mode 100644 index 0000000..7067e60 --- /dev/null +++ b/nya/services/state.py @@ -0,0 +1,114 @@ +""" +Durable runtime state across restarts. + +Restarting is how NyaProxy applies a configuration change, so anything the +gateway relies on to enforce quotas has to outlive the process. Without this, +every edit (and every deploy, and every crash) would hand out a fresh burst +allowance and release every quarantined key. + +Only rate-limit windows and key cool-downs are stored. In-flight and queued +requests deliberately are not: a queued request owns an ``asyncio.Future`` +tied to a client socket that does not survive the restart, so replaying one +would spend upstream quota on a response nobody is waiting for. +""" + +import hashlib +import json +import os +import tempfile +from pathlib import Path +from typing import Any, Dict, Optional, Union + +from loguru import logger + +STATE_VERSION = 1 + + +def state_key(name: str) -> str: + """ + Stable, non-reversible identifier for a limiter name. + + Limiter names embed the upstream credential itself (``_key_``) + and client IPs, none of which belong in a file on disk. Hashing keeps the + mapping usable across a restart while keeping the file free of secrets. + """ + return hashlib.sha256(name.encode("utf-8")).hexdigest() + + +def save_state(path: Union[str, Path], payload: Dict[str, Any]) -> bool: + """ + Write ``payload`` atomically, returning whether it was persisted. + + Never raises: failing to save runtime state must not turn a clean + shutdown into a crash. + """ + target = Path(path) + document = {"version": STATE_VERSION, **payload} + try: + target.parent.mkdir(parents=True, exist_ok=True) + # Write-and-rename so a crash mid-write cannot leave a torn file that + # the next start would have to reject. + fd, tmp_name = tempfile.mkstemp( + dir=str(target.parent), prefix=target.name, suffix=".tmp" + ) + try: + with os.fdopen(fd, "w", encoding="utf-8") as handle: + json.dump(document, handle) + os.chmod(tmp_name, 0o600) + os.replace(tmp_name, target) + except BaseException: + # mkstemp already created the file, so clean it up on any failure. + try: + os.unlink(tmp_name) + except OSError: + pass + raise + return True + except Exception as exc: # pragma: no cover - defensive + logger.warning(f"Could not persist runtime state to {target}: {exc}") + return False + + +def load_state(path: Union[str, Path]) -> Dict[str, Any]: + """ + Read state written by ``save_state``. + + A missing, unreadable, corrupt, or version-mismatched file yields an empty + result rather than an error: starting with cold counters is a degradation, + but refusing to start is an outage. + """ + target = Path(path) + if not target.exists(): + return {} + try: + with target.open("r", encoding="utf-8") as handle: + document = json.load(handle) + except (OSError, ValueError) as exc: + logger.warning(f"Ignoring unreadable runtime state at {target}: {exc}") + return {} + + if not isinstance(document, dict): + logger.warning(f"Ignoring malformed runtime state at {target}") + return {} + + version = document.get("version") + if version != STATE_VERSION: + logger.info( + f"Ignoring runtime state at {target}: version {version!r} " + f"is not {STATE_VERSION}" + ) + return {} + + return {k: v for k, v in document.items() if k != "version"} + + +def resolve_state_path(config_path: Optional[str]) -> Path: + """ + Place the state file next to the configuration it belongs to. + + Keeping the two together means a bind-mounted config directory carries the + state with it, which is what makes limits survive a container restart. + """ + if config_path: + return Path(config_path).resolve().parent / ".nya_state.json" + return Path.cwd() / ".nya_state.json" diff --git a/tests/unit/test_server_app.py b/tests/unit/test_server_app.py index 1900669..1719b4d 100644 --- a/tests/unit/test_server_app.py +++ b/tests/unit/test_server_app.py @@ -1,6 +1,7 @@ import argparse import contextlib import os +import time from types import SimpleNamespace import pytest @@ -391,14 +392,26 @@ def test_parse_args_and_trigger_reload(monkeypatch, tmp_path): watch_file = tmp_path / "reload.watch" monkeypatch.setattr(server_app, "WATCH_FILE", str(watch_file)) monkeypatch.delenv("DISABLE_HOT_RELOAD", raising=False) + monkeypatch.setattr(server_app, "_last_reload_trigger", 0.0) + # A save in the config UI can emit several change events; each restart + # drops in-flight requests, so a burst must collapse into one reload. server_app.trigger_reload() server_app.trigger_reload() + assert watch_file.read_text() == "reload\n" + # Once the debounce window has passed, a later change reloads again. + monkeypatch.setattr( + server_app, + "_last_reload_trigger", + time.monotonic() - server_app.RELOAD_DEBOUNCE_SECONDS - 1, + ) + server_app.trigger_reload() assert watch_file.read_text() == "reload\nreload\n" # With hot-reload disabled the watch file must stay untouched monkeypatch.setenv("DISABLE_HOT_RELOAD", "1") + monkeypatch.setattr(server_app, "_last_reload_trigger", 0.0) server_app.trigger_reload() assert watch_file.read_text() == "reload\nreload\n" diff --git a/tests/unit/test_state_persistence.py b/tests/unit/test_state_persistence.py new file mode 100644 index 0000000..9383681 --- /dev/null +++ b/tests/unit/test_state_persistence.py @@ -0,0 +1,336 @@ +""" +Runtime state that has to survive the restart used to apply config changes. + +The property under test throughout: a restart must not hand out a fresh burst +allowance or release a quarantined key, while still picking up a rate limit +that was edited while the process was down. +""" + +import json +import os +import stat +import time +from types import SimpleNamespace + +from nya.core.control import TrafficManager +from nya.server import app as server_app +from nya.services.limit import RateLimiter +from nya.services.state import ( + STATE_VERSION, + load_state, + resolve_state_path, + save_state, + state_key, +) + + +class StubConfig: + """Minimal ConfigManager stand-in whose limits can change between runs.""" + + def __init__(self, key_rate="2/m", endpoint_rate="0", config_path=None): + self.key_rate = key_rate + self.endpoint_rate = endpoint_rate + self.config_path = config_path + + def get_api_key_rate_limit(self, api_name): + return self.key_rate + + def get_api_endpoint_rate_limit(self, api_name): + return self.endpoint_rate + + def get_api_ip_rate_limit(self, api_name): + return "0" + + def get_api_user_rate_limit(self, api_name): + return "0" + + +# --- RateLimiter snapshots --------------------------------------------------- + + +def test_export_returns_none_when_there_is_nothing_worth_keeping(): + assert RateLimiter("10/m").export_state() is None + + +def test_consumed_window_survives_a_restart(): + limiter = RateLimiter("2/m") + limiter.record() + limiter.record() + assert limiter.is_limited() + + restored = RateLimiter("2/m") + restored.restore_state(limiter.export_state()) + + # Without this the operator would get a fresh quota on every config edit. + assert restored.is_limited() + assert len(restored.request_timestamps) == 2 + + +def test_quarantined_key_stays_quarantined_across_a_restart(): + limiter = RateLimiter("0") + limiter.block_for(300) + + restored = RateLimiter("0") + restored.restore_state(limiter.export_state()) + + assert restored.blocked_until > time.time() + assert restored.is_limited() + + +def test_expired_cooldown_is_not_restored(): + limiter = RateLimiter("0") + limiter.blocked_until = time.time() - 5 + assert limiter.export_state() is None + + restored = RateLimiter("0") + restored.restore_state({"blocked_until": time.time() - 5}) + assert restored.blocked_until == 0.0 + + +def test_timestamps_outside_the_window_are_dropped_on_restore(): + old = time.time() - 3600 + restored = RateLimiter("5/m") + restored.restore_state({"timestamps": [old, old, time.time()]}) + + assert len(restored.request_timestamps) == 1 + + +def test_future_timestamps_are_rejected(): + """A backwards clock change or edited state file must not wedge a limiter.""" + restored = RateLimiter("1/m") + restored.restore_state({"timestamps": [time.time() + 10_000]}) + + assert len(restored.request_timestamps) == 0 + assert not restored.is_limited() + + +def test_malformed_entries_are_ignored(): + restored = RateLimiter("5/m") + restored.restore_state( + {"timestamps": ["nope", None, time.time()], "blocked_until": "x"} + ) + + assert len(restored.request_timestamps) == 1 + assert restored.blocked_until == 0.0 + + +def test_concurrency_lock_is_not_restored(): + """`locked` tracks in-flight requests of a process that is going away.""" + limiter = RateLimiter("5/m") + limiter.record() + limiter.lock() + + restored = RateLimiter("5/m") + restored.restore_state(limiter.export_state()) + + assert restored.locked is False + + +# --- TrafficManager export/import -------------------------------------------- + + +def test_persisted_state_contains_no_credentials(): + """Limiter names embed the upstream key, so the file must not hold them.""" + control = TrafficManager(StubConfig()) + control.get_key_limiter("openai", "sk-super-secret").record() + + exported = control.export_state() + blob = json.dumps(exported) + + assert "sk-super-secret" not in blob + assert state_key("openai_key_sk-super-secret") in exported["rate_limiters"] + + +def test_limiter_state_is_restored_on_next_use(): + before = TrafficManager(StubConfig(key_rate="2/m")) + before.get_key_limiter("openai", "sk-a").record() + before.get_key_limiter("openai", "sk-a").record() + assert before.get_key_limiter("openai", "sk-a").is_limited() + + after = TrafficManager(StubConfig(key_rate="2/m")) + after.import_state(before.export_state()) + + assert after.get_key_limiter("openai", "sk-a").is_limited() + + +def test_restart_applies_an_edited_rate_limit_to_restored_state(): + """ + The whole reason for restarting is to pick up config, so restored state + must not drag the old limit along with it. + """ + before = TrafficManager(StubConfig(key_rate="2/m")) + before.get_key_limiter("openai", "sk-a").record() + before.get_key_limiter("openai", "sk-a").record() + + # Operator raises the limit while the process is down. + after = TrafficManager(StubConfig(key_rate="10/m")) + after.import_state(before.export_state()) + limiter = after.get_key_limiter("openai", "sk-a") + + assert limiter.rate_limit == "10/m" + assert not limiter.is_limited() # 2 of 10 consumed + assert len(limiter.request_timestamps) == 2 + + +def test_quarantine_survives_through_the_traffic_manager(): + before = TrafficManager(StubConfig()) + before.block_key("openai", "sk-bad", 300) + + after = TrafficManager(StubConfig()) + after.import_state(before.export_state()) + + assert after.get_key_limiter("openai", "sk-bad").blocked_until > time.time() + + +def test_import_tolerates_junk(): + control = TrafficManager(StubConfig()) + assert control.import_state({}) == 0 + assert control.import_state({"rate_limiters": None}) == 0 + assert control.import_state({"rate_limiters": "nope"}) == 0 + assert control.import_state({"rate_limiters": {"a": "not-a-dict"}}) == 0 + + +def test_unused_restored_entries_do_not_leak_into_other_limiters(): + before = TrafficManager(StubConfig()) + before.get_key_limiter("openai", "sk-a").record() + + after = TrafficManager(StubConfig()) + after.import_state(before.export_state()) + + # A different key must not inherit the first key's consumed window. + other = after.get_key_limiter("openai", "sk-b") + assert len(other.request_timestamps) == 0 + + +# --- state file -------------------------------------------------------------- + + +def test_save_and_load_round_trip(tmp_path): + path = tmp_path / "state.json" + assert save_state(path, {"rate_limiters": {"a": {"timestamps": [1.0]}}}) is True + + assert load_state(path) == {"rate_limiters": {"a": {"timestamps": [1.0]}}} + + +def test_state_file_is_not_world_readable(tmp_path): + """Even hashed, this is operational data about credential usage.""" + path = tmp_path / "state.json" + save_state(path, {"rate_limiters": {}}) + + mode = stat.S_IMODE(os.stat(path).st_mode) + assert mode == 0o600 + + +def test_missing_file_is_not_an_error(tmp_path): + assert load_state(tmp_path / "absent.json") == {} + + +def test_corrupt_file_does_not_prevent_startup(tmp_path): + path = tmp_path / "state.json" + path.write_text("{not json") + + assert load_state(path) == {} + + +def test_version_mismatch_is_ignored(tmp_path): + path = tmp_path / "state.json" + path.write_text( + json.dumps({"version": STATE_VERSION + 1, "rate_limiters": {"a": {}}}) + ) + + assert load_state(path) == {} + + +def test_non_object_document_is_ignored(tmp_path): + path = tmp_path / "state.json" + path.write_text(json.dumps([1, 2, 3])) + + assert load_state(path) == {} + + +def test_save_leaves_no_temporary_files_behind(tmp_path): + path = tmp_path / "state.json" + save_state(path, {"rate_limiters": {}}) + + assert [p.name for p in tmp_path.iterdir()] == ["state.json"] + + +def test_save_failure_is_reported_not_raised(tmp_path): + # A directory where the file should be makes the atomic rename fail. + path = tmp_path / "state.json" + path.mkdir() + + assert save_state(path, {"rate_limiters": {}}) is False + + +def test_state_file_lives_beside_the_config(tmp_path): + """A bind-mounted config directory then carries the state with it.""" + config = tmp_path / "sub" / "config.yaml" + config.parent.mkdir() + config.write_text("{}") + + assert resolve_state_path(str(config)) == config.parent / ".nya_state.json" + + +def test_state_path_falls_back_to_cwd_without_a_config_path(): + assert resolve_state_path(None).name == ".nya_state.json" + + +# --- app wiring -------------------------------------------------------------- + + +def _app_with_control(config_path, control): + app = object.__new__(server_app.NyaProxyApp) + app.config = SimpleNamespace(config_path=str(config_path)) + app.core = SimpleNamespace(control=control) + return app + + +def test_app_persists_and_restores_limiter_state(tmp_path): + config_path = tmp_path / "config.yaml" + config_path.write_text("{}") + + control = TrafficManager(StubConfig(key_rate="2/m")) + control.get_key_limiter("openai", "sk-a").record() + control.get_key_limiter("openai", "sk-a").record() + + assert _app_with_control(config_path, control).persist_runtime_state() is True + assert (tmp_path / ".nya_state.json").exists() + + fresh = TrafficManager(StubConfig(key_rate="2/m")) + assert _app_with_control(config_path, fresh).restore_runtime_state() == 1 + assert fresh.get_key_limiter("openai", "sk-a").is_limited() + + +def test_app_state_path_prefers_the_configured_path_over_the_environment( + tmp_path, monkeypatch +): + """ + The path can arrive as a constructor argument; falling back to the working + directory would make two instances share one another's quotas. + """ + monkeypatch.setenv("CONFIG_PATH", str(tmp_path / "from-env" / "config.yaml")) + config_path = tmp_path / "from-arg" / "config.yaml" + config_path.parent.mkdir() + + app = _app_with_control(config_path, TrafficManager(StubConfig())) + + assert app._state_path() == config_path.parent / ".nya_state.json" + + +def test_restore_is_a_no_op_without_a_core(tmp_path): + app = object.__new__(server_app.NyaProxyApp) + app.config = SimpleNamespace(config_path=str(tmp_path / "config.yaml")) + app.core = None + + assert app.restore_runtime_state() == 0 + assert app.persist_runtime_state() is False + + +def test_unreadable_state_does_not_break_startup(tmp_path): + config_path = tmp_path / "config.yaml" + (tmp_path / ".nya_state.json").write_text("{corrupt") + + app = _app_with_control(config_path, TrafficManager(StubConfig())) + + assert app.restore_runtime_state() == 0 From 15e37cfe07de45154b197a3c2fe13d430bc14b4e Mon Sep 17 00:00:00 2001 From: k3scat Date: Sat, 18 Jul 2026 15:10:29 -0400 Subject: [PATCH 3/4] docs: document the data directory mount for Docker The documented `docker run` reproduced every fault that made config edits fail in containers: it bind-mounted config.yaml as a single file, mounted it read-only, and passed --no-reload. Mount the directory instead. Saving from the /config UI writes a temporary file and renames it over the target, which fails with EBUSY against a bind-mounted file, so every save returned a 500 before the read-only flag was even reached. Add --user, because the image runs as uid 100 and cannot otherwise write a directory the operator owns, and note that Docker Desktop maps ownership itself. Drop --no-reload, which stranded any edit that did manage to save. Also describe what the directory now holds: config.yaml plus .nya_state.json, which carries rate-limit windows and key cool-downs across the restart that applies a configuration change. Applied to the Chinese and Japanese READMEs as well, since all three carried the same command. Verified by running the documented command verbatim against a build of this tree: the container starts, a UI save returns 200 and persists to ./data on the host, the reload fires ("WatchFiles detected changes in 'watch.txt'"), .nya_state.json is written 0600 beside the config, and the service stays healthy afterwards. --- README.md | 15 +++++++++++---- README_ja.md | 14 ++++++++++++-- README_zh.md | 14 ++++++++++++-- 3 files changed, 35 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 980b7dc..f367113 100644 --- a/README.md +++ b/README.md @@ -88,14 +88,21 @@ nyaproxy # or: python -m nya ### Docker ```bash -cp configs/openai.yaml config.yaml # then replace every placeholder key +mkdir -p data +cp configs/openai.yaml data/config.yaml # then replace every placeholder key + docker run -d \ -p 8080:8080 \ - -v ${PWD}/config.yaml:/app/config.yaml:ro \ - k3scat/nya-proxy:latest --config config.yaml --host 0.0.0.0 --no-reload + -v ${PWD}/data:/app \ + --user "$(id -u):$(id -g)" \ + k3scat/nya-proxy:latest --config /app/config.yaml --host 0.0.0.0 ``` -The bind-mounted file is the persistent source of truth. Edit it on the host and restart the container. Do not publish the port until `server.api_key` is set. +Mount the **directory**, not `config.yaml` itself. Saving from the `/config` UI writes a temporary file and renames it over the target, and that rename fails with `EBUSY` against a bind-mounted file, so every save returns a 500. The mount must also be writable — a read-only mount blocks saves for the same reason. + +`--user` runs the container as you, since the image runs as uid 100 and could not otherwise write a directory you own. Docker Desktop on macOS and Windows maps ownership for you, so you can drop that flag there. + +The directory holds `config.yaml` and `.nya_state.json`, which carries rate-limit windows and key cool-downs across restarts so a config change does not reset your quotas. Edit configuration on the host or through the `/config` UI; either way NyaProxy restarts itself to apply it. Do not publish the port until `server.api_key` is set. ### CLI Reference diff --git a/README_ja.md b/README_ja.md index 6b9939c..011236b 100644 --- a/README_ja.md +++ b/README_ja.md @@ -93,12 +93,22 @@ nyaproxy ### Docker ```bash +mkdir -p data +cp configs/openai.yaml data/config.yaml # プレースホルダーのキーをすべて置き換えてください + docker run -d \ -p 8080:8080 \ - -v ${PWD}/config.yaml:/app/config.yaml:ro \ - k3scat/nya-proxy:latest --config config.yaml --host 0.0.0.0 --no-reload + -v ${PWD}/data:/app \ + --user "$(id -u):$(id -g)" \ + k3scat/nya-proxy:latest --config /app/config.yaml --host 0.0.0.0 ``` +`config.yaml` そのものではなく、**ディレクトリ**をマウントしてください。`/config` UI は一時ファイルを書き出してから対象ファイルへリネームして保存しますが、ファイル単位でバインドマウントされたパスではこのリネームが `EBUSY` で失敗し、保存のたびに 500 が返ります。同じ理由で、読み取り専用マウントでも保存できません。 + +イメージは uid 100 で動作するため、そのままでは自分が所有するディレクトリに書き込めません。`--user` を指定してコンテナを現在のユーザーとして実行します。macOS と Windows の Docker Desktop は所有者を自動的にマッピングするため、このフラグは省略できます。 + +このディレクトリには `config.yaml` と `.nya_state.json` が置かれます。後者はレート制限のウィンドウとキーのクールダウンを再起動をまたいで保持するため、設定変更でクォータがリセットされることはありません。設定はホスト側でも `/config` UI からでも編集でき、いずれの場合も NyaProxy が自動的に再起動して反映します。`server.api_key` を設定するまでは、ポートを外部に公開しないでください。 + ## 設定 NyaProxy は YAML で設定します。追加例は [configs](configs/) を参照してください。 diff --git a/README_zh.md b/README_zh.md index af50d34..0b0afc1 100644 --- a/README_zh.md +++ b/README_zh.md @@ -93,12 +93,22 @@ nyaproxy ### Docker ```bash +mkdir -p data +cp configs/openai.yaml data/config.yaml # 然后替换其中的占位密钥 + docker run -d \ -p 8080:8080 \ - -v ${PWD}/config.yaml:/app/config.yaml:ro \ - k3scat/nya-proxy:latest --config config.yaml --host 0.0.0.0 --no-reload + -v ${PWD}/data:/app \ + --user "$(id -u):$(id -g)" \ + k3scat/nya-proxy:latest --config /app/config.yaml --host 0.0.0.0 ``` +请挂载**目录**，而不是 `config.yaml` 文件本身。`/config` 界面保存配置时，会先写入临时文件再重命名覆盖目标文件；对于以文件方式绑定挂载的路径，该重命名会因 `EBUSY` 失败，导致每次保存都返回 500。挂载还必须可写，只读挂载会因同样的原因导致保存失败。 + +镜像以 uid 100 运行，无法写入属于你的目录，因此使用 `--user` 让容器以当前用户身份运行。macOS 与 Windows 上的 Docker Desktop 会自动映射属主，可以省略该参数。 + +该目录存放 `config.yaml` 与 `.nya_state.json`，后者在重启后保留限流窗口与密钥冷却状态，因此修改配置不会重置配额。你可以在宿主机上编辑配置，也可以通过 `/config` 界面修改，NyaProxy 都会自动重启以应用变更。在设置 `server.api_key` 之前，请勿对外暴露端口。 + ## 配置 NyaProxy 使用 YAML 配置。更多示例见 [configs](configs/)。 From 983c6bc87bf9d098a13e3c5be1e4a33f6f458b5d Mon Sep 17 00:00:00 2001 From: k3scat Date: Sat, 18 Jul 2026 15:11:16 -0400 Subject: [PATCH 4/4] dep bump --- uv.lock | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/uv.lock b/uv.lock index 49b2533..93a1efd 100644 --- a/uv.lock +++ b/uv.lock @@ -586,7 +586,7 @@ server = [ [[package]] name = "nya-proxy" -version = "0.4.6" +version = "0.6.0" source = { editable = "." } dependencies = [ { name = "fastapi" },