feat(validator): load repo hyperparameters from the das API#1511
Draft
anderdc wants to merge 15 commits into
Draft
feat(validator): load repo hyperparameters from the das API#1511anderdc wants to merge 15 commits into
anderdc wants to merge 15 commits into
Conversation
Flip load_master_repo_weights() to fetch the repository registry from the das-gittensor API (GET api.gittensor.io/repos), the new source of truth for repository hyperparameters (maintainer/admin edits land there). - Extract parse+validate into _parse_registry(); add _fetch_registry_from_api() (requests + retry/backoff) and _load_registry_from_file(). - API-first with a bundled master_repositories.json fallback: a transient API outage or a contract-violating push falls back to the seed instead of bricking scoring. A broken bundled seed still raises. - Tests default to the seed via an autouse fixture (offline/deterministic); added explicit API-path + fallback tests. master_repositories.json is retained only as the fallback seed.
…os (#1523) Co-authored-by: anderdc <me@alexanderdc.com>
Co-authored-by: anderdc <me@alexanderdc.com>
…_pipeline, drop community-contribution (#1512)
…intainer_cut 0.30 (#1533) Co-authored-by: Ander <61125407+anderdc@users.noreply.github.com>
…1530) Co-authored-by: Ander <61125407+anderdc@users.noreply.github.com>
… in blend_emission_pools (#1534) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ander <61125407+anderdc@users.noreply.github.com>
Co-authored-by: e35ventura <e35ventura@gmail.com>
…helper (#1541) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ander <61125407+anderdc@users.noreply.github.com>
… cache Layers the disk-cache resilience on top of anderdc's API-fetch loader and removes the bundled registry entirely (das DB is now the sole source of truth). - load_master_repo_weights(): on a successful GET /repos, write the registry to an on-disk last-good cache; on API failure read that cache; with no cache, return an empty registry (logged loudly) — per-field knob defaults still apply. - _write_registry_to_cache() writes atomically (tmp + replace) and is best-effort (a cache-write failure never breaks the scoring cycle); _load_registry_from_cache() + _get_repos_cache_path() (env GITTENSOR_REPOS_CACHE_PATH, ~/.gittensor default). - constants: add REPOS_CACHE_PATH; refresh stale master_repositories.json comments. - DELETE gittensor/validator/weights/master_repositories.json and the _load_registry_from_file seed path (languages/token JSON untouched). - tests: conftest autouse fixture now points the cache at an empty per-test tmp (default load => empty); injection tests warm the cache instead of the bundled file; obsolete live-registry-content tests removed; new disk-cache tests added (cache-write on success, fallback-to-cache, empty-when-no-cache, invalid-cache). MERGE GATE: do not ship before the gt-utils seed migration has populated the prod DB so GET /repos is non-empty (critical sequencing risk in the spec).
Collaborator
|
Layered on (commit 738985a): replaced bundled-JSON fallback with on-disk last-good cache (REPOS_CACHE_PATH; atomic write on success → read on failure → empty+log), DELETED master_repositories.json + file-seed path, refactored tests/conftest. 709 validator+cli tests green. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Flips the validator's source of truth for repository hyperparameters from the bundled
master_repositories.jsonto the das-gittensor API. Part of the coordinated hyperparameter-API migration:GET /reposregistryrepository_config_editsaudit/cooldown tablefeat/hyperparam-api) — stop clobbering the API-authoritativerepositoriestableChange
load_master_repo_weights()now:GET https://api.gittensor.io/repos(requests + retry/backoff), parses + validates.master_repositories.jsonseed — a transient outage or a bad push can't brick scoring.{}only if both the API and the seed are unavailable. A broken seed still raises (real bug).Parsing/validation was extracted into
_parse_registry()and is shared by both paths, so the contract is identical regardless of source. Return type (Dict[str, RepositoryConfig]) is unchanged — no call site or downstream consumer moves.master_repositories.jsonis retained only as the fallback seed (regenerate from the DB via gt-utilsfetch_master_data.py).Tests
ruff check/ruff formatclean;pyright0 errors; CLIminer scoretests pass.Notes for review
GET /reposis@NoCacheon the API so edits propagate immediately. Consider signing/versioning the payload later (a central live source is weaker than git-pinned commits) — fallback seed mitigates availability, not authenticity.