4.9.0 groundwork: correctness fixes, dead code removal, CI - #407
Merged
Conversation
…ent plan - *.env / .env.* catch-alls with template negations (openrouter.env was previously unignored and holds a live key) - ignore emacs autosave/lock files (#*#, .#*, *~) - dedupe repeated *.onnx / downloaded_models entries - IMPROVEMENT_PLAN.md: full audit findings and phased refactor plan
…, broken mask path
- media_store.get_unuploaded_media: bind modality/source_type/limit as
parameters instead of f-string interpolation (count_unuploaded_media
was already parameterized; the two paths now match)
- media_store.cleanup_uploaded_media: remove DELETE FROM prompts keyed
by media UUIDs (wrong table); the correct prompt_ids delete below is
kept
- connection.connect: retry loop now only covers opening the connection,
and close() runs in a finally so exceptions in the caller's block no
longer leak the connection (previously a body exception also re-entered
the yield and could raise RuntimeError)
- media_storage.retrieve_media: Path.with_suffix('_mask.npy') raises
ValueError (no leading dot), swallowed by the outer except, so
require_mask retrieval always returned nothing; build the mask path
from stem to match the write/delete paths
…, epistula hardening - webhooks: guard _stats mutation with a dedicated RLock (previously per-request threads incremented counters unlocked; _file_lock only covered file writes); _do_save snapshots under the stats lock and writes under the file lock - webhooks: retry backoff was retry_delay ** attempt (1s, 2s, 4s with the default 2.0 regardless of configured delay); now retry_delay * 2**attempt - gas_api_validator: printf-style %s args were passed to loguru-style bt.logging and never interpolated — errors/status codes were missing from logs; converted to f-strings - epistula.verify_signature: reject future-dated timestamps (replay window was one-sided), return 'Invalid Timestamp' instead of raising on non-numeric input, drop the dead isinstance check
gas.scraping had zero importers anywhere in the codebase (CLI, data service, tests included) — the Selenium/Google-Images scraper is no longer part of any data path. Removed: - gas/scraping/ (base.py, google.py, __init__.py, ~900 lines) - selenium + stamina from pyproject (and transitive trio/wsproto/ tenacity etc. from uv.lock) - Google Chrome apt-repo install blocks from install.sh and Dockerfile - Chrome/xvfb/libnss3 runtime packages from the Docker runtime stage - browser-automation references from docs/Installation.md source_type='scraper' remains in the DB schema/queries since historical rows still carry it.
…e guard, verification batch size
- push_model: on-chain model hash used builtin hash(), which is
process-salted (PYTHONHASHSEED) and differs every run; use the sha256
hexdigest prefix (DiscriminatorModelId truncates to 16 chars anyway)
- on_block_interval: missing return after the None-interval error log
meant execution fell through to block % None -> TypeError
- generation_pipeline: f-string prefix missing on i2i error (emitted
literal '{model_name}'); drop unreachable return after raise
- verification_pipeline: verify_media's batch_size parameter was
accepted and ignored (hardcoded 128/32 inside), so callers passing
clip_batch_size had no effect; also hoist Path import out of loop
…mportable gas/generation/__init__.py eagerly imported the full diffusers/torch/ janus pipeline, so importing ANY submodule (including pure-stdlib prompt code) required the GPU stack. This broke 4 of 8 test files at collection time on machines without ML deps. - __init__.py: re-exports now resolve lazily via PEP 562 __getattr__ (flat import paths like 'from gas.generation import PromptGenerator' still work) - prompt_generator.py: torch/transformers imported inside load_vlm/ load_llm; cuda cache flushes go through a torch-optional helper - scene.py: torch imported inside extract_scene_with_vlm Full test suite now collects and passes: 37 passed (was 10 collectable).
Two lightweight jobs: - lint: ruff at error level only (E9/F63/F7/F82) so it passes today; widen the rule set as the cleanup phases land - test: minimal venv (pytest, bittensor, pillow, numpy — mirrors pyproject pins) instead of uv sync, since the full lock pulls CUDA torch that the tests don't need; all 37 tests run in ~10s First CI this repo has had.
Local runs used 'python -m pytest' which puts the repo root on sys.path; CI invokes the pytest binary, which doesn't, so the gas package was never importable. Editable no-deps install fixes it without pulling the GPU dependency tree.
Unpinned resolution pulled a newer release that vendors cyscale, which conflicts with the scalecodec package bittensor 9.9.0 still imports. 1.6.0 matches uv.lock.
… pyproject - pytest moves into a locked [dependency-groups] dev entry; CI now runs 'uv sync --frozen' so every version comes from uv.lock (the hardcoded workflow pins drifted from the lock on day one — the async-substrate-interface/cyscale conflict). GPU packages are skipped by name; the tests don't import torch. - [tool.pytest.ini_options] with testpaths/pythonpath so the bare pytest binary works from a checkout without editable-install tricks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First batch of cleanup work: targeted correctness fixes, removal of an unused package, and CI. No feature changes.
Fixes
get_unuploaded_mediaquery filters; remove an incorrect cross-table DELETE incleanup_uploaded_media; guarantee connection close on exception viafinally; correct the mask-file path construction inretrieve_media(previously raised and was swallowed)push_model(was using process-salted builtinhash()); missingreturninon_block_intervalwhen the interval attr is unset;verify_medianow honors itsbatch_sizeparameter instead of hardcoded values; f-string and unreachable-code fixes in the generation pipelineRemovals
gas/scraping/had no remaining callers — removed along with theselenium/staminadependencies and the Chrome install steps ininstall.sh/Dockerfile(~925 lines)Test/CI
gas/generationre-exports are now lazy (PEP 562), so pure prompt modules import without the GPU stack — the full test suite (37 tests) now collects and passes on CPU-only machinesNotes for operators
clip_batch_size(512 in the generator service) rather than hardcoded 128/32 — watch VRAM on the first verification cycle after deployTest plan
pytest tests/— 37 passedruff check --select E9,F63,F7,F82— clean🤖 Generated with Claude Code