agents.md

Freeshard App Repository - Agent Guide

This document describes how to create, configure, and modify apps in the Freeshard app store. Each app lives in apps/<app-name>/ and consists of a few key files.

Repository Structure

apps/
  <app-name>/
    app_meta.json                  # App metadata and configuration
    docker-compose.yml.template    # Docker Compose template with Jinja-like variables
    icon.svg (or .png)             # App icon
    <app-name>.zip                 # Built artifact (auto-generated, do not edit)
    .env                           # Optional, for complex multi-service apps
inactive_apps/
  template/                        # Skeleton for new apps
blocked_apps/
  README.md                        # Schema + procedure
  <app-name>.md                    # One file per analysed-and-blocked app
justfile                           # `just new-app <name>` scaffolds a new app
build_store_data.py                # Builds store_metadata.json and zip files
update.py                          # Checks GitHub releases for version updates

blocked_apps/ documents candidates that were researched but did not pass the inclusion criteria (non-FOSS license, no Docker image, paid tier, etc.). See blocked_apps/README.md for the file schema. Written automatically by the /add-app skill on every hard exit.

Creating a New App

1. Run just new-app <name> to scaffold from the template.
2. Edit apps/<name>/app_meta.json - fill in all $$edit$$ placeholders.
3. Edit apps/<name>/docker-compose.yml.template - set the correct image and configuration.
4. Add an icon file (icon.svg preferred, or icon.png).
5. Run python -m build_store_data to generate the zip and update store metadata.

The app name must be lowercase, using only letters, numbers, and dashes. It becomes the subdomain: <name>.<shard-domain>.

app_meta.json Reference

{
  "v": "1.2",                           // Format version. Use "1.2" for new apps (supports homepage/upstream_repo). "1.1" and "1.0" also exist.
  "app_version": "1.0.0",               // App version string. Must match the docker image tag used in docker-compose.yml.template.
  "name": "my-app",                     // Unique identifier. Lowercase, letters/numbers/dashes only. Must match folder name.
  "pretty_name": "My App",              // Optional. Display name with proper casing. Defaults to titlecased name.
  "icon": "icon.svg",                   // Icon filename. Must exist in the app folder.
  "homepage": "https://example.com",    // Optional (v1.2+). App's homepage URL.
  "upstream_repo": "https://github.com/org/repo",  // Optional (v1.2+). GitHub repo for automatic update checking.

  "entrypoints": [                      // Required. At least one entrypoint.
    {
      "container_name": "my-app",       // Must match a container_name in docker-compose.yml.template.
      "container_port": 8080,           // The port the container listens on internally.
      "entrypoint_port": "http"         // "http" (maps to 443) or "mqtt" (maps to 8883).
    }
  ],

  "paths": {                            // Required. Access control rules by path prefix.
    "": {                               // Default/catch-all rule (empty string = all paths). Required.
      "access": "private",              // "private" (paired devices only), "public" (anyone), or "peer" (peer shards).
      "headers": {                      // Optional. Headers forwarded to the app.
        "X-Ptl-Client-Id": "{{ auth.client_id }}",
        "X-Ptl-Client-Name": "{{ auth.client_name }}",
        "X-Ptl-Client-Type": "{{ auth.client_type }}",
        "X-Ptl-User": "admin"           // Common pattern for auth-proxy: send a static username.
      }
    },
    "/public/": {                       // More specific prefixes take priority (longest match wins).
      "access": "public",
      "headers": {
        "X-Ptl-Client-Type": "{{ auth.client_type }}"
      }
    }
  },

  "lifecycle": {                        // Optional. Controls start/stop behavior.
    "always_on": false,                 // true = never auto-stop. Cannot be used with idle_time_for_shutdown.
    "idle_time_for_shutdown": 60        // Seconds of no HTTP traffic before auto-stop. Default: 60. Use higher values for apps with background tasks.
  },

  "minimum_portal_size": "s",           // Optional. Minimum shard size required. Omit for lightweight apps (defaults to "xs").

  "store_info": {                       // Required. App store display information.
    "description_short": "One-line description of the app.",
    "description_long": [               // Optional. Array of paragraphs (strings) or a single string.
      "First paragraph.",
      "Second paragraph."
    ],
    "hint": [                           // Optional. Array of usage hints shown to the user.
      "Tip: configure X before first use."
    ],
    "is_featured": false                // Optional. Whether to highlight in the app store.
  }
}

Access Control Patterns

Most apps use one of these approaches:

Fully private (most common for single-user apps):

"paths": { "": { "access": "private" } }

Private with auth-proxy headers (for apps that support reverse proxy auth):

"paths": {
  "": {
    "access": "private",
    "headers": { "X-Ptl-User": "admin" }
  }
}

The app must be configured to trust the proxy header (see linkding, navidrome examples).

Public with app-managed auth (for apps with built-in user management):

"paths": { "": { "access": "public" } }

Used when the app handles its own authentication (e.g., immich, affine).

Mixed access (private by default, some paths public):

"paths": {
  "": { "access": "private" },
  "/share/": { "access": "public" },
  "/api/public/": { "access": "public" }
}

Header Template Variables

Available in paths[].headers values:

• {{ auth.client_type }} - "terminal", "peer", or "anonymous"
• {{ auth.client_id }} - Cryptographic client identifier
• {{ auth.client_name }} - User-assigned client name

Lifecycle Guidelines

• Simple web apps: idle_time_for_shutdown: 60 (default)
• Apps with background processing: idle_time_for_shutdown: 300 to 3600
• IoT/messaging services (mosquitto, node-red): always_on: true
• Apps that take a long time to start: higher idle timeout to avoid frequent restarts

docker-compose.yml.template Reference

Templates use Jinja-like {{ variable }} syntax. Variables are replaced at installation time.

Template Variables

Variable	Description	Example Value
{{ portal.domain }}	Shard's fully qualified domain	8271dd.example.com
{{ portal.id }}	Full shard hash-ID	8271dd...
{{ portal.short_id }}	First 6 chars of shard ID	8271dd
{{ portal.public_key_pem }}	Shard's public key (PEM)	-----BEGIN PUBLIC KEY-----...
{{ fs.app_data }}	App-specific persistent storage path	/data/apps/my-app
{{ fs.all_app_data }}	Parent directory of all app data	/data/apps
{{ fs.shared }}	Shared directory for inter-app data	/data/shared

Minimal Template (simple single-container app)

networks:
    portal:
        external: true

services:
    my-app:
        restart: always
        image: org/my-app:1.0.0
        container_name: my-app
        volumes:
        - "{{ fs.app_data }}/data:/data"
        environment:
        - BASE_URL=https://my-app.{{ portal.domain }}
        networks:
        - portal

Template with Database (e.g., PostgreSQL + Redis)

networks:
    portal:
        external: true
    my-app:

services:
    my-app:
        restart: always
        image: org/my-app:1.0.0
        container_name: my-app
        volumes:
        - "{{ fs.app_data }}/data:/app/data"
        environment:
        - DATABASE_URL=postgres://myapp:myapp@my-app-postgres:5432/myapp
        - REDIS_URL=redis://my-app-redis:6379
        - BASE_URL=https://my-app.{{ portal.domain }}
        depends_on:
        - my-app-postgres
        - my-app-redis
        networks:
        - portal
        - my-app

    my-app-postgres:
        restart: always
        image: postgres:16
        container_name: my-app-postgres
        volumes:
        - "{{ fs.app_data }}/pgdata:/var/lib/postgresql/data"
        environment:
        - POSTGRES_USER=myapp
        - POSTGRES_PASSWORD=myapp
        - POSTGRES_DB=myapp
        networks:
        - my-app

    my-app-redis:
        restart: always
        image: redis:7-alpine
        container_name: my-app-redis
        networks:
        - my-app

Key Rules

1. Portal network: Every template must declare the portal external network. All containers that need to be reachable (by the reverse proxy or by other apps) must join it.
2. Container names: Every service must have an explicit container_name. The main service's container_name must match the entrypoints[].container_name in app_meta.json.
3. Supporting services: Name them <app-name>-<service> (e.g., paperless-redis, affine-postgres).
4. Volumes: Use {{ fs.app_data }}/... for app-specific persistent data. Use {{ fs.shared }}/... for shared user data (documents, music, pictures, media).
5. restart: Always set to always (or unless-stopped). Use restart: no only for one-shot init/migration containers.
6. Image tags: Pin to a specific version matching app_version in app_meta.json. Do not use latest.
7. Internal networking: Supporting services (databases, caches) can reference each other by container_name since they're on the same network.
8. Docker socket: Can be mounted read-only if needed: /var/run/docker.sock:/var/run/docker.sock:ro (used by dozzle).
9. Private networks: For multi-service apps, only the entrypoint container should join the portal network. Create an additional private network (named after the app) for internal communication between all services. The entrypoint container joins both networks; supporting services join only the private network. See immich for examples.

Common Shared Directory Paths

• {{ fs.shared }}/documents - Documents (used by paperless-ngx)
• {{ fs.shared }}/music - Music files (used by navidrome)
• {{ fs.shared }}/pictures - Photos (used by immich, photoprism)
• {{ fs.shared }}/media - General media

Environment Variable Patterns

• Base URL: BASE_URL=https://<name>.{{ portal.domain }}
• Disable telemetry: Most apps have a telemetry opt-out env var - always disable it.
• Auto-login / auth-proxy: When using private access with header-based auth, configure the app to trust the proxy and auto-login. Examples:
  • linkding: LD_ENABLE_AUTH_PROXY=True, LD_AUTH_PROXY_USERNAME_HEADER=HTTP_X_PTL_USER
  • navidrome: ND_REVERSEPROXYUSERHEADER=X-Ptl-User, ND_REVERSEPROXYWHITELIST=0.0.0.0/0
  • paperless-ngx: PAPERLESS_AUTO_LOGIN_USERNAME=admin

Version Updates

Updates run via the /update-apps skill (.claude/skills/update-apps/). The skill orchestrates update/update.py and reasons over breaking-change candidates.

Flow on a single run:

1. python3 update/update.py check --json polls every app's apps/<name>/update_check.py in parallel, writes update/update_info/latest_check.json.
2. Skill classifies each outdated app: clean patch/minor without "breaking" notes and without non-trivial upstream-compose changes → AUTO. Anything else → candidate; the skill judges per app and picks AUTO or REVIEW.
3. python3 update/update.py apply <app> <ver> --auto|--review --branch-ts <ts> rewrites version strings, runs docker compose pull --dry-run, commits onto updates/<iso-ts> with message update <app> from <old> to <new> [AUTO|REVIEW].
4. Skill opens a PR; GH preview job builds updated_apps.zip, uploads to app-store/updates/<iso-ts>/updated_apps.zip, comments the URL.
5. User downloads bundle, smoke-installs on a fresh shard, merges PR.

Smoke-testing a bundle

update/smoke_test.py <bundle> runs step 5 unattended against a throwaway shard:

uv run update/smoke_test.py https://storageaccountportab0da.blob.core.windows.net/app-store/updates/<ts>/updated_apps.zip

It is the one script under update/ with a dependency (httpx), declared in a PEP 723 header, so uv run installs it on the fly and the repo still needs no manifest.

Shards it assigns are stamped with the owner email clayde@vtettenborn.net, which makes test shards obvious in the controller UI. It has to be a routable address. The core validates the owner email the controller hands it, and an unroutable one is not merely ignored: on core 0.40.5 it bricked the shard. enrich_identity_from_profile fires on the first pairing, writes the address into the identity row and only then validates it, so every later read of that row raises and the shard can never be paired. A reserved TLD such as .invalid (RFC 2606) is rejected by exactly that check. See diagnostic 8d52dbf2-6a5b-4ba1-818f-e6053943947c and FreeshardBase/freeshard-controller#229.

Every run saves its terminal JWT to update/smoke_test_session.json (gitignored, mode 600), so the shard stays reachable after the run finishes — for poking at an app that failed, reading logs, or installing something by hand:

uv run update/smoke_test.py <bundle> --domain abc123.freeshard.cloud

The token is what pairing produced and is valid for ten years, but the shard itself is gone 24h after assignment, so the file goes stale quickly and holds nothing of value once it has.

Without a saved session (a different machine, a shard someone else created), attach with a pairing code instead:

uv run update/smoke_test.py <bundle> --domain abc123.freeshard.cloud --pairing-code <code>

Pairing codes are single-use; issue a fresh one per attach from the controller (GET /api/shards/<db_id>/pairing_code, needs the SUPPORT_SHARD permission).

It assigns a trial shard (POST /api/shards/assign_trial on the controller — no auth, returns domain plus a single-use pairing code), pairs as a terminal, removes the apps a fresh shard ships with, then per app installs the zip, waits for the install to finish and requests https://<app>.<domain>/ until it answers. It logs progress and per-phase durations, prints a table, and exits non-zero if any app failed. The shard's hash-id is printed so a new pairing code can be issued from the controller to inspect it by hand; the shard deletes itself 24h after assignment.

Deliberately not a CI job: a run takes tens of minutes and consumes a standby shard.

A 200 is not automatically a pass. Traefik's dynamic config carries a catch-all PathPrefix("/") router for the web terminal with no host constraint, so any subdomain without a router of its own answers 200 with the terminal's page — nonexistent-app.<shard> included. The script fingerprints the terminal before installing anything and treats a body matching it as "no route yet", not success.

Three further responses while an app boots are expected and none means a broken app: the core's splash page with the upstream's 502/503, a 404 (the app has no Traefik router yet — still queued, in ERROR, or the shared dynamic config was mid-rewrite), and a connection error during a Traefik reload. The splash always carries the error status, never 200, so a 2xx can only come from the app. On a 404 the script asks the core for the app's status and only fails when it is ERROR. Keep --poll-interval below 5s: that is the shard's RECENT_ACCESS_GRACE, within which the memory-pressure tier will not demote the app being tested.

--keep-installed skips the per-app uninstall so a run doubles as a memory-pressure test. Only meaningful on a shard with apps.lifecycle.pause_enabled on; it is off by default in the core, and without it nothing reclaims memory within a run.

Per-app update_check.py

Every app folder has update_check.py defining def check(current_version: str) -> dict returning:

• latest_version (required, matches docker tag format)
• release_notes_url (optional)
• release_body (optional; release notes text used for "breaking" scan)
• upstream_compose_url (optional; raw URL of upstream docker-compose, may contain {version} placeholder)

Wire upstream_compose_url for stateful / multi-image apps (those with a bundled postgres/redis/etc.). The version bumper only string-replaces the app's own version, so a supporting image that upstream bumps independently (e.g. immich's postgres/vector-extension image) silently drifts in our frozen template. When the URL is set, the check fetches the upstream compose at the old and new versions and compose_diff flags any change beyond the app-version bump (supporting-image tag or structural change) → forces REVIEW. A stale DB image is what caused the immich pgvecto.rs outage; this is the guard against a repeat. Only wire it for upstreams that publish a version-pinned compose.

The diff has a blind spot worth knowing: it compares upstream's compose at the old and new version of the current bump, so it only ever catches drift that upstream introduces inside that range. If upstream changed a supporting image at some earlier point and we never followed, the diff is empty forever after and the drift stays invisible. Catching that needs a direct comparison of our template against upstream's compose at the version we are moving to — worth doing whenever an app's supporting image looks old (2026-08-24: immich sat on redis:6.2-alpine while upstream had long since moved to valkey:9; titra is still on mongo:5.0 against upstream's mongo:7.0).

Wired: immich, etherpad, paperless-ngx, titra. Deliberately NOT wired (and why):

• affine — uses the mutable :stable tag, check() raises NotImplementedError; no version detection, so the diff never runs.
• joplin-server — OptOut (manual updates); the diff never runs.
• overleaf — no version-pinned compose in the main repo (prod compose lives in overleaf/toolkit, not tagged to the sharelatex image version).
• photoprism — date-stamp image tags (YYMMDD) with no matching per-version git compose.

Use helpers from update/update_lib.py: latest_github_release, latest_dockerhub_tag, latest_ghcr_tag, latest_lscr_tag. For weird tag schemes, write whatever logic the app needs — the script is the escape hatch.

If an image has no resolvable tag pattern, the script may raise NotImplementedError; the orchestrator reports it as error and the run continues.

For apps whose updates must be done manually — self-built images (e.g. mosquitto), or upstreams whose Docker tags are unreliable for auto-detection (e.g. joplin-server, where Docker Hub publishes prerelease tags without flagging them) — raise update.update_lib.OptOut("<reason>") instead. The orchestrator reports them as opt_out (distinct from error) with the reason string preserved, so the choice stays discoverable in every check run.

Checklist for Adding a New App

• just new-app <name>
• Find the app's Docker image (Docker Hub, ghcr.io, etc.) and determine the correct image name and tag format
• Edit docker-compose.yml.template: set image, container_name, volumes, environment, networks
• Edit app_meta.json: set app_version, name, pretty_name, icon, entrypoints (correct port), paths, lifecycle, store_info
• Set upstream_repo if the app is on GitHub (enables auto-updates)
• Add an icon file (SVG preferred)
• If the app needs auth-proxy support, configure the appropriate environment variables
• If the app needs shared data (media, documents), mount {{ fs.shared }}/...
• If the app is resource-heavy, set minimum_portal_size: "s"
• If the app needs to run continuously (IoT, messaging), set always_on: true
• Disable telemetry/analytics via environment variables if the app supports it
• Run python -m build_store_data to generate the zip

Checklist for Modifying an Existing App

• Read the current app_meta.json and docker-compose.yml.template
• Make changes
• Ensure app_version in app_meta.json matches the image tag in docker-compose.yml.template
• Run python -m build_store_data to regenerate the zip

Commits

Scoped Commits: <scope>: <description>. The scope is the area of the tree the change touches, never a change type — write overleaf: pin mongo:8.0.4 — 8.0.x guard crashes on kernel 6.19+, not fix(overleaf): .... Body and trailers are optional; a change's reasoning belongs in the body, not in a code comment.

Scopes for this repo: the app's own name for anything under apps/ (overleaf, photoprism, affine, …), plus update docs ci meta

meta covers repo-level files (agents.md, README, justfile). For a change spanning several scopes, use a broader one, list two comma-separated, or use treewide. Merges, reverts and generated commits (update <app> from <old> to <new> [AUTO]) keep their own format. Don't generate a changelog from the commit log — release notes come from merged PRs.