Skip to content

Latest commit

 

History

History
341 lines (293 loc) · 18.9 KB

configuration.md

File metadata and controls

341 lines (293 loc) · 18.9 KB

Configuration

Octop stores all of its state under ~/.octop/. The directory is created on first server start (or by octop init / octop run).

Filesystem layout

~/.octop/
├── config.json              # process-level settings (host, port, CORS, DB, TLS, …)
├── env                      # optional dotenv (OCTOP_DATABASE_*, API keys, …); loaded at server start
├── octop.db                 # SQLite — users, agents, providers, sessions, audit
├── cli_state.json           # CLI token + pinned defaults (`octop user login`)
├── repl_history             # readline-style history for `octop chats repl`
├── secrets/
│   └── jwt_secret           # 32-byte random; rotate with `octop admin rotate-jwt-secret`
├── agents/<agent_id>/       # per-agent workspace (LangGraph state, attachments, …)
├── plugins/                 # installed third-party plugins
├── ssl/                     # self-signed certs when `octop run --ssl` is used without --cert/--key
├── logs/                    # rotating log files (when `octop service start` writes a logfile)
└── octop.log                # default foreground log path

env is dotenv format, applied before config.json is loaded (OctopServer.start → apply_env_file). Dashboard Advanced → Environment variables writes this file. Saving replaces the file and drops removed keys from the process environment (host/systemd keys that were never in the file are left alone). Every running agent inherits those keys:

  • Local shell — live process env (including this file) plus the agent's workspace .env (workspace wins on the same key; OCTOP_* / HOME / USER cannot be overridden from .env). New agents store that file at {workspace}/.octop/.env; existing agents keep {workspace}/.env when their stored internal system-files prefix uses the legacy layout. Remote object-store workspaces are not read from the host path — Octop/harness also reads .env through the agent backend on each execute. Octop also injects OCTOP_AGENT_ID, OCTOP_AUTH_DIR (under {workspace}/.octop/auth for new agents; legacy agents keep {workspace}/.octop-auth), and OCTOP_HOME into shell/sandbox env.
  • Docker sandbox — re-reads ~/.octop/env on execute, plus workspace .env and a minimal PATH (not the full host environment). Admin PATH/HOME cannot override the container toolchain; workspace .env may set PATH.
  • Web search tools (Tavily, Brave, …) register at agent start from process env. Put search API keys in the global file, not only in an agent's .env. Changing search keys triggers a background agent reload; other keys take effect on the next execute without reload.

Per-agent secrets belong in {workspace}/.env or, for new agents, {workspace}/.octop/.env (also writable via the write_env_file tool). That file is excluded from published-expert snapshots.

The root can be overridden with OCTOP_HOME (absolute path). Most sub-paths are exposed as properties on PathLayout in octop.infra.utils.paths.

Local knowledge indexing and systemd resource limits

Local ONNX embeddings process one chunk at a time with two inference threads. Indexing, retrieval, and model probes share one cached model per process and serialize inference. Switching the model or OCTOP_HOME, or deleting the cached model, releases the resident session. Knowledge indexing processes one document at a time; large imports continue in the background.

On Linux, octop service start and octop service restart install/update octop.service.d/15-resource-limits.conf alongside the existing service unit. The main unit, custom ExecStart, service user, and other drop-ins are preserved. Defaults are MemoryHigh=60% and MemoryMax=75% of physical RAM, MemorySwapMax=512M, OOMPolicy=kill, a 30-second restart delay, and at most three starts within ten minutes. These limits cover the whole service and its children; they are a host safeguard, not a guarantee that every model fits. Memory controls require the corresponding systemd/cgroup support (user services need cgroup v2 with the memory controller available).

To set a machine-specific budget, use systemctl --user edit octop.service (or sudo systemctl edit octop.service for a system service). For example:

[Service]
MemoryHigh=5G
MemoryMax=6G
MemorySwapMax=512M

The resulting override.conf sorts after the managed defaults and survives future service updates. Restart the service to apply it. If repeated failures hit the start limit, correct the cause before running systemctl --user reset-failed octop.service and starting the service again (omit --user for a system service). Foreground, Docker, and macOS launchd deployments retain their own process/container resource policies.

config.json

Generated with defaults on first run; merged with environment overrides on each start. Schema (OctopConfig in octop/config.py):

{
  "bind_host": "127.0.0.1",
  "port": 8088,
  "log_level": "info",
  "access_token_ttl_seconds": 86400,
  "login_max_attempts": 5,
  "login_lockout_seconds": 900,
  "cors_origins": [],
  "default_timezone": "Asia/Shanghai",
  "enable_dashboard": true,
  "enable_api_docs": false,
  "require_setup_password": true,
  "max_upload_mb": 100,
  "database": {
    "driver": "sqlite",
    "sqlite_path": "octop.db",
    "host": "127.0.0.1",
    "port": 5432,
    "database": "octop",
    "user": "octop",
    "password": null,
    "url": null
  },
  "tls": {
    "enabled": false,
    "mode": "",
    "domains": [],
    "cert_file": "",
    "key_file": "",
    "issued_at": "",
    "expires_at": "",
    "acme_staging": false,
    "http_port": 80
  }
}

Notes:

  • default_timezone is the process default for dashboard timestamps, cron scheduling, and harness. Legacy cron_timezone in config.json and OCTOP_CRON_TIMEZONE are still accepted; the new key/env wins when both are set.
  • database.password: when the wizard configures PostgreSQL via discrete fields, this may be written into config.json (convenient for local first-run). In production, prefer using only OCTOP_DATABASE_PASSWORD or a OCTOP_DATABASE_URL that includes the password, and restrict config.json file permissions. Environment variables always override the same-named setting in the file.
  • enable_api_docs=false keeps /api/docs (Scalar) off in production while still serving /api/openapi.json to the dashboard.
  • require_setup_password=true adds the wizard password gate to the first-run setup flow; set false for unattended bootstraps via OCTOP_ADMIN_USERNAME / OCTOP_ADMIN_PASSWORD.
  • max_upload_mb is the process-wide ceiling for dashboard chat attachments, IM inbound files, and knowledge-base documents (default 100). Existing config.json files that omit the key pick up the default on load. Change it and restart; reverse proxies may still impose their own body-size limit (for example nginx client_max_body_size). Agent workspace file upload, plugin ZIPs, and backup archives use separate limits and are not this setting.
  • plugins.<id>.enabled is the global plugin switch (Dashboard Admin → Plugins). Bundled plugins are copied into ~/.octop/plugins/ on init and server start with enabled: false. bundled_plugins_seeded lists ids already offered so uninstall does not come back on the next start.

Environment overrides

Each variable, when set, takes precedence over the matching key in config.json. Unset variables leave the on-disk value untouched.

Variable Type Default Effect
OCTOP_HOME path ~/.octop Install root (DB, secrets, workspaces, plugins)
OCTOP_BIND_HOST string 127.0.0.1 Listen address (use 0.0.0.0 for LAN access)
OCTOP_PORT int 8088 Listen port (0–65535; 0 asks the OS for a free port)
OCTOP_LOG_LEVEL string info One of debug info warning error
OCTOP_LOG_RETENTION_DAYS int 14 Keep rotated octop.YYYY-MM-DD.log files for this many days
OCTOP_LOG_MAX_BYTES int 104857600 (100 MiB) Roll the active log when it exceeds this size (in addition to daily rotation)
OCTOP_LOG_COMPRESS bool true logrotate-style compress + delaycompress: gzip prior rotated plains on the next cycle (octop.YYYY-MM-DD.log.gz); the newest rotated file stays uncompressed until then
OCTOP_ACCESS_TOKEN_TTL int (seconds) 86400 JWT access-token lifetime
OCTOP_LOGIN_MAX_ATTEMPTS int 5 Failed-login attempts before lockout
OCTOP_LOGIN_LOCKOUT_SECONDS int 900 Lockout duration after OCTOP_LOGIN_MAX_ATTEMPTS failures
OCTOP_CAPTCHA_PROVIDER slug slider Login captcha (slider, turnstile, hcaptcha, recaptcha-v3, tencent, geetest-v4; recaptcha v2 stays resolvable for existing configs but is unlisted). Boot snapshot; restart after change
OCTOP_CAPTCHA_SITE_KEY string empty Public site key (Tencent: CaptchaAppId; required when the env snapshot is a strong provider)
OCTOP_CAPTCHA_SECRET string empty Siteverify secret (Tencent: AppSecretKey; never logged; GET /api/envs redacts it)
OCTOP_CAPTCHA_CAM_SECRET_ID string empty Tencent only: CAM API SecretId signing DescribeCaptchaResult; required when the env snapshot is tencent
OCTOP_CAPTCHA_CAM_SECRET_KEY string empty Tencent only: CAM API SecretKey; never logged; GET /api/envs redacts it
OCTOP_CAPTCHA_V3_MIN_SCORE float 0.5 Minimum recaptcha-v3 score; admin UI is read-only
OCTOP_DEFAULT_TIMEZONE IANA tz Asia/Shanghai Default timezone for display, scheduling, and harness (cron_timezone / OCTOP_CRON_TIMEZONE still accepted)
OCTOP_CORS_ORIGINS comma-sep list empty Permitted CORS origins for the dashboard / external callers
OCTOP_ENABLE_DASHBOARD bool true Serve the built React SPA at /
OCTOP_ENABLE_API_DOCS bool false Expose Scalar API docs at /api/docs
OCTOP_REQUIRE_SETUP_PASSWORD bool true Require wizard password during initial setup
OCTOP_MAX_UPLOAD_MB int 100 Max upload size in MiB for chat attachments, IM inbound, and knowledge documents (1–1024)
OCTOP_DATABASE_URL string empty Full DSN — overrides the OCTOP_DATABASE_* fields below
OCTOP_DATABASE_DRIVER sqlite | postgresql sqlite Storage backend
OCTOP_DATABASE_SQLITE_PATH path octop.db SQLite file path (relative to OCTOP_HOME unless absolute)
OCTOP_DATABASE_HOST string 127.0.0.1 PostgreSQL host (when driver=postgresql)
OCTOP_DATABASE_PORT int 5432 PostgreSQL port
OCTOP_DATABASE_NAME string octop PostgreSQL database name
OCTOP_DATABASE_USER string octop PostgreSQL user
OCTOP_DATABASE_PASSWORD string empty PostgreSQL password (overrides file; prefer env in production)
OCTOP_ADMIN_USERNAME string empty Pre-fills the first-admin username in octop init
OCTOP_ADMIN_PASSWORD string empty Pre-fills the first-admin password in octop init
OCTOP_ADMIN_DISPLAY_NAME string empty Pre-fills the admin display name
OCTOP_USER string empty Default --user for CLI subcommands
OCTOP_AGENT string empty Default --agent for CLI subcommands
OCTOP_SERVICE_MODE systemd | launchd auto Override the service backend (used by octop service)
OCTOP_SERVICE_SCOPE user | system auto systemd --user vs system unit (Linux only)
OCTOP_PORTAL_ORIGIN origin https://octop.cloud Plugin and expert catalog origin. No path. Empty disables the remote catalog. See private market
OCTOP_MARKET_REMOTE_ORIGIN origin https://download.octop.cloud Origin allowed for market archive download_url values. Any path on that origin is accepted. The variable itself has no path. See private market
SKILL_MARKET_ORIGIN origin https://api.skillhub.cn Skill search, rankings, and per-skill zip download. No path. Empty keeps the default. See skill market
SKILLSET_ORIGIN origin https://api.skillhub.cn Skillset detail and skillset package used when importing a skill package. No path. Empty keeps the default. See skill market

Invalid integer values are logged and ignored — the on-disk default remains in effect. database_env_configured() returns True when any OCTOP_DATABASE_* is set, which lets OctopServer.start() pick the configured backend at boot.

Docker Compose: put OCTOP_DATABASE_* in deploy/docker/.env and ensure they are listed under environment: in deploy/docker/docker-compose.yml (Compose uses .env for interpolation only; unset keys do not enter the container). Writing the same keys to the mounted ~/.octop/env also works.

When database.driver is postgresql, skill-package sources and knowledge-base originals are stored in that same database (public.skill_package_files and public.knowledge_files), and knowledge vectors use pgvector (public.knowledge_chunks). Names sit next to skill_packages and knowledge_bases. SQLite keeps the local skill-packages/ and knowledge/ directories. The vector extension is not created by Octop migrations; enable it with deploy/docker/postgres/init-vector.sql or CREATE EXTENSION vector before indexing knowledge. There is no sqlite→PostgreSQL content migration in this release — pick the driver at install time.

Agent memory vs control plane

Control-plane database and agent memory are separate layers. Defaults:

  • Control plane SQLite → agent memory stays {workspace}/memory.sqlite (or {workspace}/.octop/memory.sqlite for new agents).
  • Control plane PostgreSQL → agent memory defaults to the same DSN (octop-memory per-agent PG schema agent_<id>). Runtime also needs langgraph-checkpoint-postgres (pulled in via octop-memory[langgraph-postgres]) so LangGraph checkpoints work. To keep file memory while the control plane is PG, set on the agent:
"memory": { "backend": { "type": "sqlite" } }

Or point at another DSN with "type": "postgres", "dsn": "…". There is no automatic SQLite→PG memory data migration.

PostgreSQL extensions (e.g. vector) are instance-level ops, not control-plane migrations. Local compose enables vector via deploy/docker/postgres/init-vector.sql; managed databases need a DBA / provider toggle. See ADR 002.

Scheduled SQLite checkpoint backups

These settings require a Harness build with the scheduled backup policy below.

Harness attempts a whole-database backup before the first scheduled cleanup with eligible checkpoints. The default OCTOP_CHECKPOINT_BACKUP_BEFORE_PRUNE=first writes memory.sqlite.pre-prune.bak alongside the database. If free space is less than the database plus WAL and 10% headroom, or if the backup fails, cleanup continues and a warning is logged. The adjacent memory.sqlite.pre-prune.done marker records when the attempt completed and its result. Remove the marker to allow another attempt the next time there are eligible checkpoints.

Set OCTOP_CHECKPOINT_BACKUP_BEFORE_PRUNE=always (or 1, true, on, or yes) to replace the backup before each cleanup and defer checkpoint pruning if it cannot be written. Set it to off (or 0, false, or no) to disable backup creation. Unrecognized values fall back to first with a warning. OCTOP_CHECKPOINT_BACKUP_KEEP_DAYS defaults to 7. Set it to 0 to disable automatic expiration. Expiration leaves the marker in place.

First-boot wizard

The first request to a fresh install lands on the setup page. Greenfield SQLite installs defer opening the control-plane DB until the database step; password verification works without a pool. The modern flow uses /api/setup/*:

  1. POST /api/setup/begin (or POST /api/setup/verify-password when require_setup_password=true) — issues a short-lived wizard token.
  2. POST /api/setup/test-database / POST /api/setup/database — choose SQLite or PostgreSQL, probe, persist config.json, bind pool + migrate.
  3. POST /api/setup/initial-admin — creates the seed admin (requires DB).
  4. POST /api/setup/test-provider — pings an optional provider draft.
  5. POST /api/setup/finish — bootstraps default main agent and unlocks the rest of the API.

GET /api/setup/status returns setup_required, wizard password fields, plus database_bound / database_driver. setup_lockdown middleware blocks non-setup routes until the wizard completes.

With OCTOP_REQUIRE_SETUP_PASSWORD=false there is no CLI password; the dashboard learns this from wizard_password_required and skips both that step and the password fields of the admin step. POST /api/setup/initial-admin then accepts a body without password and mints an unguessable one, so the row keeps a real hash while the JWT returned by the same call is the actual credential. The desktop shell pins that variable for the sidecar it owns, and neither ~/.octop/env nor PUT /api/envs can override it while it runs.

Because nobody knows that password, the shell also holds a device credential (encrypted per-install secret at <OCTOP_HOME>/desktop-device.json) and exchanges it for a fresh JWT at POST /api/auth/device/token when the session expires, so OCTOP_ACCESS_TOKEN_TTL stays at its default 86400 instead of being stretched. See desktop/README.md → Accounts (no sign-in wall).

For unattended installs, use octop init --yes with OCTOP_ADMIN_USERNAME / OCTOP_ADMIN_PASSWORD (and OCTOP_REQUIRE_SETUP_PASSWORD=false if the env-var path is used). This runs the same migrations + admin creation without the HTTP wizard.

Secrets

The JWT secret is generated on first start and stored in ~/.octop/secrets/jwt_secret. Rotate it with:

octop admin rotate-jwt-secret

Rotation invalidates every outstanding access token immediately. The old secret is overwritten in place — no zero-downtime rotation today.

Per-agent provider credentials (e.g. API keys) live in the SQLite providers table and are surfaced through infra/connectors/credential_crypto.py for connector OAuth flows.

Retired segmented history archive

Octop ignores the former history_v2_enabled and OCTOP_HISTORY_V2_ENABLED settings. At startup, Octop migrates existing archive messages and supported trajectory events into the control-plane database before starting agents. Successful migrations retain the original history_v2.sqlite and history_v2.required with a .migrated-<timestamp> suffix. If a message or checkpoint anchor cannot be read, startup reports the affected thread and leaves the source archive in place so the data can be recovered and migration retried. Stop the old server before upgrading an installation with a v2 archive. Keep checkpoint retention disabled if the old server must run with the new Harness before migration. Restore a system backup containing a v2 archive with the offline backup CLI, then start Octop.