Skip to content

Commit 428c614

Browse files
author
FZ2000
committed
docs: correct behavioural contracts the code stopped matching
An audit compared every prose claim against the implementation. These are the ones a scripting user would have been burned by. In every case the code is right and the documentation was describing an older version. EXIT CODES — eleven places across five files sync.md, search.md x3, inspect.md x4, auth.md and config.md all said an unhandled HTTPError escapes as a Python traceback and exits 1, "not the 2 that the rest of the CLI uses". main() has caught it since the error-advice refactor: except urllib.error.HTTPError as e: specific = e.code in (401, 403, 404, 429) or 500 <= e.code < 600 _fail_with_context(_advice_for_http(e), offer_traceback=not specific) and _fail_with_context ends in sys.exit(2), with the traceback written at debug level so -v shows it and a default run does not. Measured, not assumed — stubbing a 400 into `da daily`: [error] DeviantArt refused the request (HTTP 400 Bad Request). ACTUAL exit code: 2 The docs told anyone writing `case $? in` to branch on a code the CLI never returns. PermissionError was documented the same way and is likewise caught, as an OSError. Note what was NOT changed: `config get` exiting 1 for an unset key, `auth status` exiting 1 on warn, `sync watched` exiting 1 on partial failure, and exit-codes.md's statement that a genuinely unrecognised exception is left alone with its traceback. Those are all still true — main()'s except chain does not catch, say, a ValueError. `da whoami` was described as calling the API directly "rather than through the retry-on-401 wrapper", so a server-side revocation supposedly produced a traceback. cmd_whoami goes through authed_http_json and recovers by itself. The confusion is understandable: cmd_auth_status DOES call http_json directly a few hundred lines earlier, deliberately, because its job is to test the token as-is. RESUME — sync.md, four passages "There is no per-artist 'how far did I get' marker in state.json … Resuming is manual." There is: _record_gallery_progress writes state["galleries"][<artist>], _gallery_progress reads it back, and a truncated walk logs `resuming alice at offset 96 (previous walk did not finish)`. The early all-known exit is gated on the artist being recorded complete, precisely so it cannot strand the rest of the gallery. `sync watched` was documented as walking every artist at "--offset 0, always". It passes offset=None deliberately, with a comment saying why — pinning to 0 would defeat per-artist resume across a backfill, which is the case that needs it most. docs/reference/files-on-disk.md already documented the marker, so sync.md contradicted both the code and a sibling page. `da auth status` — documented as offline "no network, no locks, one small object on stdout" and "Nothing else is consulted — not the config file, not the network, not DeviantArt". It resolves an access token and calls /placebo, which means it takes the token lock and, on an hourly polling cadence, rotates the refresh token on almost every call. The page now says so, and documents the two states it can return that were missing from the three-key table — `revoked` and `unreachable`, plus the `error` key that accompanies them. The distinction matters: `unreachable` is why a dropped wifi link does not send someone to re-authenticate. SMALLER, ALL VERIFIED AGAINST THE CODE - "Eleven commands accept --json" — thirteen do, walking build_parser(). The list also omitted `daily` and `search user`. - "in particular DA_JITTER is read by install_schedule.sh … never by `da` itself" — config.py's env_map has DA_JITTER. It was documented under a heading that says the opposite; now listed as the config override it is, in the table with the other four. - "NO_COLOR … Honoured regardless of --color" — it is consulted only in the `auto` branch; `--color always` sets colour unconditionally, which is what makes forcing colour into a pipe possible. - config.md said values are "not validated or converted" and that `set` has no exit 2. Measured: `da config set jitter 40%` prints `[error] jitter must be a number — got '40%'` and exits 2. A non-object config.json warns and is replaced at exit 0, rather than raising the documented TypeError. Verified: lychee 194 OK / 0 errors with fragments checked, doc references and doc flag tables both clean. One of my own edits introduced a broken `#configuration` anchor and then a pointer to a table DA_JITTER was not actually in; lychee caught the first and the second is fixed by adding the row.
1 parent 6bd4d76 commit 428c614

7 files changed

Lines changed: 150 additions & 96 deletions

File tree

docs/commands/auth.md

Lines changed: 39 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -333,29 +333,46 @@ usage: da auth status [-h]
333333

334334
Prints one line of JSON describing how much life the refresh-token chain
335335
has left, and encodes the same answer in its exit code. It is built for
336-
cron, launchd and monitoring wrappers: no network, no locks, one small
337-
object on stdout. Reach for it when you want a machine to
338-
notice the 90-day ceiling coming; reach for `da diagnose` when you want
339-
a person to read the whole health picture.
336+
cron, launchd and monitoring wrappers: one small object on stdout, no
337+
prose. Reach for it when you want a machine to notice the 90-day ceiling
338+
coming; reach for `da diagnose` when you want a person to read the whole
339+
health picture.
340+
341+
It **does** talk to DeviantArt — see below. Poll it on a schedule, not in
342+
a tight loop.
340343

341344
This command defines no flags of its own beyond `-h`, `--help`.
342345

343346
### Behaviour
344347

345348
It reads `refresh_token_issued_at` from `state.json` and subtracts the
346-
elapsed time from 90 days (`REFRESH_TOKEN_TTL_DAYS`). Nothing else is
347-
consulted — not the config file, not the network, not DeviantArt. A
348-
green result therefore means "the chain has not aged out", not "the
349-
token works": for the second question use `da whoami` or `da diagnose`,
350-
both of which actually exercise the credentials.
351-
352-
The output object always has the same three keys:
349+
elapsed time from 90 days (`REFRESH_TOKEN_TTL_DAYS`) — and then it
350+
confirms the answer with DeviantArt, by resolving an access token and
351+
calling `/placebo`. So a green result means both "the chain has not aged
352+
out" *and* "DeviantArt still accepts it", which is the answer a
353+
monitoring wrapper actually wants; the alternative was reporting healthy
354+
for a grant that had been revoked server-side.
355+
356+
Two consequences worth planning around:
357+
358+
- **It takes the token lock and can rotate your refresh token.** An
359+
access token lives an hour and is refreshed 60 s early, so anything
360+
polling on roughly an hourly cadence will refresh — and therefore
361+
rotate — on almost every call. That is safe (rotation is serialised),
362+
but it is not a read-only probe.
363+
- **It needs the network.** No connectivity is reported as
364+
`unreachable`, distinct from `revoked`, so a dropped wifi link does not
365+
send you to re-authenticate.
366+
367+
The output object always carries these three keys, plus `error` on the
368+
two failure states:
353369

354370
| Field | Type | Meaning |
355371
| --- | --- | --- |
356-
| `state` | string | `ok`, `warn`, `crit` or `unknown` |
357-
| `days_remaining` | float or null | Days left in the chain, rounded to one decimal; `null` when `state` is `unknown` |
372+
| `state` | string | `ok`, `warn`, `crit`, `unknown`, `revoked` or `unreachable` |
373+
| `days_remaining` | float or null | Days left in the chain, rounded to one decimal; `null` when `state` is `unknown`, `0.0` when `revoked` |
358374
| `issued_at_iso` | string or null | When the chain was issued, ISO 8601 in UTC; `null` when `state` is `unknown` |
375+
| `error` | string | Only on `revoked` / `unreachable`: what DeviantArt or the network said |
359376

360377
The thresholds are `REFRESH_TOKEN_WARN_DAYS` (14) and
361378
`REFRESH_TOKEN_CRIT_DAYS` (3), and the comparisons are strict: more than
@@ -442,22 +459,19 @@ prints the first three lines, warns, and exits 0:
442459
[warn] (re-run `da auth --scope "user browse"` to broaden)
443460
```
444461

445-
Two rough edges are worth knowing, because both look alarming and
446-
neither means your installation is broken. `da whoami` calls the API
447-
directly rather than through the retry-on-401 wrapper the sync commands
448-
use, so an access token that is locally fresh but has been revoked
449-
server-side produces an unhandled `urllib.error.HTTPError: HTTP Error
450-
401: Unauthorized` traceback and exit 1, not a tidy exit 2. Any non-403
451-
error from `/user/whoami` behaves the same way, as does a network
452-
failure while refreshing. If you see that traceback, run `da refresh` or
453-
`da auth`; the corresponding path in `da sync` recovers by itself.
462+
`da whoami` goes through `authed_http_json`, the same retry-on-401
463+
wrapper the sync commands use, so an access token that is locally fresh
464+
but has been revoked server-side is recovered automatically: the wrapper
465+
forces a refresh and retries once. You see the refresh happen, not an
466+
error. If the refresh itself fails — a dead refresh token, or no network
467+
— the command reports it in one line and exits 2.
454468

455469
### Exit codes
456470

457471
0 when the token is valid, including the `browse`-only degraded case; 2
458-
when there is no refresh token or `/placebo` does not report success. An
459-
unhandled HTTP or network error exits 1 with a traceback, as described
460-
above.
472+
when there is no refresh token, when `/placebo` does not report success,
473+
or when an HTTP or network error prevents asking. Every failure path here
474+
ends in 2 — there is no case that exits 1.
461475

462476
## `da refresh`
463477

docs/commands/config.md

Lines changed: 30 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -25,11 +25,9 @@ first:
2525

2626
1. **A command-line flag** on the command being run — `da sync feed
2727
--concurrency 8`, `da auth --redirect-uri ...`.
28-
2. **An environment variable.** Only four exist: `DA_CLIENT_ID`,
29-
`DA_CLIENT_SECRET`, `DA_DESTINATION`, `DA_REDIRECT_URI`. No other key
30-
can be set from the environment — in particular `DA_JITTER` is read
31-
by `install_schedule.sh` when it writes the launchd job, never by
32-
`da` itself. See
28+
2. **An environment variable.** Five exist: `DA_CLIENT_ID`,
29+
`DA_CLIENT_SECRET`, `DA_DESTINATION`, `DA_REDIRECT_URI` and
30+
`DA_JITTER`. No other config key can be set from the environment. See
3331
[environment variables](../reference/environment-variables.md).
3432
3. **The macOS Keychain**, for secrets only. The only secret is
3533
`client_secret`; it is stored under service `da-cli`, account
@@ -85,8 +83,9 @@ A `config.json` that is not valid JSON, or that parses to something
8583
other than an object (`[]`, `"x"`, `42`), produces a warning on stderr
8684
and is then ignored — `show` continues with an empty file and still
8785
prints the environment and Keychain values. A `config.json` that exists
88-
but cannot be read at all, for example mode `000`, is not handled: the
89-
`PermissionError` propagates and you get a traceback and exit 1.
86+
but cannot be read at all, for example mode `000`, raises
87+
`PermissionError`, which `main()` catches as an `OSError`: one line
88+
naming the path, and exit 2.
9089

9190
`show` writes nothing and creates nothing, including the directories it
9291
prints.
@@ -219,14 +218,14 @@ Three things about `set` are worth knowing before you trust it:
219218
it; the sync and diagnose commands expand it when they use it. This is
220219
why `config show` prints `~/Pictures/DA` rather than an absolute path.
221220

222-
Two failure modes are less friendly than they should be. If
223-
`config.json` contains unparsable JSON, `set` treats it as empty and
224-
overwrites the file with just the key you passed — everything else in it
225-
is lost. If it contains valid JSON that is not an object, `set` raises
226-
`TypeError: list indices must be integers or slices, not str` and exits
227-
1 with a traceback. In both cases the file is worth inspecting by hand
228-
before you write to it; `config show` will tell you which case you are
229-
in.
221+
One failure mode is less friendly than it should be. If `config.json`
222+
contains unparsable JSON, `set` treats it as empty and overwrites the
223+
file with just the key you passed — everything else in it is lost, with
224+
no warning. A file containing valid JSON that is *not* an object is
225+
handled better: `set` warns
226+
`expected a JSON object; replacing it` and continues, exit 0. Either way
227+
the file is worth inspecting by hand first; `config show` will tell you
228+
which case you are in.
230229

231230
Finally, the value is a command-line argument. It appears in your shell
232231
history and, for the moment the process runs, in `ps`. For a secret you
@@ -266,10 +265,22 @@ $ da config set client_secret <your-client-secret>
266265

267266
### Exit codes
268267

269-
0 on success. There is no 2: the ways `set` fails are unhandled
270-
exceptions, which exit 1 with a traceback — an unwritable config
271-
directory (`PermissionError`) and the non-object `config.json` case
272-
described above.
268+
0 on success, 2 on a rejected value. `set` validates the numeric keys
269+
before writing, so a bad value is caught rather than written and left to
270+
break the next sync:
271+
272+
```console
273+
$ da config set jitter 40%
274+
[error] jitter must be a number — got '40%'
275+
$ echo $?
276+
2
277+
```
278+
279+
`delay_api`, `delay_image` and `jitter` must parse as numbers;
280+
`concurrency` and `time_budget` must be whole numbers. Values are stored
281+
as given and coerced on read, so `3` and `3.0` both work. An unwritable
282+
config directory raises `PermissionError`, which `main()` catches as an
283+
`OSError` — also 2.
273284

274285
## config get
275286

docs/commands/inspect.md

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -37,11 +37,12 @@ Beyond that:
3737
or write the destination folder or the SQLite index.
3838
- They take no lock, so they are safe to run while a `da sync` is in
3939
progress — the sync lock is held by sync commands only.
40-
- The only failures they handle are the ones documented in each section
41-
below. Any other non-2xx response from DeviantArt propagates as an
42-
unhandled `urllib.error.HTTPError`: a Python traceback and exit
43-
status `1`, not the usual `2`. A 5xx or a network error is retried
44-
twice before that happens (`HTTP_RETRY_DEFAULT = 2` in
40+
- The only failures they handle *themselves* are the ones documented in
41+
each section below. Any other non-2xx response from DeviantArt reaches
42+
`main()`'s backstop handler, which prints one line naming the status and
43+
exits `2` — the usual code, not a traceback. `-v` shows the traceback,
44+
which is written at debug level. A 5xx or a network error is retried
45+
twice before any of that (`HTTP_RETRY_DEFAULT = 2` in
4546
`dacli/constants.py`); 4xx responses are never retried.
4647

4748
## `da user profile`
@@ -86,9 +87,9 @@ on this command, so those six fields are the whole interface to it.
8687
Nothing is cached: each run is a fresh API call.
8788

8889
There is no "not found" handling either. A username DeviantArt rejects
89-
fails the request and surfaces as a traceback with exit `1`; a response
90-
carrying no `user` object prints the labels with `None` values rather
91-
than erroring.
90+
fails the request and surfaces as a one-line error with exit `2`; a
91+
response carrying no `user` object prints the labels with `None` values
92+
rather than erroring.
9293

9394
### Example
9495

@@ -169,8 +170,8 @@ is only worth it if you want to reshape it.
169170
Exits `2`, with `[error] no metadata for deviationid <id>` on stderr,
170171
when the response contains no metadata entry for the id you asked
171172
about. That is the empty-result path, not the bad-id path: an id
172-
DeviantArt rejects outright fails the request instead, with a traceback
173-
and exit `1`.
173+
DeviantArt rejects outright fails the request instead — also exit `2`,
174+
but with the HTTP status in the message rather than the id.
174175

175176
## `da deviation morelikethis`
176177

@@ -250,7 +251,7 @@ that needs more than the default token scope.
250251

251252
Neither number is validated or clamped by the CLI, so whatever limits
252253
DeviantArt enforces on the endpoint are the ones that apply, and a
253-
rejected value comes back as a traceback with exit `1`.
254+
rejected value comes back as a one-line error with exit `2`.
254255

255256
### Behaviour
256257

docs/commands/search.md

Lines changed: 19 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -107,13 +107,19 @@ string), `is_mature`, `is_downloadable`, an `author` object
107107
object (`comments`, `favourites`), and `preview` / `content` / `thumbs`
108108
image entries whose `src` values are signed CDN URLs.
109109

110-
**Errors.** These commands do not catch HTTP failures. A rejected
111-
`--limit`, a malformed date or an unauthorised endpoint surfaces as a
112-
Python traceback ending in a line like
113-
`urllib.error.HTTPError: HTTP Error 400: Bad Request`, and the process
114-
exits `1` — not the `2` that the rest of the CLI uses for "could not do
115-
the job" (see [exit codes](../reference/exit-codes.md)). An empty result
116-
set is not an error: nothing is printed and the exit code is `0`.
110+
**Errors.** These commands do not handle HTTP failures themselves; the
111+
failure reaches `main()`'s backstop handler. A rejected `--limit`, a
112+
malformed date or an unauthorised endpoint prints one line —
113+
114+
```text
115+
[error] DeviantArt refused the request (HTTP 400 Bad Request).
116+
```
117+
118+
— and exits `2`, the same code the rest of the CLI uses for "could not do
119+
the job" (see [exit codes](../reference/exit-codes.md)). The traceback is
120+
written at debug level, so `-v` shows it and the default run does not. An
121+
empty result set is not an error: nothing is printed and the exit code is
122+
`0`.
117123

118124
**Rate limiting.** Each invocation makes exactly one API request and
119125
sleeps for nothing, so none of the `--delay-api` / `--jitter` throttling
@@ -403,9 +409,10 @@ not hit a `/browse/` endpoint — it POSTs to `/user/whois`, because that
403409
endpoint rejects GET with HTTP 400 — and the only one with no `--limit`
404410
and no `--mature`. The request hardcodes `mature_content=true`, so a
405411
`--mature` flag would have nothing to do.
406-
Being a `/user/` endpoint, it needs a token authorised for it; a token
407-
that is not gets HTTP 401 back, which surfaces as a traceback and exit
408-
`1` rather than a message.
412+
Being a `/user/` endpoint, it needs a token authorised for it. A token
413+
that is not gets HTTP 401 back, which surfaces as
414+
`[error] DeviantArt rejected the credentials (HTTP 401 Unauthorized)` and
415+
exit `2`.
409416

410417
Each record DA returns prints as one line: `@` and the username as DA
411418
spells it, the user's UUID in parentheses, and `type=` followed by the
@@ -444,8 +451,8 @@ however many picks DA made that day — seventeen on
444451
the day this page was written — and only the human-readable form.
445452

446453
A malformed date is not validated locally. A day-first date such as
447-
`15-01-2026` is sent to DA as-is, comes back 400, and ends in a
448-
traceback with exit `1`.
454+
`15-01-2026` is sent to DA as-is, comes back 400, and ends in a one-line
455+
error with exit `2`.
449456

450457
```console
451458
$ da daily 2026-01-15 | head -5

docs/commands/sync.md

Lines changed: 36 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -198,10 +198,13 @@ corrupt index, or a 401 that survives a forced token refresh. `0` — with
198198
a `skipping:` message — when another sync already holds the lock.
199199

200200
One deviation from the norm: an unexpected HTTP status from the feed
201-
endpoint (anything that is not 429 or a retried 5xx) is **not** caught.
202-
It escapes as a Python traceback and the process exits `1`, not `2`.
203-
`sync artist` handles the same situation differently, which is worth
204-
knowing before you write a wrapper script around either.
201+
endpoint (anything that is not 429 or a retried 5xx) is not handled
202+
inside the walk. It reaches `main()`'s backstop handler, which prints a
203+
one-line explanation and exits `2` — the same code the rest of the CLI
204+
uses for "could not do the job". No traceback, unless you pass `-v`.
205+
`sync artist` handles the same situation inside the walk instead,
206+
recording a stop reason; the difference matters if you parse the summary
207+
line, not if you branch on the exit code.
205208

206209
## da sync artist
207210

@@ -255,21 +258,25 @@ Two details make the early stop safe rather than merely fast:
255258
- Known ids are filtered out before the metadata batch, so a page that
256259
is 23/24 duplicates costs one metadata call for one deviation.
257260

258-
What the early stop does **not** do is compensate for a walk that
259-
stopped part-way down. There is no per-artist "how far did I get" marker
260-
in `state.json`; the index is the only memory, and it says nothing about
261-
order. So if a run is truncated by its time budget at offset 96, the
262-
newest four pages are indexed and everything older is not — and the next
263-
plain run reads page 0, finds it entirely known, reports `caught up` and
264-
stops without ever requesting offset 96 again. Resuming is manual, which
265-
is why the summary is followed by a
266-
`resume: da sync artist <name> --offset <n>` line whenever the walk
267-
stopped for a non-terminal reason.
268-
269-
The same offset is recorded in `state.json` as `last_sync.last_offset`
270-
(only for non-terminal stops, so `gallery complete` and `caught up` do
271-
not record one). Re-run with that `--offset`, or with `--full`, or the
272-
rest of the gallery stays unsynced.
261+
A walk that stops part-way down **is** resumed automatically. The
262+
per-artist position is recorded in `state.json` under
263+
`galleries.<artist>`, so if a run is truncated by its time budget at
264+
offset 96, the next plain run starts there rather than at page 0:
265+
266+
```text
267+
resuming alice at offset 96 (previous walk did not finish)
268+
```
269+
270+
That marker is also what stops the early "everything on this page is
271+
known" exit from stranding the rest of the gallery — the early stop is
272+
gated on the artist being recorded as complete, so a truncated walk
273+
keeps going instead of reporting `caught up` at page 0.
274+
275+
`--offset` still overrides it when you pass one explicitly, and `--full`
276+
still ignores it and walks from the top. The summary's
277+
`resume: da sync artist <name> --offset <n>` line is a convenience for
278+
running the rest immediately; you do not need it for the next scheduled
279+
run to make progress.
273280

274281
**`--full`** disables the early stop only. It walks every page to the
275282
end of the gallery but still skips anything the index already knows, so
@@ -408,8 +415,12 @@ daily job; `sync feed` is.
408415
| `--concurrency CONCURRENCY` | int | config `concurrency`, else `4` | Passed to each artist walk, clamped to 1–16. |
409416
| `--dry-run` | flag | off | Passed to each artist walk. Nothing is written and the index is untouched, but every gallery is still paged and metadata is still fetched. |
410417

411-
There is no `--limit` and no `--offset`. Each artist is walked with
412-
`--limit 24` and `--offset 0`, always.
418+
There is no `--limit` and no `--offset`. Each artist is walked at the
419+
gallery page cap, and — deliberately — with no offset supplied, so each
420+
one resumes its own unfinished walk from the position recorded in
421+
`state.json`. Passing `0` here would pin every artist to page 0 and
422+
defeat that, which is precisely the case a backfill across many artists
423+
needs most.
413424

414425
### Behaviour
415426

@@ -448,10 +459,10 @@ for even one page, so the rest are skipped with a warning reading
448459

449460
Skipped artists are not failures. Re-running picks up where it stopped in
450461
the useful sense: the artists already walked are all-known, so each costs
451-
one API call before the walk moves on — but a *truncated* artist, one cut
452-
off mid-gallery, is not resumed, for exactly the reason described under
453-
[`sync artist`](#behaviour-1). Its remaining pages need
454-
`da sync artist <name> --offset N` or `--full`.
462+
one API call before the walk moves on, and a *truncated* artist one cut
463+
off mid-gallery — picks up where it left off, for the reason described
464+
under [`sync artist`](#behaviour-1). Nothing needs doing by hand; a
465+
backfill across many artists converges over successive scheduled runs.
455466

456467
**Failure handling.** Each artist runs inside a `try`. An artist that
457468
exits or crashes is logged, counted as failed, and the run moves on to

0 commit comments

Comments
 (0)