Skip to content

fix(log): retry transient JSON generation failures - #621

Open
okt-limonikas wants to merge 1 commit into
ts-factory:mainfrom
okt-limonikas:fix/log-json-fetch-retry
Open

fix(log): retry transient JSON generation failures#621
okt-limonikas wants to merge 1 commit into
ts-factory:mainfrom
okt-limonikas:fix/log-json-fetch-retry

Conversation

@okt-limonikas

@okt-limonikas okt-limonikas commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR makes the React log viewer tolerate the short interval in which an
on-demand log JSON resource is not yet complete or temporarily unavailable.

The log JSON fetch now performs a bounded retry with abort-aware backoff for:

  • JSON parsing failures caused by empty or truncated responses;
  • transient connection and response-stream failures;
  • HTTP 404, 408, 425, and 429 responses;
  • HTTP 5xx responses.

Non-transient client errors are still returned immediately, and changing the
selected test aborts the current request and any pending retry delay.

Problem

Loading a test log is a two-stage operation:

  1. Django returns metadata containing the log-server JSON URL.
  2. The React application fetches and parses that URL separately.

The log server generates missing JSON files on demand. Before the accompanying
server fix, the public filename could become visible while the converter was
still writing it. A request could therefore return HTTP 200 with an empty,
truncated, or concurrently corrupted body.

The UI previously called response.json() once. If parsing failed, the query
remained in its error state and the selected log panel was empty or unavailable.
Selecting a different test and returning to the original test triggered a new
request after generation had completed, so the same log then appeared normally.

Why the failure is intermittent

The result depends on timing:

  • already-generated logs are served normally;
  • small logs may finish before another request reaches the public filename;
  • large logs have a wider partial-write window;
  • concurrent requests make it more likely that one client reads during that
    window.

This explains why the problem is more common for some tests and difficult to
reproduce in a development environment with cached JSON files.

Fix

The JSON fetch helper now makes at most three attempts:

  1. the initial request;
  2. a retry after 250 ms;
  3. a retry after 1,000 ms.

Retries stop immediately when the request is aborted. Stable 4xx responses such
as HTTP 400 and non-transport exceptions are not retried. If all parse attempts
fail, the caller receives a clear Log is not ready error instead of a raw
SyntaxError.

@okt-limonikas okt-limonikas self-assigned this Aug 4, 2026
@okt-limonikas okt-limonikas added the 🐛 Bug Something isn't working label Aug 4, 2026
@okt-limonikas okt-limonikas linked an issue Aug 4, 2026 that may be closed by this pull request
On-demand log JSON can be served while the log server is still writing
the file, causing response.json() to fail and leaving the selected test
in an error state until navigation retries it.

Retry transient HTTP and JSON parsing failures with bounded, abort-aware
backoff. Preserve immediate cancellation and non-retryable client
errors, and return an actionable error when generation never stabilizes.

Issue: ts-factory#573
Signed-off-by: Danil Kostromin <danil.kostromin@icloud.com>
@okt-limonikas
okt-limonikas force-pushed the fix/log-json-fetch-retry branch from fcf1ddd to fb7693b Compare August 4, 2026 15:42
okt-limonikas added a commit to okt-limonikas/test-environment that referenced this pull request Aug 4, 2026
On-demand JSON files were written directly to their public paths.
Apache could serve a file as soon as it was opened, so concurrent
clients sometimes received empty or truncated JSON.

Write node files and tree.json to same-directory temporary files and
rename them only after a successful close. Preserve output permissions
and reap temporary files abandoned by interrupted conversions without
disturbing active writers.

Link: ts-factory/bublik-ui#573
Link: ts-factory/bublik-ui#621
Signed-off-by: Danil Kostromin <danil.kostromin@icloud.com>
okt-limonikas added a commit to okt-limonikas/test-environment that referenced this pull request Aug 4, 2026
Concurrent requests for a missing JSON file could start multiple
converters for the same destination. This wasted resources and made the
partial-file race more likely.

Canonicalize requested paths and serialize generation through a bounded
256-slot lock pool. Recheck the target under the lock so waiters reuse
the completed file, return explicit HTTP statuses, and keep generator
diagnostics out of client responses.

Link: ts-factory/bublik-ui#573
Link: ts-factory/bublik-ui#621
Signed-off-by: Danil Kostromin <danil.kostromin@icloud.com>
okt-limonikas added a commit to okt-limonikas/test-environment that referenced this pull request Aug 4, 2026
Concurrent requests for a missing JSON file could start multiple
converters for the same destination. This wasted resources and made the
partial-file race more likely.

Canonicalize requested paths and serialize generation through a bounded
256-slot lock pool. Recheck the target under the lock so waiters reuse
the completed file, return explicit HTTP statuses, and keep generator
diagnostics out of client responses.

Link: ts-factory/bublik-ui#573
Link: ts-factory/bublik-ui#621
Signed-off-by: Danil Kostromin <danil.kostromin@icloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🐛 Bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Logs not loading from first time

2 participants