Skip to content

fix(task): recover staged artifact deletions on startup - #49

Open
HandSonic wants to merge 227 commits into
mainfrom
fix/task-artifact-delete-recovery
Open

HandSonic wants to merge 227 commits into
mainfrom
fix/task-artifact-delete-recovery

Conversation

@HandSonic

@HandSonic HandSonic commented Aug 23, 2026 •

Copy link
Copy Markdown
Owner

Related issue

N/A — gap found by reviewing the OtterMind#2712 task-system rebuild; described below.

Summary

Deleting a task with a published artifact renames the file to a hidden .{name}.task-delete-{uuid} staging name, removes the task record, then physically deletes the staged file. A crash inside that window left either an orphaned hidden file forever (task record already gone) or the user's export stuck hidden (record alive) — startup recovery only handled -events.json.deleting files.

Staged artifact deletions are now journaned (taskId, originalPath, stagedPath) to task-artifact-deletions.json beside the task data, appended atomically after the rename, cleared on commit/restore. TaskServiceImpl replays the journal at startup: a surviving task record restores the artifact to its published name; a missing record sweeps the staged file. Corrupt journal lines are skipped individually.

Affected surfaces

  • Frontend / Web
  • Backend / API / Storage
  • Database plugin / Driver
  • JCEF / Desktop packaging
  • CI / Build / Release
  • Documentation only

Verification

  • mvn -pl chat2db-community-domain/chat2db-community-domain-core -am test -Dmaven.test.skip=false: 217/217 green, incl. 4 new ArtifactServiceTest cases (restore-when-task-survives, sweep-when-task-gone, missing-staged-file no-op, commit/restore clear entries) — journal injected via a package-private constructor for hermetic tests.
  • Fork CI green: fix(task): recover staged artifact deletions on startup #49.

Risk and compatibility

  • Public API or stored data: one new journal file under the env base path; no existing format changes.
  • Database or driver compatibility: N/A.
  • Network, privacy, or security: N/A.
  • Community / Local / Pro boundary: N/A.
  • Backward compatibility: absent journal = today's behavior.

Reviewer map

  • Start here: ArtifactService.stagePublishedDeletion/recordStagedDeletion/recoverStagedDeletion, then TaskServiceImpl.recoverInterruptedArtifactDeletions.
  • Failure condition: crash between rename and journal append (one file write) still orphans — strictly narrower than before; recovery failure keeps the entry for the next startup.
  • Rollback or disable path: revert single commit; journal file can be deleted safely.

Contributor declaration

  • I linked the Issue that defines this change.
  • I tested the affected behavior and reported the actual results above.
  • I did not include credentials, private data, or generated build output.
  • I disclosed substantial AI assistance below, or this PR contains no substantial AI-generated code.

AI assistance: substantial — fix and tests drafted with AI assistance, verified locally and by adversarial review (which tightened journal loading to per-line tolerance).

Latest-main revalidation (2026-09-04)

  • Rebased onto upstream 144a04e; current head a11ada6.
  • Adversarial review found that moving an artifact before a failed journal write could hide the published file. The recovery intent is now persisted first, and failed moves roll back the journal entry.
  • Focused ArtifactService tests: 13/13 passed; full domain-core: 231/231 passed.

Aias00 and others added 24 commits August 7, 2026 01:38
- Bounded preview endpoint (CSV/XLS/XLSX via the same EasyExcel reader as
  execution): source fields with sample values, target columns with type/
  nullable/default, and a suggested mapping by exact name match.
- Mapping-driven import: remap source fields to target columns, skip
  source fields, fill unmapped target columns with DEFAULT or explicit
  NULL; NOT NULL columns without a default block execution before any
  write when unmapped.
- Row-level error reporting (source row, target column, message) with
  per-row continue-on-error; type coercion for numeric/date columns.
- Import modal gains a mapping step after file selection; i18n in all
  five locales and MYSQL-IMPORT-001 fixtures (reordered/extra columns,
  strict table, CSV samples).
Constraint: preserve the PR's preview/mapping flow while adopting main's task submission and artifact handling
Rejected: dropping preview mapping | would silently remove the feature under review
Confidence: high
Scope-risk: narrow
Tested: NODE_PATH=/Users/aias/Work/github/Chat2DB/chat2db-community-client/node_modules node ./scripts/validate-i18n.cjs --write-source-hashes; git diff --cached --check
Add `visible` field to TableIndex domain model, read the `Visible` column
from SHOW INDEXES in MySQL metadata, generate INVISIBLE keyword in index
DDL, and add buildAlterIndexVisibility for ALTER INDEX syntax.

Frontend: add a "Visible" column to the index editor with a
VISIBLE/INVISIBLE toggle, disabled for primary keys, gated to MySQL only.

Test fixtures: init.sql, grants.sql, cleanup.sql, README.md under
script/test-fixtures/mysql/MYSQL-OBJ-006/.

Closes OtterMind#2574
Replace hardcoded INVISIBLE, ALTER INDEX strings with constants
SQL_INVISIBLE and SQL_ALTER_INDEX from MysqlSqlConstants.
…ABLE

buildAlterIndexVisibility was dead code: a visibility-only change fell
through to DROP+ADD, rebuilding the whole index. ALTER TABLE now emits
ALTER INDEX ... VISIBLE/INVISIBLE when the modified index differs from
the stored one only in visibility (and is not the primary key), and
falls back to the rebuild path otherwise.
Add `visible` field to TableColumn domain model, detect "INVISIBLE"
from the EXTRA column in information_schema, and generate the INVISIBLE
keyword in column DDL via MysqlColumnTypeEnum.

Frontend: add a "Visible" column to the column editor with a
VISIBLE/INVISIBLE toggle, gated to MySQL only.

Test fixtures: init.sql, grants.sql, cleanup.sql, README.md under
script/test-fixtures/mysql/MYSQL-OBJ-003/.

Closes OtterMind#2571
Replace hardcoded INVISIBLE string with SQL_INVISIBLE constant
from MysqlMetaDataConstants in MysqlColumnTypeEnum.
MySQL column grammar requires VISIBLE/INVISIBLE before the COMMENT
clause; the previous order produced ERROR 1064 for invisible columns
that carry a comment.
Adds a read-only Active Transactions view for the datasource node:
- Lists innodb_trx joined with processlist: transaction ID, state,
  start time, age, isolation level, rows locked/modified, thread ID,
  user, host, database, and current SQL.
- Works on MySQL 5.7 and 8.0; SQL text is null without PROCESS and
  surfaced as an explicit unavailable state instead of a blank value.
- New tree menu entry with a refreshable table dialog, i18n in all five
  locales, and MYSQL-OPS-002 fixtures (admin vs limited accounts).
…tolerant fallback

1. Add `resolveShellCandidates` method to return available shell candidates based on the operating system
2. Modify `create` method to sequentially try starting shells from the candidate list, automatically falling back to the next on failure
3. Add unit tests to verify shell candidate resolution and exception scenarios
4. Add compatibility handling for scenarios where enterprise security software blocks the default shell
…igurable default shell and improved shell candidate resolution

- Introduce DEFAULT_SHELL_PROPERTY configuration property for setting the default shell
- Add processFactory factory method to support custom process creation logic
- Refactor create method to accept pre-resolved shell candidate list
- Extract PtyProcess startup logic into a separate startProcess method
- Add proper argument configuration for PowerShell and CMD commands (-NoLogo, -NoExit, -Command, etc.)
- Improve shell candidate resolution order on Windows (pwsh → PowerShell → cmd)
- Add support for zsh and bash colored prompts
- Add test helper methods to reset factory and event publisher state
- Add comprehensive unit tests covering shell configuration and failover scenarios
- Change inner classes and enums to public to support external access
- Add command parser interface to support custom command lookup logic
# Conflicts:
#	chat2db-community-client/scripts/i18n-source-hashes.json
openai0229 and others added 29 commits September 9, 2026 10:46
…h-settles

fix(dashboard): settle failed refresh requests
…ial-browser

fix(jcef): create browser during hidden startup
…a-url-encoding

fix(clickhouse): encode schema names in JDBC URLs
…semicolon

fix(db2): drop statement terminators from JDBC query constants
…urce-tree-mutation

fix(frontend): prevent duplicate datasource mutations
…xt-references

fix(ai): restore table mention suggestions
Revert merge commit 331349e relative to its first parent. Cursor flicker was reported to persist after the attempted fix; restore the prior implementation for follow-up investigation.
…r-flicker

revert(jcef): revert cursor flicker fix (OtterMind#2893)
if (!target.getParent().equals(stagingDirectory)) {
throw new IOException("invalid staging target");
}
Files.copy(source, target, StandardCopyOption.REPLACE_EXISTING);
if (!target.getParent().equals(stagingDirectory)) {
throw new IOException("invalid staging target");
}
Files.copy(source, target, StandardCopyOption.REPLACE_EXISTING);
}

private static void validateSource(File file, String originalFileName) {
if (file == null || !file.isFile() || !file.canRead() || file.length() > MAX_SIZE_BYTES
}

private static void validateSource(File file, String originalFileName) {
if (file == null || !file.isFile() || !file.canRead() || file.length() > MAX_SIZE_BYTES
private static Path stagedFile(String id) throws IOException {
for (String extension : ALLOWED_EXTENSIONS) {
Path file = stagingFile(id, extension);
if (Files.exists(file)) {

private static void deleteQuietly(Path path) {
try {
Files.deleteIfExists(path);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants