Skip to content

fix(solparq): redact S3 credentials from ClickHouse query errors - #69

Open
picard8472 wants to merge 1 commit into
mainfrom
fix/solparq-redact-s3-credentials-in-errors
Open

fix(solparq): redact S3 credentials from ClickHouse query errors#69
picard8472 wants to merge 1 commit into
mainfrom
fix/solparq-redact-s3-credentials-in-errors

Conversation

@picard8472

Copy link
Copy Markdown
Contributor

Summary

Fixes #63 — the S3 export path leaked AWS credentials in plaintext through error logs.

build_s3_table_archive_sql embeds the S3 access key and secret as the 2nd and 3rd arguments of the ClickHouse s3(...) table function. On any export failure (bad bucket, network error, ClickHouse rejecting the query), clickhouse_status_error put the first 500 chars of the raw SQL — credentials included — into the returned error and any log line carrying it.

Fix

Redact at the single choke point. clickhouse_status_error now runs the SQL through a new redact_s3_credentials before truncating:

  • Scans for each s3( / s3Cluster( call (matched at a word boundary, case-insensitive) and replaces the access-key and secret-key literals with '[REDACTED]'.
  • Leaves the URL (1st arg), format (4th arg), and the rest of the statement intact for debugging.
  • Parses literals with the same \' / \\ escaping that escape_sql_string emits, so an escaped quote in a secret can't end the redacted span early.
  • Truncation to 500 chars is now char-boundary safe (the redacted string may contain multi-byte UTF-8).

Because redaction happens at the one place errors are constructed, it also covers the restore-style s3(...) form if it is ever routed through this crate.

Testing

6 new unit tests in crates/superbank-solparq/src/clickhouse.rs:

  • redaction of the real generated archive SQL
  • the full clickhouse_status_error preview never contains credentials
  • single-line restore-style s3(...) call
  • secrets containing escaped quotes
  • non-S3 SQL left untouched
  • no false match on s3(-like text inside identifiers

cargo test -p superbank-solparq (86 tests), cargo clippy -p superbank-solparq --all-targets --locked -- -D warnings, and cargo fmt --check all pass.

🤖 Generated with Claude Code

The S3 archive export SQL built by `build_s3_table_archive_sql` embeds the
AWS access key and secret as the second and third arguments of the
ClickHouse `s3(...)` table function. On any export failure (bad bucket,
network error, ClickHouse rejecting the query), `clickhouse_status_error`
put the first 500 chars of the raw SQL — credentials included — into the
returned error and any log line carrying it, leaking both in plaintext.

Redact the credentials at the single choke point: `clickhouse_status_error`
now scans the SQL for each `s3(` / `s3Cluster(` call and replaces the
access-key and secret-key literals with `'[REDACTED]'` before truncating,
leaving the URL, format, and rest of the statement intact for debugging.
Literal parsing honors the same `\'` / `\\` escaping that
`escape_sql_string` emits, and truncation is now char-boundary safe.

Fixes #63

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failed S3 export logs query containing plaintext S3 access key and secret

1 participant