withSecrets body produces no console output when run on a remote (Remoting) agent

[ymengesha]

withSecrets body produces no console output when run on a remote (Remoting) agent

Summary

When the body of withSecrets { ... } runs on a Jenkins Remoting agent (e.g. a swarm agent), all sh console output produced inside the body is missing from the build log. The same pipeline run on the controller's built-in node produces the expected output. The secret itself is fetched and propagated correctly — only console visibility is affected.

Plugin version

• onepassword-secrets 1.0.3
• Verified against Jenkins 2.535 with configuration-as-code 2036.v0b_c2de701dcb_

Reproducer

Pipeline:

pipeline {
  agent any
  stages {
    stage('test') {
      steps {
        withSecrets(secrets: [
          [envVar: 'TEST_VAR', secretRef: 'op://my-vault/my-item/credential'],
        ]) {
          sh 'echo "INSIDE first"'
          sh 'echo "len=${#TEST_VAR}"'
        }
        sh 'echo "OUTSIDE"'
      }
    }
  }
}

When agent any resolves to the controller built-in node

+ echo "INSIDE first"
INSIDE first
+ echo "len=7"
len=****
+ echo "OUTSIDE"
OUTSIDE

All three echoes visible. Secret value is correctly masked in the second one. Expected behaviour.

When agent any resolves to a Jenkins Remoting agent (swarm)

+ echo "OUTSIDE"
OUTSIDE

Only the post-block echo is visible. The two echoes inside the body are completely absent — not masked-to-asterisks, absent. No error, no warning, no exception. The stage succeeds normally.

Evidence the body actually executes on the agent (it does)

Side-channel test that bypasses the masking filter:

withSecrets(secrets: [
  [envVar: 'TEST_VAR', secretRef: 'op://my-vault/my-item/credential'],
]) {
  sh 'echo "INSIDE first" >> /tmp/wsec.log; echo "len=${#TEST_VAR}" >> /tmp/wsec.log'
}
sh 'cat /tmp/wsec.log'   // outside the body — masking filter not on this stream

On the swarm-agent run, the post-block cat prints:

INSIDE first
len=7

So:

• The body did execute on the agent.
• The env var was populated (${#TEST_VAR} = 7, matching the source op://... value's length).
• The sh step on the agent ran the script to completion.
• The output was produced — it just never reached the build console log.

Hypothesised root cause

MaskingConsoleLogFilter returns an anonymous LineTransformationOutputStream inner class from decorateLogger(Run, OutputStream):

@Override
public OutputStream decorateLogger(Run run, final OutputStream logger) throws IOException, InterruptedException {
    return new LineTransformationOutputStream() {
        Pattern p;
        @Override
        protected void eol(byte[] b, int len) throws IOException { ... }
    };
}

LineTransformationOutputStream buffers bytes until it sees \n (or the stream is close()d) before calling eol(). The outer MaskingConsoleLogFilter is Serializable, but the returned anonymous stream is not — its Pattern p field and captured logger reference don't survive a Remoting round trip cleanly. When a sh step runs on a remote agent, the output stream chain that the durable-task writes into either misses this wrapper entirely or hits a flush/close path that drops the buffered content.

Comparable bug shape in the hashicorp-vault-plugin (where this filter was originally copy-pasted from, per the source comment):

• jenkinsci/hashicorp-vault-plugin#203 — different visible symptom (masking not applied in some logs) but same underlying serialization/anonymous-class shape.

Suggested direction for a fix

Replace the anonymous LineTransformationOutputStream with a top-level (or static-nested) class that:

1. Implements Serializable explicitly and declares serialVersionUID.
2. Holds valuesToMask, charsetName, and the wrapped logger (as a transient OutputStream plus a SerializableOnlyOverRemoting shim for the remote case) as explicit fields.
3. Compiles the masking Pattern once in the constructor, not per eol() call (current behaviour recompiles the regex on every line).
4. Overrides flush() and close() to drain the line buffer explicitly so any trailing non-newline-terminated content is emitted before the stream is closed.

I'm happy to send a PR if a maintainer can confirm this is the right direction.

Impact / workaround

The bug doesn't break secret consumption — actual commands inside the body that use $TEST_VAR work correctly (verified by exit codes and downstream side-effects of curl/etc.). What's lost is console visibility, including:

• Intentional debug echos.
• Unintentional output: stdout/stderr from real commands (build tool output, error messages from failures inside the body).

Workaround we've adopted: keep the withSecrets body to the minimum scope — only the command that actually consumes the secret — and move chatty steps outside the block. This is also a defensible security pattern (smaller env-var exposure window) and sidesteps the bug entirely:

withSecrets(secrets: [[envVar: 'API_TOKEN', secretRef: 'op://vault/item/credential']]) {
    sh 'curl -fSs -H "Authorization: $API_TOKEN" https://… -o /tmp/data.json'
}
sh 'jq . /tmp/data.json && ./process.sh'   // chatty work lives outside, normal logging

For pipelines that genuinely need a noisy tool's output and must consume the secret inside the same step, redirect to a file inside the body and cat it outside.