Skip to content

Improve kernel supervisor wrapper to make environment more predictable and configurable #7797

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
May 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion extensions/positron-supervisor/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,12 @@
"default": false,
"description": "%configuration.attachOnStartup.description%"
},
"kernelSupervisor.runInShell": {
"scope": "window",
"type": "boolean",
"default": true,
"description": "%configuration.runInShell.description%"
},
"kernelSupervisor.sleepOnStartup": {
"scope": "window",
"type": "number",
Expand Down Expand Up @@ -144,7 +150,7 @@
},
"positron": {
"binaryDependencies": {
"kallichore": "0.1.39"
"kallichore": "0.1.41"
}
},
"extensionDependencies": [
Expand Down
1 change: 1 addition & 0 deletions extensions/positron-supervisor/package.nls.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
"configuration.connectionTimeout.description": "Timeout in seconds for connecting to the kernel's sockets",
"configuration.attachOnStartup.description": "Run <f5> before starting up Jupyter kernel (when supported)",
"configuration.sleepOnStartup.description": "Sleep for n seconds before starting up Jupyter kernel (when supported)",
"configuration.runInShell.description": "Run kernels in a login shell (on Unix-like systems)",
"command.positron.supervisor.category": "Kernel Supervisor",
"command.showKernelSupervisorLog.title": "Show the Kernel Supervisor Log",
"command.reconnectSession.title": "Reconnect the Current Session",
Expand Down
55 changes: 52 additions & 3 deletions extensions/positron-supervisor/resources/supervisor-wrapper.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env bash

# ---------------------------------------------------------------------------------------------
# Copyright (C) 2024 Posit Software, PBC. All rights reserved.
# Copyright (C) 2024-2025 Posit Software, PBC. All rights reserved.
# Licensed under the Elastic License 2.0. See LICENSE.txt for license information.
# ---------------------------------------------------------------------------------------------

Expand All @@ -14,18 +14,67 @@
# usage message and exit with an error code.
if [ $# -lt 2 ]; then
echo "Usage: $0 <output-file> <program> [program-args...]" >&2
echo " $0 nohup <output-file> <program> [program-args...]" >&2
exit 1
fi

# Check if the first argument is "nohup". If it is, we'll run the supervisor
# process with nohup, rather than running it directly. This is used to prevent
# the supervisor from exiting when Positron does when the user has configured
# the supervisor to run in the background.
use_nohup=false
if [ "$1" = "nohup" ]; then
use_nohup=true
shift

# After shifting, make sure we still have enough arguments
if [ $# -lt 2 ]; then
echo "Usage: $0 nohup <output-file> <program> [program-args...]" >&2
exit 1
fi
fi

# The first argument is the output file; consume it.
output_file="$1"
shift

# Get the user's default shell. We run the program in a login shell to allow for
# startup and environment variable customization in e.g. .bash_profile
DEFAULT_SHELL=$SHELL

# If $SHELL is not set, try to use the environment
if [ -z "$DEFAULT_SHELL" ]; then
# Fall back to bash as a reasonable default
DEFAULT_SHELL=$(which bash 2>/dev/null || which sh)
fi

# Ensure we have a valid shell
if [ -z "$DEFAULT_SHELL" ] || [ ! -x "$DEFAULT_SHELL" ]; then
echo "Error: Could not determine a valid shell." >&2
exit 1
fi

# Print the command line to the log file
echo "$@" >> "$output_file"
echo "$DEFAULT_SHELL" --login -c "$@" >> "$output_file"

# Quote the arguments to handle single quotes and spaces correctly
QUOTED_ARGS=""
for arg in "$@"; do
# Escape any single quotes in the argument
escaped_arg=$(printf "%s" "$arg" | sed "s/'/'\\\\''/g")
# Add the escaped argument with single quotes
QUOTED_ARGS="${QUOTED_ARGS} '${escaped_arg}'"
done

# Run the program with its arguments, redirecting stdout and stderr to the output file
"$@" >> "$output_file" 2>&1
if [ "$use_nohup" = true ]; then
# Use nohup and explicitly redirect its output to prevent nohup.out from being created
nohup $DEFAULT_SHELL --login -c "${QUOTED_ARGS}" >> "$output_file" 2>&1 &
# Wait for the background process to complete
wait $!
else
$DEFAULT_SHELL --login -c "${QUOTED_ARGS}" >> "$output_file" 2>&1
fi

# Save the exit code of the program
exit_code=$?
Expand Down
159 changes: 117 additions & 42 deletions extensions/positron-supervisor/src/KallichoreAdapterApi.ts
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,22 @@ export class KCApi implements PositronSupervisorApi {
_context.subscriptions.push(vscode.commands.registerCommand('positron.supervisor.restartSupervisor', () => {
this.restartSupervisor();
}));

// Listen for changes to the idle shutdown hours config setting; if the
// server is running, apply the change immediately
if (vscode.env.uiKind === vscode.UIKind.Desktop) {
const configListener = vscode.workspace.onDidChangeConfiguration((event) => {
if (event.affectsConfiguration('kernelSupervisor.shutdownTimeout')) {
if (this._started.isOpen()) {
this._log.appendLine(
'Updating server configuration with new shutdown timeout: ' +
this.getShutdownHours());
this.updateIdleTimeout();
}
}
});
_context.subscriptions.push(configListener);
}
}

/**
Expand Down Expand Up @@ -290,10 +306,10 @@ export class KCApi implements PositronSupervisorApi {
wrapperPath = 'start';
shellArgs.unshift('/b', kernelWrapper);
} else {
// Use nohup as the wrapper on Unix-like systems
// Use nohup as the wrapper on Unix-like systems; this becomes
// the first argument to the wrapper script.
this._log.appendLine(`Running Kallichore server with nohup to persist sessions`);
wrapperPath = 'nohup';
shellArgs.unshift(kernelWrapper);
shellArgs.unshift('nohup');
}
}

Expand All @@ -312,43 +328,12 @@ export class KCApi implements PositronSupervisorApi {
shellArgs.push('--connection-file', connectionFile);
}

// Compute the appropriate value for the idle shutdown hours setting.
//
// This setting is primarily used in Remote SSH mode to allow kernel
// sessions to persist even when Positron itself is closed. In this
// scenario, we want keep the sessions alive for a period of time so
// they are still running when the user reconnects to the remote host,
// but we don't want them to run forever (unless the user wants to and
// understands the implications).
if (shutdownTimeout === 'immediately') {
// In desktop mode, when not persisting sessions, set the idle
// timeout to 1 hour. This is a defensive move since we generally
// expect the server to exit when the enclosing terminal closes;
// the 1 hour idle timeout ensures that it will eventually exit if
// the process is orphaned for any reason.
if (vscode.env.uiKind === vscode.UIKind.Desktop) {
shellArgs.push('--idle-shutdown-hours', '1');
}

// In web mode, we do not set an idle timeout at all by default,
// since it is normal for the front end to be disconnected for long
// periods of time.
} else if (shutdownTimeout === 'when idle') {
// Set the idle timeout to 0 hours, which causes the server to exit
// 30 seconds after the last session becomes idle.
shellArgs.push('--idle-shutdown-hours', '0');
} else if (shutdownTimeout !== 'indefinitely') {
// All other values of this setting are numbers that we can pass
// directly to the supervisor.
try {
// Attempt to parse the value as an integer
const hours = parseInt(shutdownTimeout, 10);
shellArgs.push('--idle-shutdown-hours', hours.toString());
} catch (err) {
// Should never happen since we provide all the values, but log
// it if it does.
this._log.appendLine(`Invalid hour value for kernelSupervisor.shutdownTimeout: '${shutdownTimeout}'; persisting sessions indefinitely`);
}
// Set the idle shutdown hours from the configuration. This is used to
// determine how long to wait before shutting down the server when
// idle.
const idleShutdownHours = this.getShutdownHours();
if (idleShutdownHours >= 0) {
shellArgs.push('--idle-shutdown-hours', idleShutdownHours.toString());
}

// Start the server in a new terminal
Expand Down Expand Up @@ -418,7 +403,7 @@ export class KCApi implements PositronSupervisorApi {
}));

// Wait for the terminal to start and get the PID
await terminal.processId;
let processId = await terminal.processId;

// If an HTTP proxy is set, exempt the supervisor from it; since this
// is a local server, we generally don't want to route it through a
Expand All @@ -442,6 +427,14 @@ export class KCApi implements PositronSupervisorApi {
const status = await this._api.serverStatus();
this._log.appendLine(`Kallichore ${status.body.version} server online with ${status.body.sessions} sessions`);

// Update the process ID; this can be different than the process
// ID in the hosting terminal when the supervisor is run in an
// shell and/or with nohup
if (processId !== status.body.processId) {
this._log.appendLine(`Running as pid ${status.body.processId} (terminal pid ${processId})`);
processId = status.body.processId;
}

// Make sure the version is the one expected in package.json.
const version = this._context.extension.packageJSON.positron.binaryDependencies.kallichore;
if (status.body.version !== version) {
Expand Down Expand Up @@ -532,13 +525,74 @@ export class KCApi implements PositronSupervisorApi {
base_path: this._api.basePath,
port,
server_path: shellPath,
server_pid: await terminal.processId || 0,
server_pid: processId || 0,
bearer_token: bearerToken,
log_path: logFile
};
this._context.workspaceState.update(KALLICHORE_STATE_KEY, state);
}

/***
* Get the number of hours to wait before shutting down the server when idle.
*
* Special values:
* 0 Shut down immediately after the last session becomes idle, with a 30 minute
* grace period.
* -1 Let the server run indefinitely.
*/
getShutdownHours(): number {
// In web mode, never set an idle timeout since the server is expected to
// be running for long periods of time.
if (vscode.env.uiKind === vscode.UIKind.Web) {
return -1;
}

// In other modes, get the shutdown timeout from the configuration.
const config = vscode.workspace.getConfiguration('kernelSupervisor');
const shutdownTimeout = config.get<string>('shutdownTimeout', 'immediately');

// Compute the appropriate value for the idle shutdown hours setting.
//
// This setting is primarily used in Remote SSH mode to allow kernel
// sessions to persist even when Positron itself is closed. In this
// scenario, we want keep the sessions alive for a period of time so
// they are still running when the user reconnects to the remote host,
// but we don't want them to run forever (unless the user wants to and
// understands the implications).
if (shutdownTimeout === 'immediately') {
// In desktop mode, when not persisting sessions, set the idle
// timeout to 1 hour. This is a defensive move since we generally
// expect the server to exit when the enclosing terminal closes;
// the 1 hour idle timeout ensures that it will eventually exit if
// the process is orphaned for any reason.
if (vscode.env.uiKind === vscode.UIKind.Desktop) {
return 1;
}

// In web mode, we do not set an idle timeout at all by default,
// since it is normal for the front end to be disconnected for long
// periods of time.
} else if (shutdownTimeout === 'when idle') {
// Set the idle timeout to 0 hours, which causes the server to exit
// 30 seconds after the last session becomes idle.
return 0;
} else if (shutdownTimeout !== 'indefinitely') {
// All other values of this setting are numbers that we can pass
// directly to the supervisor.
try {
// Attempt to parse the value as an integer
const hours = parseInt(shutdownTimeout, 10);
return hours;
} catch (err) {
// Should never happen since we provide all the values, but log
// it if it does.
this._log.appendLine(`Invalid hour value for kernelSupervisor.shutdownTimeout: '${shutdownTimeout}'; persisting sessions indefinitely`);
}
}

return -1;
}

/**
* Attempt to reconnect to a Kallichore server that was previously running.
*
Expand Down Expand Up @@ -585,12 +639,33 @@ export class KCApi implements PositronSupervisorApi {
this._started.open();
this._log.appendLine(`Kallichore ${status.body.version} server reconnected with ${status.body.sessions} sessions`);

// Update the idle timeout from settings if we aren't in web mode
// (in web mode, no idle timeout is used)
if (vscode.env.uiKind !== vscode.UIKind.Web) {
this.updateIdleTimeout();
}

// Mark this a restored server
this._newSupervisor = false;

return true;
}

/**
* Update the idle timeout on the server. This is used to set the idle
* timeout on a server that has already started.
*/
async updateIdleTimeout() {
Comment on lines +654 to +658
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we remove the "Restart Positron to apply" guidance for the setting?

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to stay because there are certain transitions that do require a restart. In particular we store sessions differently when you are in "immediate" mode (ephemeral storage) vs. one of the other modes (durable storage), and there is no logic to move sessions between these storage classes. So you could change from e.g. 4 to 8 hours w/o a restart, but going from immediate to 4 does require a restart. I think this is probably too difficult to explain and we should just tell folks they always need to restart.

const timeout = this.getShutdownHours();
try {
await this._api.setServerConfiguration({
idleShutdownHours: timeout
});
} catch (err) {
this._log.appendLine(`Failed to update idle timeout: ${summarizeError(err)}`);
}
Comment on lines +660 to +666
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're hitting the error case when the idle timeout is set to "indefinitely" -1 -- is it intentional to error?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I forgot to update the supervisor side of this value! Thanks for catching that.

}

/**
* Start a long-running task that sends a heartbeat to the Kallichore server
* every 20 seconds. This is used to notify the server that we're connected,
Expand Down
5 changes: 5 additions & 0 deletions extensions/positron-supervisor/src/KallichoreSession.ts
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,10 @@ export class KallichoreSession implements JupyterLanguageRuntimeSession {
this._extra!.sleepOnStartup!.init(args, delay);
}

// Whether to run the kernel in a login shell. Kallichore ignores this
// on Windows.
const runInShell = config.get('runInShell', true);

// Create the session in the underlying API
const session: NewSession = {
argv: args,
Expand All @@ -421,6 +425,7 @@ export class KallichoreSession implements JupyterLanguageRuntimeSession {
continuationPrompt: '',
env: varActions,
workingDirectory: workingDir,
runInShell,
username: os.userInfo().username,
interruptMode,
connectionTimeout,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ model/models.ts
model/newSession.ts
model/newSession200Response.ts
model/restartSession.ts
model/serverConfiguration.ts
model/serverStatus.ts
model/sessionList.ts
model/startupError.ts
Expand Down
Loading