diff --git a/CHANGELOG.md b/CHANGELOG.md index 7559200..6f3c30d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,24 @@ All notable changes to kagglelink will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [1.2.0] - 2025-01-09 + +### Added +- **Environment variable fallback support** - Configure via `KAGGLELINK_KEYS_URL` and `KAGGLELINK_TOKEN` env vars as alternative to CLI flags (#14) +- **Unified logging system** - Consistent logging with emoji indicators (⏳ ✅ ❌), timestamps, elapsed time tracking, and error categorization (#15) +- **Success banner with zrok token** - Clear connection instructions displayed after setup with the zrok share token +- **Configuration source logging** - Shows whether config came from CLI args or environment variables +- **Save & Run All tip in README** - Guidance for avoiding Kaggle session timeouts + +### Changed +- **Shallow clone** (`--depth 1`) for faster repository setup (#16) +- **Improved error messages** - Error categorization (prerequisite, network, upstream) with actionable suggestions (#16) +- **Commit hash logging** - Shows git commit after successful clone for debugging (#16) + +### Fixed +- **gum color codes in Kaggle logs** - Removed ANSI escape codes from success banner for cleaner output in Kaggle's minimal log viewer (#17) +- **SSH command removed from banner** - Prevents confusion with host key warnings on ephemeral instances; SSH instructions now in README only (#17) +- **gum --yes flag** - Fixed interactive prompt during package installation for non-interactive environments ## [1.1.0] - 2025-12-07 diff --git a/README.md b/README.md index 9c11375..3e1e12c 100644 --- a/README.md +++ b/README.md @@ -4,93 +4,139 @@ A streamlined solution for accessing Kaggle computational resources via SSH and ## Overview -kagglelink allows you to ssh into Kaggle and leverage those kaggle resources, or you can run kaggles notebook remotely using VSCode, with more coding support, and better development environment +KaggleLink allows you to connect to Kaggle environments via SSH, enabling you to leverage Kaggle's computational resources -![Image](https://github.com/user-attachments/assets/db4454ff-5545-4094-adeb-47b74ab0c33a) +![](https://github.com/user-attachments/assets/db4454ff-5545-4094-adeb-47b74ab0c33a) -## Requirements +## Getting Started -1. A Zrok token is required for establishing the tunnel. Create an account at [myZrok.io](https://myzrok.io/) to get your token. +### Requirements -2. Ensure your account is on the Starter plan to utilize NetFoundry's public Zrok instance. +To use KaggleLink, you need: -3. You need to upload your public key to a github repository or a public file hosting service +1. **Zrok Token**: A Zrok token is essential for establishing the secure tunnel. Create an account at [myZrok.io](https://myzrok.io/) to obtain your token. Ensure your account is on the **Starter plan** to utilize NetFoundry's public Zrok instance, which offers 2 environment connections (one for your local machine, one for the Kaggle instance). +2. **Public SSH Key**: Your public SSH key needs to be accessible via a URL, either from a GitHub repository or another public file hosting service. -## Quick Setup +### Quick Setup (on Kaggle) -One line command setup? - -Paste this into Kaggle cell +Execute the following one-line command in a Kaggle notebook cell. This script will set up Zrok and SSH on your Kaggle instance. ```bash !curl -sS https://bhdai.github.io/setup | bash -s -- -k -t ``` > [!NOTE] -> -> replace with the URL of your public key file and with your Zrok token. +> Replace `` with the URL of your public SSH key file and `` with your Zrok token. +Wait for the setup to complete. You should see output similar to this upon successful configuration: -Wait for the setup to finish, you should see something like this at the end +![](https://github.com/user-attachments/assets/22f564f3-8622-4c6c-bb82-9c9c63dd322a) -![Image](https://github.com/user-attachments/assets/22f564f3-8622-4c6c-bb82-9c9c63dd322a) +> [!TIP] +> **Avoiding Session Timeouts**: Kaggle's interactive notebook sessions have idle timeouts. For long-running remote development, use the **"Save & Run All"** feature by clicking the **Save Version** button (top right) and selecting "Save". This runs your notebook as a background job, avoiding timeout interruptions. You can still get the zrok share token from the log screen(click active event at bottom left -> Open Logs in Viewer) -### How to setup public key? +#### How to set up your public SSH key? -Generate a new SSH key pair on your local machine (if you haven't already): +1. **Generate an SSH key pair** on your local machine (if you haven't already). Use a descriptive filename, for example: -```bash -ssh-keygen -t rsa -b 4096 -C "kaggle_remote_ssh" -f ~/.ssh/kaggle_rsa -``` + ```bash + ssh-keygen -t rsa -b 4096 -C "kaggle_remote_ssh" -f ~/.ssh/kaggle_rsa + ``` -Create a github repository and push the `~/.ssh/kaggle_rsa.pub` file to it. Make sure the repository is public. Once finished, you can get the public key URL by navigating to the file in your repository and clicking on the "Raw" button. +2. **Upload your public key** (`~/.ssh/kaggle_rsa.pub`) to a public GitHub repository or a similar public file hosting service. +3. **Obtain the Raw URL**: Navigate to your uploaded public key file in your repository and click the "Raw" button. -![Image](https://github.com/user-attachments/assets/ec9a884c-1c97-4be6-bd6d-03ac5dd16de7) + ![](https://private-user-images.githubusercontent.com/140616004/444039100-ec9a884c-1c97-4be6-bd6d-03ac5dd16de7.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NjU0NjQyMzMsIm5iZiI6MTc2NTQ2MzkzMywicGF0aCI6Ii8xNDA2MTYwMDQvNDQ0MDM5MTAwLWVjOWE4ODRjLTFjOTctNGJlNi1iZDZkLTAzYWM1ZGQxNmRlNy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUxMjExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MTIxMVQxNDM4NTNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT04YjZiY2M1OWRiMDUzYWZiMDUwODUzMjg2NDA4ZTU5NDAxZTM3YWU3ZGJmMDRlMjFiZjA0YmFmOGJlNTJmNzg1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.wDGsBk1CyVVAWFLSGh8wRldUbz2hiAOzw6t3Zf39K5A) -Copy the URL from your browser's address bar. It usually takes the form like this `https://raw.githubusercontent.com///refs/heads/main/` + Copy the URL from your browser's address bar. It typically looks like `https://raw.githubusercontent.com///refs/heads/main/`. -### How to get zrok token? +#### How to get your Zrok token? -Create your zrok account, if you haven't already, go [here](https://myzrok.io/billing) and change your plan to Starter plan, and then create a new token. Finally visit [https://api-v1.zrok.io](https://api-v1.zrok.io/), you should setup and get your token there +1. If you don't have one, create your Zrok account at [myZrok.io](https://myzrok.io/). +2. Go to the [billing page](https://myzrok.io/billing) and ensure your plan is set to **Starter**. +3. Create a new token. +4. Visit [https://api-v1.zrok.io](https://api-v1.zrok.io/) to retrieve and manage your Zrok tokens. -## Client Setup +### Advanced: Environment Variables -After completing the Kaggle setup, you'll receive a token. Follow these steps on your local machine: +For automated pipelines or power users, you can configure KaggleLink using environment variables instead of CLI flags. -1. Install Zrok locally by following the [official installation guide](https://docs.zrok.io/docs/guides/install/). +| Variable | CLI Equivalent | Description | +|----------|----------------|-------------| +| `KAGGLELINK_KEYS_URL` | `-k` | URL to your public SSH key | +| `KAGGLELINK_TOKEN` | `-t` | Your Zrok token | - For Arch-based distributions, you can use: - ```bash - yay -S zrok-bin - ``` +> [!NOTE] +> CLI arguments (`-k`, `-t`) always override environment variables if both are present. -2. Enable zrok in your local machine - ```bash - zrok enable - ``` +#### Setting Environment Variables in Kaggle -2. Access your Kaggle instance using the token: - ```bash - zrok access private - ``` +The most secure way to pass these credentials is using **Kaggle Secrets**. -3. This will open a dashboard displaying your connection details, including a local address like `127.0.0.1:9191`. +1. Add your secrets in the Kaggle notebook sidebar (**Add-ons** -> **Secrets**). +2. Use the following Python snippet in a cell *before* running the setup script: -## SSH Connection +```python +from kaggle_secrets import UserSecretsClient +import os -*For VSCode check out the [old instrunction](https://github.com/bhdai/kagglelink/blob/ngrok/README.md#connect-via-ssh) (will update this eventually)* +user_secrets = UserSecretsClient() + +# Set environment variables from secrets +# Ensure you have added 'KAGGLELINK_TOKEN' and 'KAGGLELINK_KEYS_URL' (optional) to your secrets +os.environ['KAGGLELINK_TOKEN'] = user_secrets.get_secret("KAGGLELINK_TOKEN") + +# You can also set the URL directly if it's public and not stored as a secret +os.environ['KAGGLELINK_KEYS_URL'] = "https://raw.githubusercontent.com/your/repo/main/key.pub" +``` -Connect to your Kaggle instance via SSH: +Once the environment variables are set, you can run the setup script without arguments: + +```bash +!curl -sS https://bhdai.github.io/setup | bash +``` + +## Usage + +After completing the Kaggle setup, your Kaggle instance is ready for connection. The script will output a Zrok private token at the end which you'll use to connect from your local machine. + +### Client Setup (on your Local Machine) + +1. **Install Zrok locally**: Follow the [official Zrok installation guide](https://docs.zrok.io/docs/guides/install/). + For Arch-based distributions, you can use: + + ```bash + yay -S zrok-bin + ``` + +2. **Enable Zrok**: Enable Zrok on your local machine using your personal Zrok token: + + ```bash + zrok enable + ``` + +3. **Access the private tunnel**: Use the Zrok `private_token` obtained from the Kaggle setup output to establish the connection: + + ```bash + zrok access private + ``` + + This command will open a dashboard in your terminal, displaying your connection details, including a local address like `127.0.0.1:9191`. + +### SSH Connection + +Connect to your Kaggle instance via SSH using the local address and port provided by Zrok (e.g., `127.0.0.1:9191`). ```bash ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/kaggle_rsa -p 9191 root@127.0.0.1 ``` -Note: The port (e.g., 9191) generally remains consistent across sessions, so no need to adjust it for each new instance. +> [!NOTE] +> The port (e.g., 9191) generally remains consistent across sessions, so you typically won't need to adjust it for each new instance. -### SSH Configuration +#### SSH Configuration -To simplify future connections, add this configuration to your `~/.ssh/config` file: +To simplify future connections, add the following configuration to your `~/.ssh/config` file: ``` Host Kaggle @@ -104,21 +150,36 @@ Host Kaggle With this configuration, you can simply use `ssh Kaggle` to connect. -## File Transfer with Rsync +### File Transfer with Rsync -Transfer files between your local machine and Kaggle instance: +Transfer files between your local machine and Kaggle instance using `rsync`: ```bash # From local to remote -rsync -e "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/kaggle_rsa -p 9191" root@127.0.0.1:/kaggle/working +rsync -e "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/kaggle_rsa -p 9191" root@127.0.0.1: +# or if you have you SSH config set up (see above) +rsync -avz Kaggle: # From remote to local rsync -e "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/kaggle_rsa -p 9191" root@127.0.0.1: +# or if you have you SSH config set up (see above) +rsync -avz Kaggle: ``` -> [!NOTE] -> -> If you're using the Starter plan, they only offer 2 environment connection on this plan one for you local machine, one for kaggle instance. While the script will automatically release the Kaggle instance when you turn off Kaggle, but it's best to check [https://api-v1.zrok.io/](https://api-v1.zrok.io/) to make sure your local machine is connected and there are no other active connections before running the script again. +> [!IMPORTANT] +> The Zrok Starter plan limits you to two environment connections. While the script automatically releases the Kaggle instance's connection upon shutdown, it's good practice to verify your active connections at [https://api-v1.zrok.io/](https://api-v1.zrok.io/) before rerunning the script, ensuring your local machine is the primary active connection. + +## Contributing + +We welcome contributions to KaggleLink! If you're interested in improving this project, please follow these steps: + +1. **Fork the repository**. +2. **Create a new branch** for your feature or bug fix (`git checkout -b feature/your-feature-name` or `bugfix/issue-description`). +3. **Make your changes**, adhering to the existing coding style and standards. +4. **Write and run tests** to ensure your changes work as expected and don't introduce regressions. +5. **Commit your changes** with clear and concise commit messages. +6. **Push your branch** to your forked repository. +7. **Open a Pull Request** to the main branch, providing a detailed description of your changes. ## License diff --git a/logging_utils.sh b/logging_utils.sh new file mode 100644 index 0000000..28fe9e0 --- /dev/null +++ b/logging_utils.sh @@ -0,0 +1,146 @@ +#!/bin/bash +# Shared logging utilities for kagglelink scripts +# +# This library provides consistent logging functions with emojis, +# timestamps, and error categorization for all kagglelink scripts. +# +# Usage: +# source logging_utils.sh +# log_info "Starting operation..." +# log_success "Operation completed" +# log_error "Something went wrong" + +# Store step start times for elapsed time calculation +declare -A _STEP_START_TIMES + +# Log an informational message with ⏳ emoji and timestamp +# Args: +# $1: Message to log +# Output: Formatted message to stdout +log_info() { + echo "⏳ [$(date +%H:%M:%S)] $1" +} + +# Log a success message with ✅ emoji and timestamp +# Args: +# $1: Message to log +# Output: Formatted message to stdout +log_success() { + echo "✅ [$(date +%H:%M:%S)] $1" +} + +# Log an error message with ❌ emoji and timestamp to stderr +# Args: +# $1: Message to log +# Output: Formatted error message to stderr +log_error() { + echo "❌ [$(date +%H:%M:%S)] ERROR: $1" >&2 +} + +# Start tracking a step for elapsed time calculation +# Args: +# $1: Step name +# Output: Informational log message +log_step_start() { + local step_name="$1" + _STEP_START_TIMES["$step_name"]=$(date +%s) + log_info "$step_name..." +} + +# Complete a step and display elapsed time +# Args: +# $1: Step name +# Output: Success message with elapsed time +log_step_complete() { + local step_name="$1" + local start_time="${_STEP_START_TIMES[$step_name]}" + if [ -n "$start_time" ]; then + local elapsed=$(($(date +%s) - start_time)) + log_success "$step_name completed (${elapsed}s)" + else + log_success "$step_name completed" + fi +} + +# Categorize and display error with contextual guidance +# Args: +# $1: Error type (prerequisite, network, upstream) +# $2: Error message +# $3: Suggested action +# Output: Formatted error with category-specific emoji and guidance to stderr +categorize_error() { + local error_type="$1" + local message="$2" + local suggestion="$3" + + case "$error_type" in + "prerequisite") + log_error "$message" + echo " 💡 Action required: $suggestion" >&2 + ;; + "network") + log_error "$message" + echo " 🌐 Check connectivity: $suggestion" >&2 + ;; + "upstream") + log_error "$message" + echo " 🔧 Upstream issue: $suggestion" >&2 + ;; + *) + log_error "$message" + ;; + esac +} + +# Display success banner with Zrok share token and connection instructions +# Args: +# $1: Zrok share token +# Output: Formatted success banner to stdout +show_success_banner() { + local share_token="$1" + + if command -v gum &>/dev/null; then + local header + header=$(gum style --border double --padding "1 2" --align center --width 60 "✅ Setup Complete!") + local message + message=$(gum style --align center --width 60 "Your Kaggle instance is ready for remote access!") + + local token_label + token_label=$(gum style "📡 Zrok Share Token:") + local token_value + token_value=$(gum style "$share_token") + local token_section + token_section=$(gum join --vertical --align center "$token_label" "$token_value") + local token_box + token_box=$(gum style --border rounded --padding "1 2" --width 60 --align center "$token_section") + + local instr_label + instr_label=$(gum style "🖥️ On your LOCAL machine, run:") + local cmd1 + cmd1=$(gum style "zrok access private $share_token") + + local cmds_content + cmds_content=$(gum join --vertical --align center "$instr_label" " " "$cmd1") + local cmds_box + cmds_box=$(gum style --border rounded --padding "1 2" --width 60 --align center "$cmds_content") + + printf "\n" + gum join --vertical --align center "$header" " " "$message" " " "$token_box" " " "$cmds_box" + else + echo "" + echo "╔════════════════════════════════════════════════════════════════╗" + echo "║ ✅ Setup Complete! ║" + echo "╠════════════════════════════════════════════════════════════════╣" + echo "║ ║" + echo "║ Your Kaggle instance is ready for remote access! ║" + echo "║ ║" + echo "║ 📡 Zrok Share Token: $share_token" + echo "║ ║" + echo "║ 🖥️ On your LOCAL machine, run: ║" + echo "║ ║" + echo "║ zrok access private $share_token" + echo "║ ║" + echo "╚════════════════════════════════════════════════════════════════╝" + echo "" + fi +} diff --git a/setup.sh b/setup.sh index 0edf999..ec903d0 100755 --- a/setup.sh +++ b/setup.sh @@ -2,25 +2,84 @@ set -e +# ============================================================================ +# Inline Logging Functions (embedded for bootstrap phase) +# ============================================================================ +# These are embedded directly in setup.sh because this script is downloaded +# standalone before the repository is cloned. Other scripts (setup_kaggle_zrok.sh, +# start_zrok.sh) source logging_utils.sh from the cloned repository. + +# Store step start times for elapsed time calculation +declare -A _STEP_START_TIMES + +log_info() { + echo "⏳ [$(date +%H:%M:%S)] $1" +} + +log_success() { + echo "✅ [$(date +%H:%M:%S)] $1" +} + +log_error() { + echo "❌ [$(date +%H:%M:%S)] ERROR: $1" >&2 +} + +log_step_start() { + local step_name="$1" + _STEP_START_TIMES["$step_name"]=$(date +%s) + log_info "$step_name..." +} + +log_step_complete() { + local step_name="$1" + local start_time="${_STEP_START_TIMES[$step_name]}" + if [ -n "$start_time" ]; then + local elapsed=$(($(date +%s) - start_time)) + log_success "$step_name completed (${elapsed}s)" + else + log_success "$step_name completed" + fi +} + +categorize_error() { + local error_type="$1" + local message="$2" + local suggestion="$3" + + case "$error_type" in + "prerequisite") + log_error "$message" + echo " 💡 Action required: $suggestion" >&2 + ;; + "network") + log_error "$message" + echo " 🌐 Check connectivity: $suggestion" >&2 + ;; + "upstream") + log_error "$message" + echo " 🔧 Upstream issue: $suggestion" >&2 + ;; + *) + log_error "$message" + ;; + esac +} +# ============================================================================ + # Version and branch configuration -KAGGLELINK_VERSION="1.1.0" +KAGGLELINK_VERSION="1.2.0" KAGGLELINK_BRANCH="${BRANCH:-main}" # Security: Validate KAGGLELINK_BRANCH to prevent argument injection # Branch names must not start with '-' to prevent git argument injection if [[ "$KAGGLELINK_BRANCH" =~ ^- ]]; then - echo "❌ Error: Invalid branch name '$KAGGLELINK_BRANCH'" - echo " Branch names cannot start with '-' (security: prevents argument injection)" + categorize_error "prerequisite" "Invalid branch name '$KAGGLELINK_BRANCH'" "Branch names cannot start with '-' (security: prevents argument injection)" exit 1 fi # Reliability: Check for git installation if ! command -v git &> /dev/null; then - echo "❌ Error: git is not installed" - echo " Please install git and try again" - echo " - Debian/Ubuntu: sudo apt-get install git" - echo " - RHEL/CentOS: sudo yum install git" - echo " - macOS: brew install git" + categorize_error "prerequisite" "git is not installed" "Install git: apt-get install git (Debian/Ubuntu), yum install git (RHEL/CentOS), or brew install git (macOS)" exit 1 fi @@ -45,93 +104,136 @@ usage() { echo " -t, --token TOKEN Your zrok token" echo " -h, --help Display this help message" echo "" - echo "Environment Variables:" + echo "Environment Variables (fallback when CLI flags not provided):" + echo " KAGGLELINK_KEYS_URL URL to your authorized_keys file" + echo " KAGGLELINK_TOKEN Your zrok token" echo " BRANCH Override default branch (current: ${KAGGLELINK_BRANCH})" exit "$exit_code" } # Parse command line arguments +# Initialize source tracking variables +AUTH_KEYS_SOURCE="" +ZROK_TOKEN_SOURCE="" + while [[ $# -gt 0 ]]; do case $1 in - -k | --keys-url) - AUTH_KEYS_URL="$2" - shift 2 - ;; - -t | --token) - ZROK_TOKEN="$2" - shift 2 - ;; - -h | --help) - usage 0 - ;; - *) - echo "Unknown option: $1" - usage - ;; + -k | --keys-url) + AUTH_KEYS_URL="$2" + AUTH_KEYS_SOURCE="CLI argument" + shift 2 + ;; + -t | --token) + ZROK_TOKEN="$2" + ZROK_TOKEN_SOURCE="CLI argument" + shift 2 + ;; + -h | --help) + usage 0 + ;; + *) + echo "Unknown option: $1" + usage + ;; esac done # Apply environment variable fallback if CLI args not provided if [ -z "$AUTH_KEYS_URL" ] && [ -n "$KAGGLELINK_KEYS_URL" ]; then AUTH_KEYS_URL="$KAGGLELINK_KEYS_URL" + AUTH_KEYS_SOURCE="KAGGLELINK_KEYS_URL env var" fi if [ -z "$ZROK_TOKEN" ] && [ -n "$KAGGLELINK_TOKEN" ]; then ZROK_TOKEN="$KAGGLELINK_TOKEN" + ZROK_TOKEN_SOURCE="KAGGLELINK_TOKEN env var" +fi + +# Log configuration source for transparency +if [ -n "$AUTH_KEYS_URL" ]; then + echo "ℹ️ Using keys URL from: $AUTH_KEYS_SOURCE" +fi +if [ -n "$ZROK_TOKEN" ]; then + echo "ℹ️ Using token from: $ZROK_TOKEN_SOURCE" fi # Check for required parameters if [ -z "$AUTH_KEYS_URL" ]; then - echo "Error: Public key URL (-k or --keys-url) is required" - usage + echo "Error: Public key URL is required" + echo " Provide via: -k or --keys-url " + echo " Or set: KAGGLELINK_KEYS_URL environment variable" + echo " Run with --help for more information" + exit 1 fi if [ -z "$ZROK_TOKEN" ]; then - echo "Error: zrok token (-t or --token) is required" - usage + echo "Error: zrok token is required" + echo " Provide via: -t or --token " + echo " Or set: KAGGLELINK_TOKEN environment variable" + echo " Run with --help for more information" + exit 1 fi # Validate that AUTH_KEYS_URL uses HTTPS (security requirement) if [[ ! "$AUTH_KEYS_URL" =~ ^https:// ]]; then - echo "❌ Error: Keys URL must use HTTPS (not HTTP)" - echo " Insecure URL: $AUTH_KEYS_URL" + categorize_error "prerequisite" "Keys URL must use HTTPS (not HTTP): $AUTH_KEYS_URL" "Use HTTPS URL instead" if [[ "$AUTH_KEYS_URL" =~ ^http:// ]]; then - echo " Use: ${AUTH_KEYS_URL/http:/https:}" - else - echo " URL must start with https://" + echo " Suggested: ${AUTH_KEYS_URL/http:/https:}" >&2 fi exit 1 fi -echo "⏳ Cloning repository..." +log_step_start "Cloning repository" if [ -d "$INSTALL_DIR" ]; then - echo "Repository directory already exists. Removing it..." + echo "⚠️ WARNING: specific directory already exists at $INSTALL_DIR" + echo " This script is designed to run on a FRESH Kaggle instance." + echo " Re-running on a used instance may cause issues." + echo " Continuing with cleanup..." + log_info "Repository directory already exists. Removing it..." rm -rf "$INSTALL_DIR" fi -if ! git clone -b "$KAGGLELINK_BRANCH" "$REPO_URL" "$INSTALL_DIR"; then - echo "❌ Error: Failed to clone branch '$KAGGLELINK_BRANCH'" - echo " Possible reasons:" - echo " - Branch does not exist" - echo " - Network connectivity issues" - echo " - GitHub is unreachable" +# Capture clone output for better error categorization +if clone_output=$(git clone --depth 1 -b "$KAGGLELINK_BRANCH" "$REPO_URL" "$INSTALL_DIR" 2>&1); then + clone_status=0 +else + clone_status=$? +fi + +if [ $clone_status -ne 0 ]; then + # Check if branch doesn't exist + if [[ "$clone_output" == *"Remote branch"*"not found"* ]] || [[ "$clone_output" == *"couldn't find remote ref"* ]]; then + categorize_error "prerequisite" \ + "Branch '$KAGGLELINK_BRANCH' does not exist in repository" \ + "Use BRANCH=main or check available branches at https://github.com/bhdai/kagglelink" + # Check for network issues + elif [[ "$clone_output" == *"Could not resolve host"* ]] || \ + [[ "$clone_output" == *"Connection refused"* ]] || \ + [[ "$clone_output" == *"Failed to connect"* ]]; then + categorize_error "network" \ + "Network connectivity issue during clone" \ + "Check internet connection and try again" + else + categorize_error "upstream" \ + "Failed to clone repository" \ + "GitHub may be temporarily unavailable or repository access restricted" + fi exit 1 fi -echo "✅ Cloned repository (branch: ${KAGGLELINK_BRANCH})" -echo "⏳ Changing to repository directory..." +# Log commit hash for debugging purposes cd "$INSTALL_DIR" +COMMIT_HASH=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") +log_success "Cloned repository (branch: ${KAGGLELINK_BRANCH}, commit: ${COMMIT_HASH})" +log_step_complete "Cloning repository" -echo "⏳ Making scripts executable..." +log_info "Making scripts executable..." chmod +x setup_kaggle_zrok.sh start_zrok.sh -echo "⏳ Setting up SSH with your public keys..." +log_step_start "Setting up SSH with your public keys" ./setup_kaggle_zrok.sh "$AUTH_KEYS_URL" +log_step_complete "Setting up SSH with your public keys" -echo "⏳ Starting zrok service with your token..." +log_info "Starting zrok service with your token..." +# Note: start_zrok.sh is a blocking process that will display success banner ./start_zrok.sh "$ZROK_TOKEN" - -echo "✅ Setup complete!" -echo "✅ You should now be able to connect to your Kaggle instance via SSH." -echo "✅ If you see a URL above, use that to connect from your local machine." -echo "✅ For more information, visit: https://github.com/bhdai/kagglelink" diff --git a/setup_kaggle_zrok.sh b/setup_kaggle_zrok.sh index 6694aa2..9f61331 100644 --- a/setup_kaggle_zrok.sh +++ b/setup_kaggle_zrok.sh @@ -2,6 +2,11 @@ set -e +# Source logging utilities +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# shellcheck source=logging_utils.sh +source "$SCRIPT_DIR/logging_utils.sh" + if [ "$#" -ne 1 ]; then echo "Usage: ./setup_kaggle_zrok.sh " exit 1 @@ -10,16 +15,16 @@ fi AUTH_KEYS_URL=$1 setup_ssh_directory() { - echo "Setting up SSH directory in user's home..." + log_info "Setting up SSH directory in user's home..." # If running as root, $HOME/.ssh becomes /root/.ssh local ssh_dir_path="$HOME/.ssh" mkdir -p "$ssh_dir_path" if wget -qO "$ssh_dir_path/authorized_keys" "$AUTH_KEYS_URL"; then chmod 700 "$ssh_dir_path" chmod 600 "$ssh_dir_path/authorized_keys" - echo "SSH directory and authorized_keys set up in $ssh_dir_path" + log_success "SSH directory and authorized_keys set up in $ssh_dir_path" else - echo "Failed to download authorized keys from $AUTH_KEYS_URL to $ssh_dir_path/authorized_keys." + categorize_error "network" "Failed to download authorized keys from $AUTH_KEYS_URL" "Check URL is accessible and internet connectivity" exit 1 fi } @@ -30,26 +35,26 @@ copy_vscode_dir() { [ -d "/kaggle/.vscode" ] && rm -rf "/kaggle/.vscode" mkdir -p "/kaggle/.vscode" cp -r "$vscode_dir_in_repo/"* "/kaggle/.vscode/" - echo ".vscode folder copied to /kaggle directory." + log_info ".vscode folder copied to /kaggle directory." mkdir -p "/kaggle/tmp" [ -d "/kaggle/working/.vscode" ] && rm -rf "/kaggle/working/.vscode" mkdir -p "/kaggle/working/.vscode" cp -r "$vscode_dir_in_repo/"* "/kaggle/working/.vscode/" - echo ".vscode folder copied to /kaggle/working directory." + log_info ".vscode folder copied to /kaggle/working directory." - echo "Contents of /kaggle/.vscode:" + log_info "Contents of /kaggle/.vscode:" ls -l "/kaggle/.vscode" - echo "Contents of /kaggle/working/.vscode:" + log_info "Contents of /kaggle/working/.vscode:" ls -l "/kaggle/working/.vscode" else - echo ".vscode directory not found in repository at $vscode_dir_in_repo." + log_error ".vscode directory not found in repository at $vscode_dir_in_repo." fi } configure_sshd() { mkdir -p /var/run/sshd - echo "Configuring sshd..." + log_info "Configuring sshd..." cat <>/etc/ssh/sshd_config Port 22 Protocol 2 @@ -73,22 +78,22 @@ ClientAliveInterval 60 ClientAliveCountMax 2 EOF echo "" >>/etc/ssh/sshd_config - echo "sshd_config updated. Note: Appended settings. Ensure no conflicting duplicates exist if run multiple times." + log_success "sshd_config updated. Note: Appended settings. Ensure no conflicting duplicates exist if run multiple times." - echo "Configuring debconf for non-interactive mode..." + log_info "Configuring debconf for non-interactive mode..." echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections - echo "debconf configured to use Noninteractive frontend." + log_success "debconf configured to use Noninteractive frontend." # Disable pam_systemd for container compatibility - echo "Disabling pam_systemd..." + log_info "Disabling pam_systemd..." sed -i 's/^session.*pam_systemd.so/#&/' /etc/pam.d/common-session # Disable man-db postinst to prevent crashes - echo "Disabling man-db postinst script..." + log_info "Disabling man-db postinst script..." dpkg-divert --quiet --local --rename --add /var/lib/dpkg/info/man-db.postinst ln -sf /bin/true /var/lib/dpkg/info/man-db.postinst - echo "Container compatibility fixes applied." + log_success "Container compatibility fixes applied." } setup_environment_variables() { @@ -142,40 +147,49 @@ EOT } install_packages() { - echo "Installing openssh-server..." + log_step_start "Installing packages" + + # Install gum + log_info "Installing gum..." + sudo mkdir -p /etc/apt/keyrings + curl -fsSL https://repo.charm.sh/apt/gpg.key | sudo gpg --dearmor --yes -o /etc/apt/keyrings/charm.gpg + echo "deb [signed-by=/etc/apt/keyrings/charm.gpg] https://repo.charm.sh/apt/ * *" | sudo tee /etc/apt/sources.list.d/charm.list + sudo apt-get update - sudo apt-get install -y openssh-server nvtop screen lshw + sudo apt-get install -y openssh-server nvtop screen lshw gum + log_step_complete "Installing packages" + log_info "Installing uv..." curl -LsSf https://astral.sh/uv/install.sh | sh } install_zrok() { - echo "Downloading latest zrok release" + log_step_start "Downloading and installing Zrok" curl -s https://api.github.com/repos/openziti/zrok/releases/latest | grep "browser_download_url.*linux_amd64.tar.gz" | cut -d : -f 2,3 | tr -d \" | wget -qi - - echo "Extracting Zrok" + log_info "Extracting Zrok" if ! tar -xzf zrok_*_linux_amd64.tar.gz -C /usr/local/bin/; then - echo "ERROR: Failed to extract Zrok" + categorize_error "network" "Failed to extract Zrok" "Check downloaded tar file integrity" exit 1 fi rm zrok_*_linux_amd64.tar.gz # check if zrok is installed correctly if ! zrok version &>/dev/null; then - echo "Error: Zrok install failed" + categorize_error "upstream" "Zrok install failed" "Try manual installation or check Zrok service" exit 1 fi - echo "Zrok installed successfully:" - zrok version + log_step_complete "Downloading and installing Zrok" + log_success "Zrok version: $(zrok version)" } setup_install_extensions_command() { - echo "Setting up 'install_extensions' command..." + log_info "Setting up 'install_extensions' command..." # SCRIPT_DIR will point to the directory where setup_kaggle_zrok.sh is located local SCRIPT_DIR SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) @@ -186,16 +200,15 @@ setup_install_extensions_command() { mkdir -p /usr/local/bin # Ensure target directory exists cp "$install_script_source" "$install_script_target" chmod +x "$install_script_target" - echo "'install_extensions' command is now available from $install_script_target." - echo "You can run 'install_extensions' in your terminal after SSHing." + log_success "'install_extensions' command is now available from $install_script_target." else - echo "Warning: $install_script_source not found. 'install_extensions' command not set up." + log_error "$install_script_source not found. 'install_extensions' command not set up." fi } start_ssh_service() { service ssh start - echo "SSH service should be running." + log_success "SSH service is running." } copy_screenrc() { @@ -203,9 +216,9 @@ copy_screenrc() { local dest="$HOME/.screenrc" if [ -f "$src" ]; then cp "$src" "$dest" - echo ".screenrc installed to $dest" + log_info ".screenrc installed to $dest" else - echo "Warning: $src not found; skipping .screenrc install." + log_error "$src not found; skipping .screenrc install." fi } @@ -222,4 +235,4 @@ copy_screenrc() { start_ssh_service ) -echo "Setup script completed. SSH service is running. Use start_zrok.sh to start zrok service." +log_success "Setup script completed. SSH service is running. Use start_zrok.sh to start zrok service." diff --git a/start_zrok.sh b/start_zrok.sh index 160754a..28f9329 100755 --- a/start_zrok.sh +++ b/start_zrok.sh @@ -2,6 +2,11 @@ set -e +# Source logging utilities +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# shellcheck source=logging_utils.sh +source "$SCRIPT_DIR/logging_utils.sh" + if [ "$#" -ne 1 ]; then echo "Usage: ./start_zrok.sh " exit 1 @@ -10,27 +15,86 @@ fi ZROK_TOKEN=$1 cleanup() { - echo "Disabling zrok environment..." + log_info "Disabling zrok environment..." zrok disable - echo "Cleanup complete." + log_success "Cleanup complete." } # trap the exit signal to run the cleanup function trap cleanup EXIT -echo "Starting zrok service..." +log_info "Starting zrok service..." if [ -z "$ZROK_TOKEN" ]; then - echo "Error: ZROK_TOKEN not provided." + categorize_error "prerequisite" "ZROK_TOKEN not provided" "Provide token via -t flag" exit 1 fi -echo "Enabling zrok with provided token..." -zrok enable "$ZROK_TOKEN" || { - echo "Failed to enable zrok with provided token." +log_step_start "Enabling zrok with provided token" +if ! zrok enable "$ZROK_TOKEN"; then + categorize_error "upstream" "Failed to enable zrok with provided token" "Verify token is valid or try again later" exit 1 -} +fi +log_step_complete "Enabling zrok with provided token" + +# CRITICAL: Start zrok share in background to capture token BEFORE blocking +log_info "Starting zrok tunnel (capturing share token)..." +SHARE_OUTPUT=$(mktemp) +SHARE_OUTPUT_RAW=$(mktemp) + +# Redirect all output to raw file for debugging +zrok share private --headless --backend-mode tcpTunnel localhost:22 > "$SHARE_OUTPUT_RAW" 2>&1 & +ZROK_PID=$! + +# Give zrok more time to establish tunnel and output token (increased from 2s to 8s) +sleep 8 + +# Poll for share token with timeout (max 60 seconds) +SHARE_TOKEN="" +for i in {1..60}; do + # Copy current output for parsing + cp "$SHARE_OUTPUT_RAW" "$SHARE_OUTPUT" 2>/dev/null || true + + # Try multiple regex patterns to find the token + # Pattern 1: JSON format (non-TTY) - look inside "msg" field for "zrok access private TOKEN" + SHARE_TOKEN=$(grep -oP '"msg":"[^"]*zrok access private \K[a-zA-Z0-9]+' "$SHARE_OUTPUT" 2>/dev/null || true) + + # Pattern 2: Plain text format (TTY) - look for "zrok access private TOKEN" + if [ -z "$SHARE_TOKEN" ]; then + SHARE_TOKEN=$(grep -oP 'zrok access private \K[a-zA-Z0-9]+' "$SHARE_OUTPUT" 2>/dev/null || true) + fi + + # Pattern 3: Look for token on line containing "allow other to access" + if [ -z "$SHARE_TOKEN" ]; then + SHARE_TOKEN=$(grep "allow other to access" "$SHARE_OUTPUT" 2>/dev/null | grep -oP 'zrok access private \K[a-zA-Z0-9]+' || true) + fi + + if [ -n "$SHARE_TOKEN" ]; then + log_success "Token captured: $SHARE_TOKEN (attempt $i)" + break + fi + + # Debug output every 10 seconds + if [ $((i % 10)) -eq 0 ]; then + log_info "Still waiting for token... (${i}s elapsed)" + fi + + sleep 1 +done + +# Clean up temp files +rm -f "$SHARE_OUTPUT" "$SHARE_OUTPUT_RAW" + +if [ -z "$SHARE_TOKEN" ]; then + categorize_error "upstream" "Failed to capture share token within timeout" "Check Zrok service status and logs" + kill $ZROK_PID 2>/dev/null || true + exit 1 +fi + +# Display success banner NOW (before blocking on tunnel) +show_success_banner "$SHARE_TOKEN" -echo "Starting zrok share in headless mode..." -echo "Starting zrok share now..." -zrok share private --headless --backend-mode tcpTunnel localhost:22 +# Keep tunnel alive - wait on background process (blocks here) +log_info "Tunnel is active. Keeping connection alive..." +log_info "Press Ctrl+C to stop the tunnel and clean up." +wait $ZROK_PID diff --git a/tests/unit/test_argument_parsing.bats b/tests/unit/test_argument_parsing.bats index 91a7cc1..3a242d4 100644 --- a/tests/unit/test_argument_parsing.bats +++ b/tests/unit/test_argument_parsing.bats @@ -9,6 +9,7 @@ load '../test_helper/common' setup() { create_test_dir # Copy setup.sh to temp dir for isolated testing + # Note: setup.sh has inline logging functions, no need to copy logging_utils.sh cp "${PROJECT_ROOT}/setup.sh" "${TEST_TEMP_DIR}/" cd "${TEST_TEMP_DIR}" } diff --git a/tests/unit/test_clone_recovery.bats b/tests/unit/test_clone_recovery.bats new file mode 100644 index 0000000..a5c9f48 --- /dev/null +++ b/tests/unit/test_clone_recovery.bats @@ -0,0 +1,277 @@ +#!/usr/bin/env bats +# Test suite for repository clone with graceful recovery (Story 1.4) +# +# Tests: +# - Existing directory removal before clone +# - Git prerequisite check +# - Commit hash logging +# - Network error categorization +# - Branch not found error handling + +load '../test_helper/common' + +setup() { + # Store original directory + export ORIGINAL_DIR="$PWD" + + # Create isolated test environment + export TEST_TEMP_DIR="$(mktemp -d)" + export HOME="$TEST_TEMP_DIR/home" + mkdir -p "$HOME" + + # Store original PATH + export ORIGINAL_PATH="$PATH" + + # Export PROJECT_ROOT for tests + export PROJECT_ROOT="${BATS_TEST_DIRNAME}/../.." +} + +teardown() { + # Restore PATH + export PATH="$ORIGINAL_PATH" + + # Clean up test environment + if [ -n "$TEST_TEMP_DIR" ] && [ -d "$TEST_TEMP_DIR" ]; then + rm -rf "$TEST_TEMP_DIR" + fi + + # Return to original directory + cd "$ORIGINAL_DIR" +} + +# ============================================================================ +# AC3: Git Prerequisite Check +# ============================================================================ + +@test "P0: should detect missing git command" { + # Verify git check exists in source code (functional verification) + run grep -A 2 'if ! command -v git' "$PROJECT_ROOT/setup.sh" + + [ "$status" -eq 0 ] + [[ "$output" == *"git is not installed"* ]] + + # Verify error categorization is used + run grep 'categorize_error "prerequisite".*git' "$PROJECT_ROOT/setup.sh" + [ "$status" -eq 0 ] +} + +# ============================================================================ +# AC1: Handle Existing Directory +# ============================================================================ + +@test "P0: should remove existing /tmp/kagglelink before clone" { + # Create mock git that simulates clone behavior + mkdir -p "$TEST_TEMP_DIR/bin" + + # Mock git clone that creates the directory + cat > "$TEST_TEMP_DIR/bin/git" <<'MOCKGIT' +#!/bin/sh +if [ "$1" = "clone" ]; then + # Extract target directory (last argument) + for last; do true; done + mkdir -p "$last/.git" + printf '#!/bin/sh\necho "abc123"' > "$last/.git/rev-parse" + chmod +x "$last/.git/rev-parse" + exit 0 +elif [ "$1" = "rev-parse" ]; then + echo "abc123" + exit 0 +fi +exit 1 +MOCKGIT + chmod +x "$TEST_TEMP_DIR/bin/git" + + # Pre-create directory with marker file + mkdir -p /tmp/kagglelink + touch /tmp/kagglelink/pre_existing_marker + + # Run with mocked git + export PATH="$TEST_TEMP_DIR/bin:$ORIGINAL_PATH" + run bash "$PROJECT_ROOT/setup.sh" -k "https://example.com/keys" -t "test-token" + + # Verify marker file is gone (directory was recreated) + [ ! -f /tmp/kagglelink/pre_existing_marker ] +} + +# ============================================================================ +# AC4: Clone Success Verification - Commit Hash Logging +# ============================================================================ + +@test "P1: should log commit hash after successful clone" { + # Create mock git that returns commit hash + mkdir -p "$TEST_TEMP_DIR/bin" + + cat > "$TEST_TEMP_DIR/bin/git" <<'MOCKGIT' +#!/bin/sh +if [ "$1" = "clone" ]; then + for last; do true; done + mkdir -p "$last" + exit 0 +elif [ "$1" = "rev-parse" ]; then + echo "abc1234" + exit 0 +fi +exit 1 +MOCKGIT + chmod +x "$TEST_TEMP_DIR/bin/git" + + export PATH="$TEST_TEMP_DIR/bin:$ORIGINAL_PATH" + run bash "$PROJECT_ROOT/setup.sh" -k "https://example.com/keys" -t "test-token" + + # Verify commit hash appears in output + [[ "$output" =~ commit:\ abc1234 ]] +} + +@test "P1: should handle git rev-parse failure gracefully" { + # Create mock git where rev-parse fails + mkdir -p "$TEST_TEMP_DIR/bin" + + cat > "$TEST_TEMP_DIR/bin/git" <<'MOCKGIT' +#!/bin/sh +if [ "$1" = "clone" ]; then + for last; do true; done + mkdir -p "$last" + exit 0 +elif [ "$1" = "rev-parse" ]; then + exit 1 +fi +exit 1 +MOCKGIT + chmod +x "$TEST_TEMP_DIR/bin/git" + + export PATH="$TEST_TEMP_DIR/bin:$ORIGINAL_PATH" + run bash "$PROJECT_ROOT/setup.sh" -k "https://example.com/keys" -t "test-token" + + # Should fallback to "unknown" + [[ "$output" =~ commit:\ unknown ]] +} + +# ============================================================================ +# AC2: Network Failure Handling - Error Categorization +# ============================================================================ + +@test "P1: should categorize network connectivity errors" { + # Mock git to simulate network error + mkdir -p "$TEST_TEMP_DIR/bin" + cat > "$TEST_TEMP_DIR/bin/git" <<'MOCKGIT' +#!/bin/sh +echo "fatal: Could not resolve host: github.com" >&2 +exit 128 +MOCKGIT + chmod +x "$TEST_TEMP_DIR/bin/git" + + export PATH="$TEST_TEMP_DIR/bin:$ORIGINAL_PATH" + run bash "$PROJECT_ROOT/setup.sh" -k "https://example.com/keys" -t "test-token" + + [ "$status" -eq 1 ] + # Check that our categorize_error logic caught it + [[ "$output" =~ "Network connectivity issue" ]] + [[ "$output" =~ "Check connectivity" ]] +} + +@test "P1: should categorize branch not found errors" { + # Mock git to simulate branch error + mkdir -p "$TEST_TEMP_DIR/bin" + cat > "$TEST_TEMP_DIR/bin/git" <<'MOCKGIT' +#!/bin/sh +echo "fatal: Remote branch feature/missing not found in upstream origin" >&2 +exit 128 +MOCKGIT + chmod +x "$TEST_TEMP_DIR/bin/git" + + export PATH="$TEST_TEMP_DIR/bin:$ORIGINAL_PATH" + run bash "$PROJECT_ROOT/setup.sh" -k "https://example.com/keys" -t "test-token" + + [ "$status" -eq 1 ] + [[ "$output" =~ "does not exist" ]] + [[ "$output" =~ "Use BRANCH=main" ]] +} + +@test "P1: should categorize upstream errors for other failures" { + # Mock git to simulate generic failure + mkdir -p "$TEST_TEMP_DIR/bin" + cat > "$TEST_TEMP_DIR/bin/git" <<'MOCKGIT' +#!/bin/sh +echo "fatal: something went wrong on github side" >&2 +exit 128 +MOCKGIT + chmod +x "$TEST_TEMP_DIR/bin/git" + + export PATH="$TEST_TEMP_DIR/bin:$ORIGINAL_PATH" + run bash "$PROJECT_ROOT/setup.sh" -k "https://example.com/keys" -t "test-token" + + [ "$status" -eq 1 ] + [[ "$output" =~ "Failed to clone repository" ]] + [[ "$output" =~ "GitHub may be temporarily unavailable" ]] +} + +# ============================================================================ +# AC5: Shallow Clone Optimization +# ============================================================================ + +@test "P1: should use shallow clone with --depth 1" { + # Create mock git that captures and validates arguments + mkdir -p "$TEST_TEMP_DIR/bin" + + cat > "$TEST_TEMP_DIR/bin/git" <<'MOCKGIT' +#!/bin/sh +# Log all git commands to a file +echo "$*" >> /tmp/git_commands.log + +if [ "$1" = "clone" ]; then + # Check if --depth 1 is present + if ! echo "$*" | grep -q "\-\-depth 1"; then + echo "ERROR: --depth 1 not found in git clone command" >&2 + exit 99 + fi + for last; do true; done + mkdir -p "$last" + exit 0 +elif [ "$1" = "rev-parse" ]; then + echo "abc1234" + exit 0 +fi +exit 1 +MOCKGIT + chmod +x "$TEST_TEMP_DIR/bin/git" + + # Clear log file + rm -f /tmp/git_commands.log + + export PATH="$TEST_TEMP_DIR/bin:$ORIGINAL_PATH" + run bash "$PROJECT_ROOT/setup.sh" -k "https://example.com/keys" -t "test-token" + + # Should succeed (exit 99 would indicate --depth 1 was missing) + [ "$status" -ne 99 ] + + # Verify git clone was called with --depth 1 + [ -f /tmp/git_commands.log ] + run cat /tmp/git_commands.log + [[ "$output" == *"--depth 1"* ]] + + # Cleanup + rm -f /tmp/git_commands.log +} + +# ============================================================================ +# Edge Cases and Regression Prevention +# ============================================================================ + +@test "P2: setup.sh should still have inline logging functions" { + # Verify logging functions are still embedded (bootstrap requirement) + run grep -n "^log_info()" "$PROJECT_ROOT/setup.sh" + [ "$status" -eq 0 ] + + run grep -n "^log_success()" "$PROJECT_ROOT/setup.sh" + [ "$status" -eq 0 ] + + run grep -n "^categorize_error()" "$PROJECT_ROOT/setup.sh" + [ "$status" -eq 0 ] +} + +@test "P2: should handle empty INSTALL_DIR variable gracefully" { + # Test that script validates INSTALL_DIR exists + run grep -n 'if \[ -d "\$INSTALL_DIR" \]' "$PROJECT_ROOT/setup.sh" + [ "$status" -eq 0 ] +} + diff --git a/tests/unit/test_env_fallback.bats b/tests/unit/test_env_fallback.bats index 04bd8f5..d496224 100755 --- a/tests/unit/test_env_fallback.bats +++ b/tests/unit/test_env_fallback.bats @@ -19,11 +19,12 @@ if [[ "$*" == *"clone"* ]]; then # Extract the target directory (last argument) target="${@: -1}" mkdir -p "$target" - mkdir -p "$target" echo '#!/bin/bash' > "$target/setup_kaggle_zrok.sh" echo '#!/bin/bash' > "$target/start_zrok.sh" chmod +x "$target/setup_kaggle_zrok.sh" "$target/start_zrok.sh" + exit 0 fi +# For any other git command, just succeed exit 0 EOF chmod +x "$TEST_TEMP_DIR/git" @@ -74,7 +75,9 @@ teardown() { run bash "${PROJECT_ROOT}/setup.sh" -k "https://cli.com/keys" -t "cli-token" [ "$status" -eq 0 ] - # The CLI values should be used (verified by checking they were passed to scripts) + # Verify CLI values are actually used by checking source logging + [[ "$output" == *"Using keys URL from: CLI argument"* ]] + [[ "$output" == *"Using token from: CLI argument"* ]] } @test "P0: should fail when both CLI and env are missing (keys URL)" { @@ -102,3 +105,72 @@ teardown() { # Check for actionable error message [[ "$output" == *"Error"* ]] || [[ "$output" == *"required"* ]] } + +# ============================================================================= +# Configuration Source Logging Tests (AC4) +# ============================================================================= + +@test "P0: should log CLI source when -k provided" { + run bash "${PROJECT_ROOT}/setup.sh" -k "https://example.com/keys" -t "test-token" + [ "$status" -eq 0 ] + [[ "$output" == *"Using keys URL from: CLI argument"* ]] +} + +@test "P0: should log CLI source when -t provided" { + run bash "${PROJECT_ROOT}/setup.sh" -k "https://example.com/keys" -t "test-token" + [ "$status" -eq 0 ] + [[ "$output" == *"Using token from: CLI argument"* ]] +} + +@test "P0: should log env var source when KAGGLELINK_KEYS_URL used" { + export KAGGLELINK_KEYS_URL="https://example.com/keys" + export KAGGLELINK_TOKEN="test-token" + + run bash "${PROJECT_ROOT}/setup.sh" + [ "$status" -eq 0 ] + [[ "$output" == *"Using keys URL from: KAGGLELINK_KEYS_URL env var"* ]] +} + +@test "P0: should log env var source when KAGGLELINK_TOKEN used" { + export KAGGLELINK_KEYS_URL="https://example.com/keys" + export KAGGLELINK_TOKEN="test-token" + + run bash "${PROJECT_ROOT}/setup.sh" + [ "$status" -eq 0 ] + [[ "$output" == *"Using token from: KAGGLELINK_TOKEN env var"* ]] +} + +@test "P0: should log CLI source when both CLI and env var provided" { + export KAGGLELINK_KEYS_URL="https://env.com/keys" + export KAGGLELINK_TOKEN="env-token" + + run bash "${PROJECT_ROOT}/setup.sh" -k "https://cli.com/keys" -t "cli-token" + [ "$status" -eq 0 ] + [[ "$output" == *"Using keys URL from: CLI argument"* ]] + [[ "$output" == *"Using token from: CLI argument"* ]] +} + +# ============================================================================= +# Improved Error Messages Tests (AC5) +# ============================================================================= + +@test "P0: error message mentions both -k flag AND KAGGLELINK_KEYS_URL env var" { + run env -u KAGGLELINK_KEYS_URL KAGGLELINK_TOKEN="test-token" PATH="$TEST_TEMP_DIR:$PATH" bash "${PROJECT_ROOT}/setup.sh" + [ "$status" -ne 0 ] + [[ "$output" == *"-k"* ]] || [[ "$output" == *"--keys-url"* ]] + [[ "$output" == *"KAGGLELINK_KEYS_URL"* ]] +} + +@test "P0: error message mentions both -t flag AND KAGGLELINK_TOKEN env var" { + run env -u KAGGLELINK_TOKEN KAGGLELINK_KEYS_URL="https://example.com/keys" PATH="$TEST_TEMP_DIR:$PATH" bash "${PROJECT_ROOT}/setup.sh" + [ "$status" -ne 0 ] + [[ "$output" == *"-t"* ]] || [[ "$output" == *"--token"* ]] + [[ "$output" == *"KAGGLELINK_TOKEN"* ]] +} + +@test "P0: usage output includes environment variable documentation" { + run bash "${PROJECT_ROOT}/setup.sh" -h + [ "$status" -eq 0 ] + [[ "$output" == *"KAGGLELINK_KEYS_URL"* ]] + [[ "$output" == *"KAGGLELINK_TOKEN"* ]] +} diff --git a/tests/unit/test_logging.bats b/tests/unit/test_logging.bats new file mode 100644 index 0000000..adef752 --- /dev/null +++ b/tests/unit/test_logging.bats @@ -0,0 +1,124 @@ +#!/usr/bin/env bats + +# Unit Tests for Logging Utilities +# Story: 1-3-unified-logging-and-user-feedback-system +# Tests AC1-3 and AC5 + +load '../test_helper/common.bash' + +setup() { + # Create temporary test directory + TEST_TEMP_DIR="$(mktemp -d)" + + # Set PROJECT_ROOT for sourcing + PROJECT_ROOT="${BATS_TEST_DIRNAME}/../.." +} + +teardown() { + # Clean up temporary directory + rm -rf "$TEST_TEMP_DIR" +} + +# Task 7.1: Test log_info outputs ⏳ emoji and timestamp +@test "P0: log_info should include ⏳ emoji and timestamp" { + source "${PROJECT_ROOT}/logging_utils.sh" + + run log_info "Test message" + + [ "$status" -eq 0 ] + [[ "$output" == *"⏳"* ]] + [[ "$output" =~ \[[0-9]{2}:[0-9]{2}:[0-9]{2}\] ]] + [[ "$output" == *"Test message"* ]] +} + +# Task 7.2: Test log_success outputs ✅ emoji +@test "P0: log_success should include ✅ emoji and timestamp" { + source "${PROJECT_ROOT}/logging_utils.sh" + + run log_success "Operation completed" + + [ "$status" -eq 0 ] + [[ "$output" == *"✅"* ]] + [[ "$output" =~ \[[0-9]{2}:[0-9]{2}:[0-9]{2}\] ]] + [[ "$output" == *"Operation completed"* ]] +} + +# Task 7.3: Test log_error outputs ❌ emoji to stderr +@test "P0: log_error should output ❌ emoji to stderr" { + source "${PROJECT_ROOT}/logging_utils.sh" + + run bash -c "source ${PROJECT_ROOT}/logging_utils.sh; log_error 'Error occurred' 2>&1 1>/dev/null" + + [ "$status" -eq 0 ] + [[ "$output" == *"❌"* ]] + [[ "$output" == *"ERROR:"* ]] + [[ "$output" == *"Error occurred"* ]] +} + +# Task 7.4: Test elapsed time calculation +@test "P0: log_step_start and log_step_complete should calculate elapsed time" { + # Run in a bash subshell to maintain context between start and complete + run bash -c " + source ${PROJECT_ROOT}/logging_utils.sh + log_step_start 'Test Step' > /dev/null + sleep 1 + log_step_complete 'Test Step' + " + + [ "$status" -eq 0 ] + [[ "$output" == *"✅"* ]] + [[ "$output" == *"Test Step completed"* ]] + [[ "$output" =~ \([0-9]+s\) ]] +} + +# Test categorize_error function with different types +@test "P0: categorize_error should format prerequisite errors" { + source "${PROJECT_ROOT}/logging_utils.sh" + + run bash -c "source ${PROJECT_ROOT}/logging_utils.sh; categorize_error 'prerequisite' 'git is not installed' 'Install git: apt-get install git' 2>&1" + + [ "$status" -eq 0 ] + [[ "$output" == *"❌"* ]] + [[ "$output" == *"git is not installed"* ]] + [[ "$output" == *"💡 Action required:"* ]] + [[ "$output" == *"Install git: apt-get install git"* ]] +} + +@test "P0: categorize_error should format network errors" { + source "${PROJECT_ROOT}/logging_utils.sh" + + run bash -c "source ${PROJECT_ROOT}/logging_utils.sh; categorize_error 'network' 'Failed to download keys' 'Check URL is accessible' 2>&1" + + [ "$status" -eq 0 ] + [[ "$output" == *"❌"* ]] + [[ "$output" == *"Failed to download keys"* ]] + [[ "$output" == *"🌐 Check connectivity:"* ]] + [[ "$output" == *"Check URL is accessible"* ]] +} + +@test "P0: categorize_error should format upstream errors" { + source "${PROJECT_ROOT}/logging_utils.sh" + + run bash -c "source ${PROJECT_ROOT}/logging_utils.sh; categorize_error 'upstream' 'Zrok API failed' 'Try again later' 2>&1" + + [ "$status" -eq 0 ] + [[ "$output" == *"❌"* ]] + [[ "$output" == *"Zrok API failed"* ]] + [[ "$output" == *"🔧 Upstream issue:"* ]] + [[ "$output" == *"Try again later"* ]] +} + +# Task 7.5: Test success banner format +@test "P0: show_success_banner should display formatted banner with token" { + source "${PROJECT_ROOT}/logging_utils.sh" + + test_token="abc123xyz" + run show_success_banner "$test_token" + + [ "$status" -eq 0 ] + [[ "$output" == *"✅ Setup Complete!"* ]] + [[ "$output" == *"$test_token"* ]] + [[ "$output" == *"zrok access private $test_token"* ]] + [[ "$output" == *"╔"* ]] # Box drawing characters + [[ "$output" == *"╚"* ]] +} diff --git a/tests/unit/test_url_validation.bats b/tests/unit/test_url_validation.bats index afed019..c0297e1 100755 --- a/tests/unit/test_url_validation.bats +++ b/tests/unit/test_url_validation.bats @@ -18,10 +18,18 @@ setup() { if [[ "$*" == *"clone"* ]]; then target="${@: -1}" mkdir -p "$target" + # Copy logging_utils.sh to mocked repo directory + cp /workspace/logging_utils.sh "$target/" 2>/dev/null || true echo '#!/bin/bash' > "$target/setup_kaggle_zrok.sh" + echo 'source "$(dirname "$0")/logging_utils.sh" 2>/dev/null || true' >> "$target/setup_kaggle_zrok.sh" + echo 'exit 0' >> "$target/setup_kaggle_zrok.sh" echo '#!/bin/bash' > "$target/start_zrok.sh" + echo 'source "$(dirname "$0")/logging_utils.sh" 2>/dev/null || true' >> "$target/start_zrok.sh" + echo 'exit 0' >> "$target/start_zrok.sh" chmod +x "$target/setup_kaggle_zrok.sh" "$target/start_zrok.sh" + exit 0 fi +# For any other git command, just succeed exit 0 EOF chmod +x "$TEST_TEMP_DIR/git" @@ -75,8 +83,8 @@ teardown() { @test "P1: error message should explain why HTTP is rejected" { run bash "${PROJECT_ROOT}/setup.sh" -k "http://example.com/keys" -t "test-token" [ "$status" -ne 0 ] - # Should have actionable error message - [[ "$output" == *"Error"* ]] + # Should have actionable error message (case-insensitive check for ERROR or Error) + [[ "$output" =~ [Ee][Rr][Rr][Oo][Rr] ]] [[ "$output" == *"HTTPS"* ]] || [[ "$output" == *"secure"* ]] } diff --git a/tests/unit/test_version_tracking.bats b/tests/unit/test_version_tracking.bats index ac0b310..bdfd7ef 100644 --- a/tests/unit/test_version_tracking.bats +++ b/tests/unit/test_version_tracking.bats @@ -53,19 +53,19 @@ teardown() { run bash -c 'unset BRANCH && bash setup.sh -h 2>&1 | head -5' # Verify version and branch appear in output (usage exits with 1, which is expected) - [[ "$output" =~ "Version: 1.1.0 (branch: main)" ]] + [[ "$output" =~ "Version: 1.2.0 (branch: main)" ]] } @test "setup.sh displays custom branch when BRANCH env var is set" { run bash -c 'export BRANCH=develop && bash setup.sh -h 2>&1 | head -5' - [[ "$output" =~ "Version: 1.1.0 (branch: develop)" ]] + [[ "$output" =~ "Version: 1.2.0 (branch: develop)" ]] } @test "setup.sh displays feature branch when BRANCH env var is custom" { run bash -c 'export BRANCH=feature/test && bash setup.sh -h 2>&1 | head -5' - [[ "$output" =~ "Version: 1.1.0 (branch: feature/test)" ]] + [[ "$output" =~ "Version: 1.2.0 (branch: feature/test)" ]] } @test "setup.sh uses default main branch for git clone" { @@ -74,7 +74,7 @@ teardown() { #!/bin/bash set -e # Extract just the version and clone logic from setup.sh -KAGGLELINK_VERSION="1.1.0" +KAGGLELINK_VERSION="1.2.0" KAGGLELINK_BRANCH="${BRANCH:-main}" REPO_URL="https://github.com/bhdai/kagglelink.git" INSTALL_DIR="/tmp/kagglelink-test-$$" @@ -102,7 +102,7 @@ WRAPPER cat > "$TEST_TEMP_DIR/test_wrapper.sh" << 'WRAPPER' #!/bin/bash set -e -KAGGLELINK_VERSION="1.1.0" +KAGGLELINK_VERSION="1.2.0" KAGGLELINK_BRANCH="${BRANCH:-main}" REPO_URL="https://github.com/bhdai/kagglelink.git" INSTALL_DIR="/tmp/kagglelink-test-$$" @@ -127,7 +127,7 @@ WRAPPER cat > "$TEST_TEMP_DIR/test_wrapper.sh" << 'WRAPPER' #!/bin/bash set -e -KAGGLELINK_VERSION="1.1.0" +KAGGLELINK_VERSION="1.2.0" KAGGLELINK_BRANCH="${BRANCH:-main}" REPO_URL="https://github.com/bhdai/kagglelink.git" INSTALL_DIR="/tmp/kagglelink-test-$$" @@ -178,7 +178,7 @@ WRAPPER @test "setup.sh accepts valid branch names with dashes in middle" { # Dashes in the middle are fine, just not at the start run bash -c 'export BRANCH="feature-test" && bash setup.sh -h 2>&1 | head -5' - [[ "$output" =~ "Version: 1.1.0 (branch: feature-test)" ]] + [[ "$output" =~ "Version: 1.2.0 (branch: feature-test)" ]] } @test "setup.sh checks for git installation" { @@ -187,7 +187,7 @@ WRAPPER #!/bin/bash set -e export PATH="/usr/bin:/bin" # Minimal PATH without git location -KAGGLELINK_VERSION="1.1.0" +KAGGLELINK_VERSION="1.2.0" KAGGLELINK_BRANCH="${BRANCH:-main}" # Check for git installation