diff --git a/README.md b/README.md index 39d4ec4..d4c9131 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ This library is designed to be integrated into robot data collection systems, ET - 🤖 **Teleoperation Session Management**: Track robot data collection sessions with `TeleopSession` - 🔄 **Data Conversion Session Management**: Track data conversion pipelines with `ConversionSession` -- ☁️ **Wasabi Upload Management**: Track Wasabi cloud storage uploads with `WasabiUploadSession` +- ☁️ **S3 Upload Management**: Track S3 object storage uploads with `S3UploadSession` - 🔌 **USB Data Copy Management**: Track USB data copy operations with `USBCopySession` - 📊 **OpenLineage Integration**: Full OpenLineage specification support (START/COMPLETE/RUNNING/FAIL/ABORT) - 🏷️ **Custom Run Facets**: Robot metadata (robotId, location, repository info) via `CommonRunFacet` @@ -227,13 +227,13 @@ uv run python examples/teleop_session.py Demonstrates a complete teleoperation session with nominal time tracking. -### Wasabi Upload Session +### S3 Upload Session ```bash -uv run python examples/wasabi_upload_session.py +uv run python examples/s3_upload_session.py ``` -Demonstrates a complete Wasabi cloud storage upload session with nominal time tracking. +Demonstrates a complete S3 object storage upload session with nominal time tracking. ### Batch ETL Job diff --git a/docs/architecture.md b/docs/architecture.md index 8641c02..98e5a66 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -93,27 +93,27 @@ ConversionSession.complete(output_datasets) # Emits COMPLETE event **API Documentation**: See docstrings in [session.py](../src/airoa_lineage/conversion/session.py) **Usage Example**: [examples/data_conversion.py](../examples/data_conversion.py) -### WasabiUploadSession +### S3UploadSession -**Location**: [src/airoa_lineage/wasabi_upload/session.py](../src/airoa_lineage/wasabi_upload/session.py) +**Location**: [src/airoa_lineage/s3_upload/session.py](../src/airoa_lineage/s3_upload/session.py) -**Purpose**: Manages the lifecycle of Wasabi cloud storage upload operations with OpenLineage tracking. +**Purpose**: Manages the lifecycle of S3 object storage upload operations with OpenLineage tracking. **Key Responsibilities**: -- Track Wasabi data upload sessions (START → COMPLETE) +- Track S3 data upload sessions (START → COMPLETE) - Automatic run_id generation and management - Nominal time support for historical data processing - State validation (prevents duplicate start/complete calls) **Lifecycle**: ``` -WasabiUploadSession.start() # Emits START event - └─> Data upload to Wasabi... -WasabiUploadSession.complete() # Emits COMPLETE event +S3UploadSession.start() # Emits START event + └─> Data upload to S3... +S3UploadSession.complete() # Emits COMPLETE event ``` -**API Documentation**: See docstrings in [session.py](../src/airoa_lineage/wasabi_upload/session.py) -**Usage Example**: [examples/wasabi_upload_session.py](../examples/wasabi_upload_session.py) +**API Documentation**: See docstrings in [session.py](../src/airoa_lineage/s3_upload/session.py) +**Usage Example**: [examples/s3_upload_session.py](../examples/s3_upload_session.py) ### USBCopySession @@ -278,7 +278,7 @@ For detailed event schemas, see the [OpenLineage specification](https://openline - [MarquezClient](../src/airoa_lineage/marquez_client/client.py) - Event emission and REST API - [TeleopSession](../src/airoa_lineage/teleop/session.py) - Robot data collection lifecycle - [ConversionSession](../src/airoa_lineage/conversion/session.py) - Data conversion pipeline lifecycle -- [WasabiUploadSession](../src/airoa_lineage/wasabi_upload/session.py) - Wasabi cloud storage upload lifecycle +- [S3UploadSession](../src/airoa_lineage/s3_upload/session.py) - S3 object storage upload lifecycle - [USBCopySession](../src/airoa_lineage/usb_copy/session.py) - USB data copy lifecycle - [Timestamp Utilities](../src/airoa_lineage/utils/timestamps.py) - UTC timestamp generation - [CommonRunFacet](../src/airoa_lineage/facets/common.py) - Robot metadata facet @@ -289,7 +289,7 @@ For detailed event schemas, see the [OpenLineage specification](https://openline - [examples/teleop_session.py](../examples/teleop_session.py) - Complete teleoperation session with nominal time - [examples/data_conversion.py](../examples/data_conversion.py) - Data conversion pipeline with RUNNING events -- [examples/wasabi_upload_session.py](../examples/wasabi_upload_session.py) - Wasabi upload session with nominal time +- [examples/s3_upload_session.py](../examples/s3_upload_session.py) - S3 upload session with nominal time - [examples/usb_copy_session.py](../examples/usb_copy_session.py) - USB data copy session with nominal time - [examples/simple_batch_etl.py](../examples/simple_batch_etl.py) - Batch ETL job with input/output datasets diff --git a/docs/cli/overview.md b/docs/cli/overview.md index b5d9ac3..1e4a93f 100644 --- a/docs/cli/overview.md +++ b/docs/cli/overview.md @@ -5,7 +5,7 @@ This guide covers common features and configuration methods for the `airoa-linea ## Available CLI Commands - **[airoa-lineage-usb-copy](usb-copy.md)** - Track USB data copy sessions -- **[airoa-lineage-wasabi-upload](wasabi-upload.md)** - Track Wasabi upload sessions +- **[airoa-lineage-s3-upload](s3-upload.md)** - Track S3 upload sessions --- @@ -23,7 +23,7 @@ Verify installation: ```bash airoa-lineage-usb-copy --version -airoa-lineage-wasabi-upload --version +airoa-lineage-s3-upload --version ``` --- @@ -321,13 +321,13 @@ This shows what would be sent to Marquez without actually sending it. For complete working examples, see the respective documentation pages: - [USBCopy CLI Examples](usb-copy.md#examples) -- [WasabiUpload CLI Examples](wasabi-upload.md#examples) +- [S3Upload CLI Examples](s3-upload.md#examples) --- ## Getting Help -- CLI help: `airoa-lineage-usb-copy --help` or `airoa-lineage-wasabi-upload --help` +- CLI help: `airoa-lineage-usb-copy --help` or `airoa-lineage-s3-upload --help` - Command help: ` start --help` - Version: ` --version` - GitHub Issues: https://github.com/AIRoA/airoa-lineage/issues diff --git a/docs/cli/wasabi-upload.md b/docs/cli/s3-upload.md similarity index 70% rename from docs/cli/wasabi-upload.md rename to docs/cli/s3-upload.md index cc4eb19..af5e6e8 100644 --- a/docs/cli/wasabi-upload.md +++ b/docs/cli/s3-upload.md @@ -1,17 +1,17 @@ -# WasabiUpload CLI +# S3Upload CLI -The `airoa-lineage-wasabi-upload` command provides a CLI interface for tracking Wasabi data upload operations using OpenLineage. +The `airoa-lineage-s3-upload` command provides a CLI interface for tracking S3 data upload operations using OpenLineage. ## Commands -### `start` - Start Wasabi Upload Session +### `start` - Start S3 Upload Session -Start a Wasabi upload session and emit a START event to Marquez. +Start a S3 upload session and emit a START event to Marquez. **Synopsis:** ```bash -airoa-lineage-wasabi-upload start [OPTIONS] +airoa-lineage-s3-upload start [OPTIONS] ``` **Options:** @@ -27,7 +27,7 @@ airoa-lineage-wasabi-upload start [OPTIONS] | `--repository-branch` | string | Yes* | Git branch | | `--nominal-start-time` | string | No | Nominal start time (ISO 8601 format) | | `--nominal-end-time` | string | No | Nominal end time (ISO 8601 format) | -| `--job-name` | string | No | Job name (default: `wasabi-data-upload`) | +| `--job-name` | string | No | Job name (default: `s3-data-upload`) | | `--marquez-url` | string | No | Marquez server URL | | `--facet-prefix` | string | No | Facet prefix | | `--dry-run` | flag | No | Show what would be done without executing | @@ -45,7 +45,7 @@ airoa-lineage-wasabi-upload start [OPTIONS] ```bash # Basic usage -RUN_ID=$(airoa-lineage-wasabi-upload start \ +RUN_ID=$(airoa-lineage-s3-upload start \ --namespace airoa_examples \ --robot-id hsr001 \ --location weblab \ @@ -55,34 +55,34 @@ RUN_ID=$(airoa-lineage-wasabi-upload start \ --repository-branch main) # With nominal times (for batch processing) -RUN_ID=$(airoa-lineage-wasabi-upload start \ +RUN_ID=$(airoa-lineage-s3-upload start \ --nominal-start-time "2025-11-26T00:00:00+00:00" \ --nominal-end-time "2025-11-26T05:00:00+00:00") # With custom job name -RUN_ID=$(airoa-lineage-wasabi-upload start \ - --job-name "wasabi-experiment-upload") +RUN_ID=$(airoa-lineage-s3-upload start \ + --job-name "s3-experiment-upload") # Dry-run mode (verify configuration) -airoa-lineage-wasabi-upload start --dry-run +airoa-lineage-s3-upload start --dry-run # JSON output -RUN_ID=$(airoa-lineage-wasabi-upload start --json | jq -r '.run_id') +RUN_ID=$(airoa-lineage-s3-upload start --json | jq -r '.run_id') # Verbose mode -airoa-lineage-wasabi-upload start --verbose +airoa-lineage-s3-upload start --verbose ``` --- -### `complete` - Complete Wasabi Upload Session +### `complete` - Complete S3 Upload Session -Complete a Wasabi upload session and emit a COMPLETE event to Marquez. +Complete a S3 upload session and emit a COMPLETE event to Marquez. **Synopsis:** ```bash -airoa-lineage-wasabi-upload complete --run-id RUN_ID [OPTIONS] +airoa-lineage-s3-upload complete --run-id RUN_ID [OPTIONS] ``` **Options:** @@ -115,16 +115,16 @@ airoa-lineage-wasabi-upload complete --run-id RUN_ID [OPTIONS] ```bash # Basic usage -airoa-lineage-wasabi-upload complete --run-id "$RUN_ID" +airoa-lineage-s3-upload complete --run-id "$RUN_ID" # With namespace override -airoa-lineage-wasabi-upload complete --run-id "$RUN_ID" --namespace airoa_examples +airoa-lineage-s3-upload complete --run-id "$RUN_ID" --namespace airoa_examples # Specify all arguments explicitly (using same values as start) -airoa-lineage-wasabi-upload complete \ +airoa-lineage-s3-upload complete \ --run-id "$RUN_ID" \ --namespace airoa_examples \ - --job-name wasabi-data-upload \ + --job-name s3-data-upload \ --robot-id hsr001 \ --location weblab \ --repository-hash abc123 \ @@ -133,22 +133,22 @@ airoa-lineage-wasabi-upload complete \ --repository-branch main # JSON output -airoa-lineage-wasabi-upload complete --run-id "$RUN_ID" --json +airoa-lineage-s3-upload complete --run-id "$RUN_ID" --json # Quiet mode -airoa-lineage-wasabi-upload complete --run-id "$RUN_ID" --quiet +airoa-lineage-s3-upload complete --run-id "$RUN_ID" --quiet ``` --- -### `cancel` - Cancel Wasabi Upload Session +### `cancel` - Cancel S3 Upload Session -Cancel a Wasabi upload session and emit an ABORT event to Marquez. +Cancel a S3 upload session and emit an ABORT event to Marquez. **Synopsis:** ```bash -airoa-lineage-wasabi-upload cancel --run-id RUN_ID [OPTIONS] +airoa-lineage-s3-upload cancel --run-id RUN_ID [OPTIONS] ``` **Options:** @@ -181,19 +181,19 @@ airoa-lineage-wasabi-upload cancel --run-id RUN_ID [OPTIONS] ```bash # Basic usage -airoa-lineage-wasabi-upload cancel --run-id "$RUN_ID" +airoa-lineage-s3-upload cancel --run-id "$RUN_ID" # Override specific CommonRunFacet fields -airoa-lineage-wasabi-upload cancel \ +airoa-lineage-s3-upload cancel \ --run-id "$RUN_ID" \ --robot-id hsr002 \ --location lab2 # Specify all arguments explicitly (using same values as start) -airoa-lineage-wasabi-upload cancel \ +airoa-lineage-s3-upload cancel \ --run-id "$RUN_ID" \ --namespace airoa_examples \ - --job-name wasabi-data-upload \ + --job-name s3-data-upload \ --robot-id hsr001 \ --location weblab \ --repository-hash abc123 \ @@ -202,52 +202,52 @@ airoa-lineage-wasabi-upload cancel \ --repository-branch main # JSON output -airoa-lineage-wasabi-upload cancel --run-id "$RUN_ID" --json +airoa-lineage-s3-upload cancel --run-id "$RUN_ID" --json # Quiet mode -airoa-lineage-wasabi-upload cancel --run-id "$RUN_ID" --quiet +airoa-lineage-s3-upload cancel --run-id "$RUN_ID" --quiet ``` --- ## Examples -For complete working examples, see [examples/cli/wasabi_upload/](../../examples/cli/wasabi_upload/). +For complete working examples, see [examples/cli/s3_upload/](../../examples/cli/s3_upload/). ### Available Examples The following executable shell scripts demonstrate different usage patterns: -1. **[basic_workflow.sh](../../examples/cli/wasabi_upload/basic_workflow.sh)** - Simple start → complete workflow +1. **[basic_workflow.sh](../../examples/cli/s3_upload/basic_workflow.sh)** - Simple start → complete workflow - All configuration via CLI arguments - Ideal for quick testing and learning -2. **[with_config_file.sh](../../examples/cli/wasabi_upload/with_config_file.sh)** - Configuration file usage +2. **[with_config_file.sh](../../examples/cli/s3_upload/with_config_file.sh)** - Configuration file usage - Loads settings from `config.example.json` - Minimal CLI arguments required - Includes dry-run mode for testing -3. **[with_env_vars.sh](../../examples/cli/wasabi_upload/with_env_vars.sh)** - Environment variables +3. **[with_env_vars.sh](../../examples/cli/s3_upload/with_env_vars.sh)** - Environment variables - Configuration via `AIROA_*` environment variables - Dynamically retrieves repository info from git -4. **[error_handling.sh](../../examples/cli/wasabi_upload/error_handling.sh)** - Error handling with trap +4. **[error_handling.sh](../../examples/cli/s3_upload/error_handling.sh)** - Error handling with trap - Automatic session cancellation on error - Cleanup function with proper exit codes -5. **[retry_logic.sh](../../examples/cli/wasabi_upload/retry_logic.sh)** - Connection retry logic +5. **[retry_logic.sh](../../examples/cli/s3_upload/retry_logic.sh)** - Connection retry logic - Automatic retry on connection errors - Maximum 3 attempts with exponential backoff -See [examples/cli/wasabi_upload/README.md](../../examples/cli/wasabi_upload/README.md) for detailed documentation on each example, including prerequisites, usage instructions, and integration patterns. +See [examples/cli/s3_upload/README.md](../../examples/cli/s3_upload/README.md) for detailed documentation on each example, including prerequisites, usage instructions, and integration patterns. --- ## Quick Start ```bash -# 1. Start a Wasabi upload session -RUN_ID=$(airoa-lineage-wasabi-upload start \ +# 1. Start a S3 upload session +RUN_ID=$(airoa-lineage-s3-upload start \ --namespace airoa_examples \ --robot-id hsr001 \ --location weblab \ @@ -258,14 +258,14 @@ RUN_ID=$(airoa-lineage-wasabi-upload start \ --nominal-start-time "2025-11-26T00:00:00+00:00" \ --nominal-end-time "2025-11-26T05:00:00+00:00") -# 2. Perform your Wasabi upload operation +# 2. Perform your S3 upload operation # (your upload processing here) # 3. Complete the session (using same metadata as start) -airoa-lineage-wasabi-upload complete \ +airoa-lineage-s3-upload complete \ --run-id "$RUN_ID" \ --namespace airoa_examples \ - --job-name wasabi-data-upload \ + --job-name s3-data-upload \ --robot-id hsr001 \ --location weblab \ --repository-hash $(git rev-parse HEAD) \ diff --git a/docs/cli/usb-copy.md b/docs/cli/usb-copy.md index 7e52468..d244c72 100644 --- a/docs/cli/usb-copy.md +++ b/docs/cli/usb-copy.md @@ -263,4 +263,4 @@ airoa-lineage-usb-copy complete \ ## Related Documentation - [CLI Overview](overview.md) - Common configuration, environment variables, troubleshooting -- [WasabiUpload CLI](wasabi-upload.md) - Track Wasabi upload operations +- [S3Upload CLI](s3-upload.md) - Track S3 upload operations diff --git a/docs/development.md b/docs/development.md index 4b0ffe7..9a8b4f3 100644 --- a/docs/development.md +++ b/docs/development.md @@ -146,7 +146,7 @@ tests/ │ └── test_session.py ├── conversion/ │ └── test_session.py - ├── wasabi_upload/ + ├── s3_upload/ │ └── test_session.py ├── usb_copy/ │ └── test_session.py @@ -516,7 +516,7 @@ airoa-lineage/ │ │ └── test_session.py │ ├── conversion/ │ │ └── test_session.py -│ ├── wasabi_upload/ +│ ├── s3_upload/ │ │ └── test_session.py │ ├── usb_copy/ │ │ └── test_session.py diff --git a/examples/cli/README.md b/examples/cli/README.md index 5ab87fe..7161eb8 100644 --- a/examples/cli/README.md +++ b/examples/cli/README.md @@ -17,9 +17,9 @@ Track USB data copy operations from the command line. - Error handling with automatic cancellation - Connection retry logic -### Wasabi Upload Session *(Coming Soon)* +### S3 Upload Session *(Coming Soon)* -Track Wasabi cloud upload operations from the command line. +Track S3 cloud upload operations from the command line. **Status**: Planned for future release diff --git a/examples/cli/wasabi_upload/README.md b/examples/cli/s3_upload/README.md similarity index 89% rename from examples/cli/wasabi_upload/README.md rename to examples/cli/s3_upload/README.md index 5aa4134..fda4cd5 100644 --- a/examples/cli/wasabi_upload/README.md +++ b/examples/cli/s3_upload/README.md @@ -1,6 +1,6 @@ -# Wasabi Upload CLI Examples +# S3 Upload CLI Examples -Command-line interface examples for tracking Wasabi data upload operations with OpenLineage. +Command-line interface examples for tracking S3 data upload operations with OpenLineage. ## Prerequisites @@ -15,7 +15,7 @@ pip install airoa-lineage Verify installation: ```bash -airoa-lineage-wasabi-upload --version +airoa-lineage-s3-upload --version ``` ### Marquez Server (Optional) @@ -154,7 +154,7 @@ All configuration file fields can be set via environment variables with the `AIR All examples can be tested without a Marquez server using `--dry-run`: ```bash -airoa-lineage-wasabi-upload start \ +airoa-lineage-s3-upload start \ --namespace production \ --robot-id hsr001 \ --location weblab \ @@ -171,26 +171,26 @@ This will display the configuration without sending events to Marquez. ### Integration with Your Scripts -To integrate Wasabi upload tracking into your existing scripts: +To integrate S3 upload tracking into your existing scripts: -1. **Start a session** before your Wasabi upload operation: +1. **Start a session** before your S3 upload operation: ```bash - RUN_ID=$(airoa-lineage-wasabi-upload start [options]) + RUN_ID=$(airoa-lineage-s3-upload start [options]) ``` -2. **Perform your Wasabi upload operation**: +2. **Perform your S3 upload operation**: ```bash - # Your actual Wasabi upload logic (rsync, cp, etc.) + # Your actual S3 upload logic (rsync, cp, etc.) rsync -av /source/path /destination/path ``` 3. **Complete or cancel** the session: ```bash # On success: - airoa-lineage-wasabi-upload complete --run-id "$RUN_ID" + airoa-lineage-s3-upload complete --run-id "$RUN_ID" # On failure: - airoa-lineage-wasabi-upload cancel --run-id "$RUN_ID" + airoa-lineage-s3-upload cancel --run-id "$RUN_ID" ``` ## Troubleshooting diff --git a/examples/cli/wasabi_upload/basic_workflow.sh b/examples/cli/s3_upload/basic_workflow.sh similarity index 73% rename from examples/cli/wasabi_upload/basic_workflow.sh rename to examples/cli/s3_upload/basic_workflow.sh index e69c6d3..4892299 100755 --- a/examples/cli/wasabi_upload/basic_workflow.sh +++ b/examples/cli/s3_upload/basic_workflow.sh @@ -1,10 +1,10 @@ #!/bin/bash ################################################################################ -# Basic Wasabi Upload Session Workflow +# Basic S3 Upload Session Workflow # # Purpose: -# Demonstrates the simplest start → complete flow for Wasabi upload operations. +# Demonstrates the simplest start → complete flow for S3 upload operations. # All configuration is provided via CLI arguments. # # Usage: @@ -22,7 +22,7 @@ export PATH="/opt/homebrew/bin:$PATH" # Common arguments shared between start and complete commands NAMESPACE="airoa_examples" -JOB_NAME="wasabi-data-upload" +JOB_NAME="s3-data-upload" ROBOT_ID="hsr001" LOCATION="weblab" REPOSITORY_HASH=$(git rev-parse HEAD) @@ -30,11 +30,11 @@ REPOSITORY_URI="https://github.com/AIRoA/airoa-lineage.git" REPOSITORY_TAG="v1.0.0" REPOSITORY_BRANCH="main" -# 1. Start a Wasabi upload session +# 1. Start a S3 upload session # The start command returns a RUN_ID which uniquely identifies this session -echo "Starting Wasabi upload session..." +echo "Starting S3 upload session..." -RUN_ID=$(uv run airoa-lineage-wasabi-upload start \ +RUN_ID=$(uv run airoa-lineage-s3-upload start \ --namespace "$NAMESPACE" \ --job-name "$JOB_NAME" \ --robot-id "$ROBOT_ID" \ @@ -46,18 +46,18 @@ RUN_ID=$(uv run airoa-lineage-wasabi-upload start \ --nominal-start-time "2025-11-26T00:00:00+00:00" \ --nominal-end-time "2025-11-26T05:00:00+00:00") -echo "Started Wasabi upload session with RUN_ID: $RUN_ID" +echo "Started S3 upload session with RUN_ID: $RUN_ID" -# 2. Perform your Wasabi upload operation -# Replace this placeholder with your actual Wasabi upload logic -echo "Performing Wasabi upload operation... (this is a placeholder)" +# 2. Perform your S3 upload operation +# Replace this placeholder with your actual S3 upload logic +echo "Performing S3 upload operation... (this is a placeholder)" # Example: rsync -av /source/path /destination/path # 3. Complete the session # Mark the session as successfully completed # IMPORTANT: Use the same namespace, job_name, and common_facet as the start command -echo "Completing Wasabi upload session..." -uv run airoa-lineage-wasabi-upload complete \ +echo "Completing S3 upload session..." +uv run airoa-lineage-s3-upload complete \ --run-id "$RUN_ID" \ --namespace "$NAMESPACE" \ --job-name "$JOB_NAME" \ @@ -68,4 +68,4 @@ uv run airoa-lineage-wasabi-upload complete \ --repository-tag "$REPOSITORY_TAG" \ --repository-branch "$REPOSITORY_BRANCH" -echo "Wasabi upload session completed successfully!" +echo "S3 upload session completed successfully!" diff --git a/examples/cli/wasabi_upload/config.example.json b/examples/cli/s3_upload/config.example.json similarity index 91% rename from examples/cli/wasabi_upload/config.example.json rename to examples/cli/s3_upload/config.example.json index 17adb6b..26653c8 100644 --- a/examples/cli/wasabi_upload/config.example.json +++ b/examples/cli/s3_upload/config.example.json @@ -1,7 +1,7 @@ { "namespace": "airoa_examples", "marquez_url": "http://localhost:9000", - "job_name": "wasabi-data-upload", + "job_name": "s3-data-upload", "facet_prefix": "airoa", "common_facet": { "robotId": "hsr001", diff --git a/examples/cli/wasabi_upload/error_handling.sh b/examples/cli/s3_upload/error_handling.sh similarity index 85% rename from examples/cli/wasabi_upload/error_handling.sh rename to examples/cli/s3_upload/error_handling.sh index 2e412a1..3fe92fb 100755 --- a/examples/cli/wasabi_upload/error_handling.sh +++ b/examples/cli/s3_upload/error_handling.sh @@ -1,7 +1,7 @@ #!/bin/bash ################################################################################ -# Wasabi Upload Session with Error Handling +# S3 Upload Session with Error Handling # # Purpose: # Demonstrates proper error handling using trap to automatically cancel @@ -28,7 +28,7 @@ set -u # Exit on undefined variable # Common arguments shared between start, complete, and cancel commands NAMESPACE="airoa_examples" -JOB_NAME="wasabi-data-upload" +JOB_NAME="s3-data-upload" ROBOT_ID="hsr001" LOCATION="weblab" REPOSITORY_HASH=$(git rev-parse HEAD) @@ -47,11 +47,11 @@ cleanup() { if [ $EXIT_CODE -ne 0 ]; then echo "" echo "Error detected (exit code: $EXIT_CODE)" - echo "Cancelling Wasabi upload session..." + echo "Cancelling S3 upload session..." # Cancel the session on error # IMPORTANT: Use the same namespace, job_name, and common_facet as the start command - if uv run airoa-lineage-wasabi-upload cancel \ + if uv run airoa-lineage-s3-upload cancel \ --run-id "$RUN_ID" \ --namespace "$NAMESPACE" \ --job-name "$JOB_NAME" \ @@ -73,8 +73,8 @@ cleanup() { trap cleanup EXIT # Start session -echo "Starting Wasabi upload session..." -RUN_ID=$(uv run airoa-lineage-wasabi-upload start \ +echo "Starting S3 upload session..." +RUN_ID=$(uv run airoa-lineage-s3-upload start \ --namespace "$NAMESPACE" \ --job-name "$JOB_NAME" \ --robot-id "$ROBOT_ID" \ @@ -91,21 +91,21 @@ if [ -z "$RUN_ID" ]; then exit 1 fi -echo "Started Wasabi upload session with RUN_ID: $RUN_ID" +echo "Started S3 upload session with RUN_ID: $RUN_ID" -# Simulate Wasabi upload operation (replace with actual logic) -echo "Performing Wasabi upload operation..." +# Simulate S3 upload operation (replace with actual logic) +echo "Performing S3 upload operation..." # Example: Uncomment the following line to simulate an error # exit 1 # If we get here, operation was successful -echo "Wasabi upload operation completed" +echo "S3 upload operation completed" # Complete the session # IMPORTANT: Use the same namespace, job_name, and common_facet as the start command echo "Completing session..." -if ! uv run airoa-lineage-wasabi-upload complete \ +if ! uv run airoa-lineage-s3-upload complete \ --run-id "$RUN_ID" \ --namespace "$NAMESPACE" \ --job-name "$JOB_NAME" \ diff --git a/examples/cli/wasabi_upload/retry_logic.sh b/examples/cli/s3_upload/retry_logic.sh similarity index 89% rename from examples/cli/wasabi_upload/retry_logic.sh rename to examples/cli/s3_upload/retry_logic.sh index d09552e..0efca43 100755 --- a/examples/cli/wasabi_upload/retry_logic.sh +++ b/examples/cli/s3_upload/retry_logic.sh @@ -1,7 +1,7 @@ #!/bin/bash ################################################################################ -# Wasabi Upload Session with Retry Logic +# S3 Upload Session with Retry Logic # # Purpose: # Demonstrates automatic retry logic for handling connection errors. @@ -27,7 +27,7 @@ set -e # Common arguments shared between start and complete commands NAMESPACE="airoa_examples" -JOB_NAME="wasabi-data-upload" +JOB_NAME="s3-data-upload" ROBOT_ID="hsr001" LOCATION="weblab" REPOSITORY_HASH=$(git rev-parse HEAD) @@ -43,11 +43,11 @@ start_session_with_retry() { local attempt=1 while [ $attempt -le $MAX_RETRIES ]; do - echo "Attempt $attempt/$MAX_RETRIES: Starting Wasabi upload session..." + echo "Attempt $attempt/$MAX_RETRIES: Starting S3 upload session..." # Disable 'set -e' temporarily to capture exit code set +e - RUN_ID=$(uv run airoa-lineage-wasabi-upload start \ + RUN_ID=$(uv run airoa-lineage-s3-upload start \ --namespace "$NAMESPACE" \ --job-name "$JOB_NAME" \ --robot-id "$ROBOT_ID" \ @@ -91,13 +91,13 @@ start_session_with_retry() { # Start session with retry if ! start_session_with_retry; then - echo "Failed to start Wasabi upload session after $MAX_RETRIES attempts" + echo "Failed to start S3 upload session after $MAX_RETRIES attempts" exit 1 fi -# Perform Wasabi upload operation -echo "Performing Wasabi upload operation..." -# Your Wasabi upload logic here +# Perform S3 upload operation +echo "Performing S3 upload operation..." +# Your S3 upload logic here # Complete the session with retry logic # IMPORTANT: Use the same namespace, job_name, and common_facet as the start command @@ -107,7 +107,7 @@ while [ $attempt -le $MAX_RETRIES ]; do echo "Attempt $attempt/$MAX_RETRIES: Completing session..." set +e - uv run airoa-lineage-wasabi-upload complete \ + uv run airoa-lineage-s3-upload complete \ --run-id "$RUN_ID" \ --namespace "$NAMESPACE" \ --job-name "$JOB_NAME" \ diff --git a/examples/cli/wasabi_upload/with_config_file.sh b/examples/cli/s3_upload/with_config_file.sh similarity index 79% rename from examples/cli/wasabi_upload/with_config_file.sh rename to examples/cli/s3_upload/with_config_file.sh index 4add68b..9286944 100755 --- a/examples/cli/wasabi_upload/with_config_file.sh +++ b/examples/cli/s3_upload/with_config_file.sh @@ -1,7 +1,7 @@ #!/bin/bash ################################################################################ -# Wasabi Upload Session with Configuration File +# S3 Upload Session with Configuration File # # Purpose: # Demonstrates using a configuration file to minimize CLI arguments. @@ -23,7 +23,7 @@ export PATH="/opt/homebrew/bin:$PATH" # { # "namespace": "production", # "marquez_url": "http://localhost:9000", -# "job_name": "wasabi-data-upload", +# "job_name": "s3-data-upload", # "facet_prefix": "airoa", # "common_facet": { # "robotId": "hsr001", @@ -41,7 +41,7 @@ set -e # Test configuration with dry-run mode first echo "Testing configuration with dry-run mode..." -uv run airoa-lineage-wasabi-upload start \ +uv run airoa-lineage-s3-upload start \ --nominal-start-time "2025-11-26T00:00:00+00:00" \ --nominal-end-time "2025-11-26T05:00:00+00:00" \ --dry-run @@ -51,18 +51,18 @@ echo "Configuration looks good! Starting actual session..." # Start session with minimal CLI arguments # All other settings are loaded from config file -RUN_ID=$(uv run airoa-lineage-wasabi-upload start \ +RUN_ID=$(uv run airoa-lineage-s3-upload start \ --nominal-start-time "2025-11-26T00:00:00+00:00" \ --nominal-end-time "2025-11-26T05:00:00+00:00") -echo "Started Wasabi upload session with RUN_ID: $RUN_ID" +echo "Started S3 upload session with RUN_ID: $RUN_ID" -# Perform Wasabi upload operation -echo "Performing Wasabi upload operation..." -# Your Wasabi upload logic here +# Perform S3 upload operation +echo "Performing S3 upload operation..." +# Your S3 upload logic here # Complete the session echo "Completing session..." -uv run airoa-lineage-wasabi-upload complete --run-id "$RUN_ID" +uv run airoa-lineage-s3-upload complete --run-id "$RUN_ID" echo "Session completed successfully!" diff --git a/examples/cli/wasabi_upload/with_env_vars.sh b/examples/cli/s3_upload/with_env_vars.sh similarity index 84% rename from examples/cli/wasabi_upload/with_env_vars.sh rename to examples/cli/s3_upload/with_env_vars.sh index e97b86a..080c52e 100755 --- a/examples/cli/wasabi_upload/with_env_vars.sh +++ b/examples/cli/s3_upload/with_env_vars.sh @@ -1,7 +1,7 @@ #!/bin/bash ################################################################################ -# Wasabi Upload Session with Environment Variables +# S3 Upload Session with Environment Variables # # Purpose: # Demonstrates using environment variables for configuration. @@ -24,7 +24,7 @@ set -e # Export configuration via environment variables export AIROA_NAMESPACE="airoa_examples" export AIROA_MARQUEZ_URL="http://localhost:9000" -export AIROA_JOB_NAME="wasabi-data-upload" +export AIROA_JOB_NAME="s3-data-upload" export AIROA_FACET_PREFIX="airoa" # Robot and location settings @@ -55,18 +55,18 @@ echo " Repository Branch: $AIROA_REPOSITORY_BRANCH" echo "" # Start session - configuration is read from environment variables -RUN_ID=$(uv run airoa-lineage-wasabi-upload start \ +RUN_ID=$(uv run airoa-lineage-s3-upload start \ --nominal-start-time "2025-11-26T00:00:00+00:00" \ --nominal-end-time "2025-11-26T05:00:00+00:00") -echo "Started Wasabi upload session with RUN_ID: $RUN_ID" +echo "Started S3 upload session with RUN_ID: $RUN_ID" -# Perform Wasabi upload operation -echo "Performing Wasabi upload operation..." -# Your Wasabi upload logic here +# Perform S3 upload operation +echo "Performing S3 upload operation..." +# Your S3 upload logic here # Complete the session echo "Completing session..." -uv run airoa-lineage-wasabi-upload complete --run-id "$RUN_ID" +uv run airoa-lineage-s3-upload complete --run-id "$RUN_ID" echo "Session completed successfully!" diff --git a/examples/wasabi_upload_session.py b/examples/s3_upload_session.py similarity index 87% rename from examples/wasabi_upload_session.py rename to examples/s3_upload_session.py index 5a05c1f..50cf5fe 100644 --- a/examples/wasabi_upload_session.py +++ b/examples/s3_upload_session.py @@ -1,20 +1,20 @@ """ -Wasabi Data Upload Session Example +S3 Data Upload Session Example -This example demonstrates a complete Wasabi data upload session: +This example demonstrates a complete S3 data upload session: 1. Send START event when upload begins with nominal time period -2. Simulate data upload to Wasabi cloud storage +2. Simulate data upload to S3 object storage 3. Send COMPLETE event when upload ends It shows how to: -- Use WasabiUploadSession class to manage upload session lifecycle +- Use S3UploadSession class to manage upload session lifecycle - Create CommonRunFacet with robot and repository information - Automatically track run_id without manual management - Specify robot_id and location to identify the robot and its location - Specify repository information (hash, URI, tag, branch) for traceability - Specify nominal time period (the time range of data being processed) - Use facet_prefix to namespace custom facets (e.g., "airoa_common") -- Track a complete Wasabi data upload session with OpenLineage +- Track a complete S3 data upload session with OpenLineage """ import os @@ -23,16 +23,16 @@ from airoa_lineage.facets import CommonRunFacet from airoa_lineage.marquez_client import MarquezClient -from airoa_lineage.wasabi_upload import WasabiUploadSession +from airoa_lineage.s3_upload import S3UploadSession def main(): - """Run a complete Wasabi data upload session simulation.""" + """Run a complete S3 data upload session simulation.""" # Session configuration marquez_url = os.getenv("MARQUEZ_URL", "http://localhost:9000") namespace = "airoa_examples" - job_name = "wasabi-data-upload" + job_name = "s3-data-upload" # Robot and repository information robot_id = "hsr001" # Robot identifier @@ -62,7 +62,7 @@ def main(): # Create session instance # This automatically generates a run_id and initializes the Marquez client # Using facet_prefix="airoa" to namespace our custom facets - session = WasabiUploadSession( + session = S3UploadSession( namespace=namespace, common_facet=common_facet, job_name=job_name, @@ -71,7 +71,7 @@ def main(): ) print("=" * 60) - print("Wasabi Data Upload Session Example") + print("S3 Data Upload Session Example") print("=" * 60) print(f"Namespace: {namespace}") print(f"Job Name: {job_name}") @@ -106,12 +106,12 @@ def main(): print() # ========== Simulate Data Upload ========== - print("[2/4] Simulating data upload to Wasabi...") + print("[2/4] Simulating data upload to S3...") # In a real scenario, this is where you would: # - Prepare data files for upload - # - Connect to Wasabi S3-compatible API - # - Upload files to Wasabi bucket + # - Connect to S3 API + # - Upload files to S3 bucket # - Verify upload integrity upload_duration = 3 # seconds time.sleep(upload_duration) @@ -119,8 +119,8 @@ def main(): print(f"✓ Data upload completed ({upload_duration} seconds)") print(" - In a real scenario, this would:") print(" • Prepare data files for upload") - print(" • Connect to Wasabi S3-compatible API") - print(" • Upload files to Wasabi bucket") + print(" • Connect to S3 API") + print(" • Upload files to S3 bucket") print(" • Verify upload integrity") print() @@ -158,7 +158,7 @@ def main(): print("=" * 60) print("Session Summary") print("=" * 60) - print("✓ Wasabi data upload session completed successfully") + print("✓ S3 data upload session completed successfully") print(f"✓ Robot ID: {robot_id}") print(f"✓ Location: {location}") print(f"✓ Repository: {repository_uri}") diff --git a/pyproject.toml b/pyproject.toml index b86fdf1..6b8bac3 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -13,7 +13,7 @@ dependencies = [ [project.scripts] airoa-lineage-usb-copy = "airoa_lineage.cli.usb_copy:main" -airoa-lineage-wasabi-upload = "airoa_lineage.cli.wasabi_upload:main" +airoa-lineage-s3-upload = "airoa_lineage.cli.s3_upload:main" [dependency-groups] dev = [ diff --git a/src/airoa_lineage/cli/wasabi_upload.py b/src/airoa_lineage/cli/s3_upload.py similarity index 91% rename from src/airoa_lineage/cli/wasabi_upload.py rename to src/airoa_lineage/cli/s3_upload.py index 3bf7b0a..f4bbaa9 100644 --- a/src/airoa_lineage/cli/wasabi_upload.py +++ b/src/airoa_lineage/cli/s3_upload.py @@ -1,12 +1,12 @@ -"""CLI for WasabiUploadSession. +"""CLI for S3UploadSession. -This module provides a command-line interface for tracking Wasabi data upload +This module provides a command-line interface for tracking S3 data upload operations using OpenLineage. Usage: - airoa-lineage-wasabi-upload start [options] - airoa-lineage-wasabi-upload complete --run-id RUN_ID [options] - airoa-lineage-wasabi-upload cancel --run-id RUN_ID [options] + airoa-lineage-s3-upload start [options] + airoa-lineage-s3-upload complete --run-id RUN_ID [options] + airoa-lineage-s3-upload cancel --run-id RUN_ID [options] """ from __future__ import annotations @@ -18,7 +18,7 @@ from typing import Any, Dict from airoa_lineage.cli.config import build_common_facet, load_config -from airoa_lineage.wasabi_upload import WasabiUploadSession +from airoa_lineage.s3_upload import S3UploadSession def _add_common_args(parser: argparse.ArgumentParser) -> None: @@ -87,20 +87,20 @@ def _add_common_facet_args(parser: argparse.ArgumentParser) -> None: def create_parser() -> argparse.ArgumentParser: """ - Create argument parser for airoa-lineage-wasabi-upload CLI. + Create argument parser for airoa-lineage-s3-upload CLI. Returns: Configured ArgumentParser instance """ parser = argparse.ArgumentParser( - prog="airoa-lineage-wasabi-upload", - description="Track Wasabi upload operations with OpenLineage", + prog="airoa-lineage-s3-upload", + description="Track S3 upload operations with OpenLineage", ) parser.add_argument( "--version", action="version", - version="airoa-lineage-wasabi-upload 0.1.0", + version="airoa-lineage-s3-upload 0.1.0", ) parser.add_argument( @@ -135,7 +135,7 @@ def create_parser() -> argparse.ArgumentParser: # START command start_parser = subparsers.add_parser( "start", - help="Start Wasabi upload session and emit START event", + help="Start S3 upload session and emit START event", ) _add_common_args(start_parser) _add_common_facet_args(start_parser) @@ -156,7 +156,7 @@ def create_parser() -> argparse.ArgumentParser: # COMPLETE command complete_parser = subparsers.add_parser( "complete", - help="Complete Wasabi upload session and emit COMPLETE event", + help="Complete S3 upload session and emit COMPLETE event", ) complete_parser.add_argument( "--run-id", @@ -169,7 +169,7 @@ def create_parser() -> argparse.ArgumentParser: # CANCEL command cancel_parser = subparsers.add_parser( "cancel", - help="Cancel Wasabi upload session and emit ABORT event", + help="Cancel S3 upload session and emit ABORT event", ) cancel_parser.add_argument( "--run-id", @@ -273,7 +273,7 @@ def cmd_start(args: argparse.Namespace, config: Dict[str, Any]) -> int: if args.dry_run: print("[DRY RUN] Would start session with:") print(f" Namespace: {config['namespace']}") - print(f" Job name: {config.get('job_name', 'wasabi-data-upload')}") + print(f" Job name: {config.get('job_name', 's3-data-upload')}") print(f" Robot ID: {common_facet.robotId}") print(f" Location: {common_facet.location}") print(f" Repository: {common_facet.repositoryUri}") @@ -288,10 +288,10 @@ def cmd_start(args: argparse.Namespace, config: Dict[str, Any]) -> int: return 0 # Create session - session = WasabiUploadSession( + session = S3UploadSession( namespace=config["namespace"], common_facet=common_facet, - job_name=config.get("job_name", "wasabi-data-upload"), + job_name=config.get("job_name", "s3-data-upload"), marquez_url=config.get("marquez_url"), facet_prefix=config.get("facet_prefix", ""), ) @@ -355,10 +355,10 @@ def cmd_complete(args: argparse.Namespace, config: Dict[str, Any]) -> int: return 1 # Create session - session = WasabiUploadSession( + session = S3UploadSession( namespace=config["namespace"], common_facet=common_facet, - job_name=config.get("job_name", "wasabi-data-upload"), + job_name=config.get("job_name", "s3-data-upload"), marquez_url=config.get("marquez_url"), facet_prefix=config.get("facet_prefix", ""), ) @@ -419,10 +419,10 @@ def cmd_cancel(args: argparse.Namespace, config: Dict[str, Any]) -> int: return 1 # Create session - session = WasabiUploadSession( + session = S3UploadSession( namespace=config["namespace"], common_facet=common_facet, - job_name=config.get("job_name", "wasabi-data-upload"), + job_name=config.get("job_name", "s3-data-upload"), marquez_url=config.get("marquez_url"), facet_prefix=config.get("facet_prefix", ""), ) @@ -460,7 +460,7 @@ def cmd_cancel(args: argparse.Namespace, config: Dict[str, Any]) -> int: def main() -> int: """ - Main entry point for airoa-lineage-wasabi-upload CLI. + Main entry point for airoa-lineage-s3-upload CLI. Returns: Exit code (0=success, 1=general error, 2=connection error, 3=config error) diff --git a/src/airoa_lineage/s3_upload/__init__.py b/src/airoa_lineage/s3_upload/__init__.py new file mode 100644 index 0000000..0c33b8d --- /dev/null +++ b/src/airoa_lineage/s3_upload/__init__.py @@ -0,0 +1,5 @@ +"""S3 data upload session management with OpenLineage tracking.""" + +from airoa_lineage.s3_upload.session import S3UploadSession + +__all__ = ["S3UploadSession"] diff --git a/src/airoa_lineage/wasabi_upload/session.py b/src/airoa_lineage/s3_upload/session.py similarity index 77% rename from src/airoa_lineage/wasabi_upload/session.py rename to src/airoa_lineage/s3_upload/session.py index 49758d0..37b1e4c 100644 --- a/src/airoa_lineage/wasabi_upload/session.py +++ b/src/airoa_lineage/s3_upload/session.py @@ -1,4 +1,4 @@ -"""Wasabi data upload session management with OpenLineage tracking.""" +"""S3 data upload session management with OpenLineage tracking.""" from typing import Dict, Optional @@ -8,17 +8,17 @@ from airoa_lineage.facets import CommonRunFacet -class WasabiUploadSession(BaseSession): +class S3UploadSession(BaseSession): """ - Manages a Wasabi data upload session with OpenLineage tracking. + Manages an S3 data upload session with OpenLineage tracking. - This class handles the lifecycle of a Wasabi cloud storage upload operation, + This class handles the lifecycle of an S3 object storage upload operation, automatically managing the run_id and ensuring proper event sequencing. Examples: >>> # Basic usage >>> from airoa_lineage.facets import CommonRunFacet - >>> from airoa_lineage.wasabi_upload import WasabiUploadSession + >>> from airoa_lineage.s3_upload import S3UploadSession >>> common_facet = CommonRunFacet( ... robotId="hsr001", ... location="weblab", @@ -27,19 +27,19 @@ class WasabiUploadSession(BaseSession): ... repositoryTag="v1.0.0", ... repositoryBranch="main" ... ) - >>> session = WasabiUploadSession( + >>> session = S3UploadSession( ... namespace="my_namespace", ... common_facet=common_facet ... ) >>> run_id = session.start() - >>> # Upload data to Wasabi... + >>> # Upload data to S3... >>> session.complete() >>> # Custom configuration - >>> session = WasabiUploadSession( + >>> session = S3UploadSession( ... namespace="production", ... common_facet=common_facet, - ... job_name="my-wasabi-upload-job", + ... job_name="my-s3-upload-job", ... run_id="existing-run-id-123" ... ) >>> session.start() @@ -50,18 +50,18 @@ def __init__( self, namespace: str, common_facet: CommonRunFacet, - job_name: str = "wasabi-data-upload", + job_name: str = "s3-data-upload", marquez_url: Optional[str] = None, run_id: Optional[str] = None, facet_prefix: str = "", ): """ - Initialize a Wasabi data upload session. + Initialize an S3 data upload session. Args: namespace: OpenLineage namespace (required) common_facet: Common metadata (required, includes robotId, location, repository info) - job_name: Job name (default: "wasabi-data-upload") + job_name: Job name (default: "s3-data-upload") marquez_url: Marquez server URL (default: from MARQUEZ_URL env or localhost:9000) run_id: Session run ID (auto-generated UUID if not provided) facet_prefix: Prefix for custom facet names (default: "", no prefix). @@ -70,7 +70,7 @@ def __init__( Examples: >>> # Basic usage >>> from airoa_lineage.facets import CommonRunFacet - >>> from airoa_lineage.wasabi_upload import WasabiUploadSession + >>> from airoa_lineage.s3_upload import S3UploadSession >>> common_facet = CommonRunFacet( ... robotId="hsr001", ... location="weblab", @@ -79,13 +79,13 @@ def __init__( ... repositoryTag="v1.0.0", ... repositoryBranch="main" ... ) - >>> session = WasabiUploadSession( + >>> session = S3UploadSession( ... namespace="my_namespace", ... common_facet=common_facet ... ) >>> # Custom configuration - >>> session = WasabiUploadSession( + >>> session = S3UploadSession( ... namespace="my_namespace", ... common_facet=common_facet, ... job_name="my_job", @@ -93,7 +93,7 @@ def __init__( ... ) >>> # With facet prefix - >>> session = WasabiUploadSession( + >>> session = S3UploadSession( ... namespace="my_namespace", ... common_facet=common_facet, ... facet_prefix="airoa" @@ -109,4 +109,4 @@ def _get_session_facets(self) -> Dict[str, BaseFacet]: def _get_producer(self) -> str: """Return producer identifier.""" - return "airoa-wasabi-system" + return "airoa-s3-system" diff --git a/src/airoa_lineage/wasabi_upload/__init__.py b/src/airoa_lineage/wasabi_upload/__init__.py deleted file mode 100644 index 0784a7e..0000000 --- a/src/airoa_lineage/wasabi_upload/__init__.py +++ /dev/null @@ -1,5 +0,0 @@ -"""Wasabi data upload session management with OpenLineage tracking.""" - -from airoa_lineage.wasabi_upload.session import WasabiUploadSession - -__all__ = ["WasabiUploadSession"] diff --git a/tests/unit/cli/test_wasabi_upload.py b/tests/unit/cli/test_s3_upload.py similarity index 98% rename from tests/unit/cli/test_wasabi_upload.py rename to tests/unit/cli/test_s3_upload.py index 9359898..25513c1 100644 --- a/tests/unit/cli/test_wasabi_upload.py +++ b/tests/unit/cli/test_s3_upload.py @@ -1,9 +1,9 @@ -"""Tests for CLI wasabi_upload module.""" +"""Tests for CLI s3_upload module.""" import argparse -from airoa_lineage.cli.wasabi_upload import ( +from airoa_lineage.cli.s3_upload import ( _add_common_args, _add_common_facet_args, create_parser, diff --git a/tests/unit/helpers/base_session_test.py b/tests/unit/helpers/base_session_test.py index 0699578..d4926ce 100644 --- a/tests/unit/helpers/base_session_test.py +++ b/tests/unit/helpers/base_session_test.py @@ -1,7 +1,7 @@ """Base test class for session tests. This module provides a base test class that contains common test methods -shared across all session test classes (USBCopy, WasabiUpload, Teleop, Conversion). +shared across all session test classes (USBCopy, S3Upload, Teleop, Conversion). Subclasses must define the following class attributes: - SESSION_CLASS: The session class to test (e.g., USBCopySession) @@ -440,7 +440,7 @@ def test_cancel_event_has_empty_inputs_outputs(self, helper, common_facet): """Test that ABORT event has empty inputs and outputs. Note: This test is for sessions without inputs/outputs (TeleopSession, - WasabiUploadSession, USBCopySession). ConversionSession will override this test. + S3UploadSession, USBCopySession). ConversionSession will override this test. """ with helper: session, mock_client = helper.create_session( diff --git a/tests/unit/wasabi_upload/__init__.py b/tests/unit/s3_upload/__init__.py similarity index 100% rename from tests/unit/wasabi_upload/__init__.py rename to tests/unit/s3_upload/__init__.py diff --git a/tests/unit/wasabi_upload/test_session.py b/tests/unit/s3_upload/test_session.py similarity index 54% rename from tests/unit/wasabi_upload/test_session.py rename to tests/unit/s3_upload/test_session.py index ff95a62..d09615d 100644 --- a/tests/unit/wasabi_upload/test_session.py +++ b/tests/unit/s3_upload/test_session.py @@ -1,19 +1,19 @@ -"""Unit tests for WasabiUploadSession.""" +"""Unit tests for S3UploadSession.""" import uuid from airoa_lineage.facets import CommonRunFacet -from airoa_lineage.wasabi_upload import WasabiUploadSession +from airoa_lineage.s3_upload import S3UploadSession from tests.unit.helpers.base_session_test import BaseSessionTest -class TestWasabiUploadSessionInitialization(BaseSessionTest): - """Test WasabiUploadSession initialization.""" +class TestS3UploadSessionInitialization(BaseSessionTest): + """Test S3UploadSession initialization.""" - SESSION_CLASS = WasabiUploadSession - DEFAULT_JOB_NAME = "wasabi-data-upload" - PRODUCER_NAME = "airoa-wasabi-system" + SESSION_CLASS = S3UploadSession + DEFAULT_JOB_NAME = "s3-data-upload" + PRODUCER_NAME = "airoa-s3-system" def test_initialization_with_custom_values(self): """Test initialization with custom values.""" @@ -26,7 +26,7 @@ def test_initialization_with_custom_values(self): repositoryTag="v2.0.0", repositoryBranch="develop", ) - session = WasabiUploadSession( + session = S3UploadSession( namespace="custom_namespace", common_facet=common_facet, job_name="custom_job", @@ -50,41 +50,41 @@ def test_initialization_with_custom_values(self): assert session._completed is False -class TestWasabiUploadSessionStart(BaseSessionTest): - """Test WasabiUploadSession start() method.""" +class TestS3UploadSessionStart(BaseSessionTest): + """Test S3UploadSession start() method.""" - SESSION_CLASS = WasabiUploadSession - DEFAULT_JOB_NAME = "wasabi-data-upload" - PRODUCER_NAME = "airoa-wasabi-system" + SESSION_CLASS = S3UploadSession + DEFAULT_JOB_NAME = "s3-data-upload" + PRODUCER_NAME = "airoa-s3-system" -class TestWasabiUploadSessionComplete(BaseSessionTest): - """Test WasabiUploadSession complete() method.""" +class TestS3UploadSessionComplete(BaseSessionTest): + """Test S3UploadSession complete() method.""" - SESSION_CLASS = WasabiUploadSession - DEFAULT_JOB_NAME = "wasabi-data-upload" - PRODUCER_NAME = "airoa-wasabi-system" + SESSION_CLASS = S3UploadSession + DEFAULT_JOB_NAME = "s3-data-upload" + PRODUCER_NAME = "airoa-s3-system" -class TestWasabiUploadSessionNominalTime(BaseSessionTest): - """Test WasabiUploadSession nominal time support.""" +class TestS3UploadSessionNominalTime(BaseSessionTest): + """Test S3UploadSession nominal time support.""" - SESSION_CLASS = WasabiUploadSession - DEFAULT_JOB_NAME = "wasabi-data-upload" - PRODUCER_NAME = "airoa-wasabi-system" + SESSION_CLASS = S3UploadSession + DEFAULT_JOB_NAME = "s3-data-upload" + PRODUCER_NAME = "airoa-s3-system" class TestCommonFacet(BaseSessionTest): - """Test CommonRunFacet in WasabiUploadSession.""" + """Test CommonRunFacet in S3UploadSession.""" - SESSION_CLASS = WasabiUploadSession - DEFAULT_JOB_NAME = "wasabi-data-upload" - PRODUCER_NAME = "airoa-wasabi-system" + SESSION_CLASS = S3UploadSession + DEFAULT_JOB_NAME = "s3-data-upload" + PRODUCER_NAME = "airoa-s3-system" class TestFacetPrefix(BaseSessionTest): """Test facet_prefix functionality.""" - SESSION_CLASS = WasabiUploadSession - DEFAULT_JOB_NAME = "wasabi-data-upload" - PRODUCER_NAME = "airoa-wasabi-system" + SESSION_CLASS = S3UploadSession + DEFAULT_JOB_NAME = "s3-data-upload" + PRODUCER_NAME = "airoa-s3-system"