Feature/wasabi upload cli

docs/cli/overview.md

@@ -0,0 +1,335 @@
+# CLI Overview
+
+This guide covers common features and configuration methods for the `airoa-lineage` CLI interfaces.
+
+## Available CLI Commands
+
+- **[airoa-lineage-usb-copy](usb-copy.md)** - Track USB data copy sessions
+- **[airoa-lineage-wasabi-upload](wasabi-upload.md)** - Track Wasabi upload sessions
+
+---
+
+## Installation
+
+```bash
+# Using uv (recommended)
+uv pip install -e .
+
+# Or using pip
+pip install -e .
+```
+
+Verify installation:
+
+```bash
+airoa-lineage-usb-copy --version
+airoa-lineage-wasabi-upload --version
+```
+
+---
+
+## Configuration
+
+The CLI supports three configuration sources with the following priority:
+
+**CLI arguments > Environment variables > Configuration file > Defaults**
+
+### Configuration File
+
+Default path: `~/.config/airoa-lineage/config.json`
+
+You can override this with the `--config` option or by setting `XDG_CONFIG_HOME`.
+
+**Example config.json:**
+
+```json
+{
+  "marquez_url": "http://localhost:9000",
+  "namespace": "airoa_examples",
+  "facet_prefix": "airoa",
+  "job_name": "usb-data-copy",
+  "common_facet": {
+    "robotId": "hsr001",
+    "location": "weblab",
+    "repositoryHash": "df110d5a8e3b9c2f1a7d6e4f5c3a2b1",
+    "repositoryUri": "https://github.com/AIRoA/airoa-lineage.git",
+    "repositoryTag": "v1.0.0",
+    "repositoryBranch": "main"
+  }
+}
+```
+
+**Configuration Schema:**
+
+| Field | Type | Required | Default | Description |
+|-------|------|----------|---------|-------------|
+| `marquez_url` | string | No | `http://localhost:9000` | Marquez server URL |
+| `namespace` | string | **Yes** | - | OpenLineage namespace |
+| `job_name` | string | No | Command-specific | Job identifier |
+| `facet_prefix` | string | No | `""` (empty) | Prefix for custom facets |
+| `common_facet` | object | **Yes** | - | CommonRunFacet fields |
+
+**CommonRunFacet Fields:**
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `robotId` | string | **Yes** | Robot identifier (e.g., `hsr001`) |
+| `location` | string | **Yes** | Location identifier (e.g., `weblab`) |
+| `repositoryHash` | string | **Yes** | Git commit hash |
+| `repositoryUri` | string | **Yes** | Repository URI |
+| `repositoryTag` | string | **Yes** | Git tag (e.g., `v1.0.0`) |
+| `repositoryBranch` | string | **Yes** | Git branch (e.g., `main`) |
+
+### Environment Variables
+
+All configuration values can be set via environment variables:
+
+```bash
+# Top-level configuration
+export AIROA_MARQUEZ_URL=http://localhost:9000
+export AIROA_NAMESPACE=airoa_examples
+export AIROA_FACET_PREFIX=airoa
+export AIROA_JOB_NAME=usb-data-copy
+
+# CommonRunFacet fields
+export AIROA_ROBOT_ID=hsr001
+export AIROA_LOCATION=weblab
+export AIROA_REPOSITORY_HASH=$(git rev-parse HEAD)
+export AIROA_REPOSITORY_URI=https://github.com/AIRoA/airoa-lineage.git
+export AIROA_REPOSITORY_TAG=$(git describe --tags)
+export AIROA_REPOSITORY_BRANCH=$(git rev-parse --abbrev-ref HEAD)
+```
+
+**Note:** `MARQUEZ_URL` is also supported for backward compatibility, but `AIROA_MARQUEZ_URL` takes precedence.
+
+### Priority Order Example
+
+```bash
+# config.json has: "namespace": "dev"
+# Environment has: AIROA_NAMESPACE=staging
+
+# This uses CLI argument (production)
+airoa-lineage-usb-copy start --namespace production
+
+# This uses environment variable (staging)
+airoa-lineage-usb-copy start
+
+# If no env var, uses config file (dev)
+unset AIROA_NAMESPACE
+airoa-lineage-usb-copy start
+```
+
+---
+
+## Global Options
+
+These options apply to all commands:
+
+| Option | Short | Description |
+|--------|-------|-------------|
+| `--config PATH` | - | Path to config file (default: `~/.config/airoa-lineage/config.json`) |
+| `--verbose` | `-v` | Enable verbose output (messages to stderr) |
+| `--quiet` | `-q` | Suppress output except errors |
+| `--json` | - | Output in JSON format |
+| `--version` | - | Show version and exit |
+| `--help` | `-h` | Show help message and exit |
+
+**Examples:**
+
+```bash
+# Show version
+airoa-lineage-usb-copy --version
+
+# Show help
+airoa-lineage-usb-copy --help
+airoa-lineage-usb-copy start --help
+
+# Use custom config file
+airoa-lineage-usb-copy --config /path/to/config.json start
+
+# Verbose mode
+airoa-lineage-usb-copy -v start
+
+# JSON output
+airoa-lineage-usb-copy --json start
+```
+
+---
+
+## Output Modes
+
+### Default Mode
+
+- **start:** Prints only `run_id` to stdout (for piping)
+- **complete/cancel:** Prints success message to stdout
+
+```bash
+RUN_ID=$(airoa-lineage-usb-copy start)
+# Output: 550e8400-e29b-41d4-a716-446655440000
+```
+
+### Verbose Mode (`-v`, `--verbose`)
+
+- All messages go to **stderr**
+- `run_id` goes to **stdout** (for piping)
+- Includes configuration details and event status
+
+```bash
+airoa-lineage-usb-copy -v start
+# stderr: Loading config from ~/.config/airoa-lineage/config.json
+# stderr: Connecting to Marquez at http://localhost:9000
+# stderr: Session started successfully
+# stdout: 550e8400-e29b-41d4-a716-446655440000
+```
+
+### Quiet Mode (`-q`, `--quiet`)
+
+- **start:** Prints only `run_id` to stdout
+- **complete/cancel:** No output (silent)
+- Errors still go to stderr
+
+```bash
+airoa-lineage-usb-copy -q complete --run-id "$RUN_ID"
+# (no output)
+```
+
+### JSON Mode (`--json`)
+
+- All output in JSON format
+- Suitable for programmatic parsing
+
+```bash
+airoa-lineage-usb-copy --json start
+# {"run_id": "550e8400-e29b-41d4-a716-446655440000", "status": "started"}
+
+airoa-lineage-usb-copy --json complete --run-id "$RUN_ID"
+# {"run_id": "550e8400-e29b-41d4-a716-446655440000", "status": "completed"}
+```
+
+---
+
+## Exit Codes
+
+The CLI uses standard exit codes to indicate success or failure:
+
+| Code | Meaning | Description |
+|------|---------|-------------|
+| `0` | Success | Command completed successfully |
+| `1` | General error | Session error, runtime error, or unknown error |
+| `2` | Connection error | Failed to connect to Marquez server |
+| `3` | Configuration error | Missing or invalid configuration |
+
+**Example:**
+
+```bash
+airoa-lineage-usb-copy start
+EXIT_CODE=$?
+
+if [ $EXIT_CODE -eq 0 ]; then
+  echo "Success"
+elif [ $EXIT_CODE -eq 3 ]; then
+  echo "Configuration error - check your config file"
+elif [ $EXIT_CODE -eq 2 ]; then
+  echo "Connection error - is Marquez running?"
+else
+  echo "Unknown error"
+fi
+```
+
+---
+
+## Troubleshooting
+
+### Common Errors
+
+#### 1. "Missing required CommonRunFacet fields"
+
+**Solution:**
+
+Provide all required fields via one of these methods:
+
+```bash
+# Option 1: CLI arguments
+airoa-lineage-usb-copy start \
+  --robot-id hsr001 \
+  --location weblab \
+  --repository-hash $(git rev-parse HEAD) \
+  --repository-uri https://github.com/AIRoA/airoa-lineage.git \
+  --repository-tag v1.0.0 \
+  --repository-branch main
+
+# Option 2: Environment variables
+export AIROA_ROBOT_ID=hsr001
+export AIROA_LOCATION=weblab
+# ... other fields
+
+# Option 3: Config file
+# Add common_facet to ~/.config/airoa-lineage/config.json
+```
+
+#### 2. "namespace is required"
+
+**Solution:**
+
+```bash
+# Option 1: CLI argument
+airoa-lineage-usb-copy start --namespace airoa_examples
+
+# Option 2: Environment variable
+export AIROA_NAMESPACE=airoa_examples
+
+# Option 3: Config file
+echo '{"namespace": "airoa_examples"}' > ~/.config/airoa-lineage/config.json
+```
+
+#### 3. Connection refused / Marquez server not running
+
+**Solution:**
+
+Start the Marquez server:
+
+```bash
+# Using Docker
+docker run -d -p 3000:3000 -p 9000:9000 marquezproject/marquez:latest
+
+# Verify it's running
+curl http://localhost:9000/api/v1/namespaces
+```
+
+### Debugging Tips
+
+#### Use verbose mode
+
+```bash
+airoa-lineage-usb-copy -v start
+```
+
+This shows config file path, loaded configuration values, Marquez connection details, and event emission status.
+
+#### Use dry-run mode
+
+```bash
+airoa-lineage-usb-copy start --dry-run
+```
+
+This shows what would be sent to Marquez without actually sending it.
+
+---
+
+## Examples
+
+For complete working examples, see the respective documentation pages:
+
+- [USBCopy CLI Examples](usb-copy.md#examples)
+- [WasabiUpload CLI Examples](wasabi-upload.md#examples)
+
+---
+
+## Getting Help
+
+- CLI help: `airoa-lineage-usb-copy --help` or `airoa-lineage-wasabi-upload --help`
+- Command help: `<command> start --help`
+- Version: `<command> --version`
+- GitHub Issues: https://github.com/AIRoA/airoa-lineage/issues
+- OpenLineage docs: https://openlineage.io/docs
+- Marquez docs: https://marquezproject.github.io/marquez/