Feature/issue122

README.md

@@ -12,8 +12,8 @@ This library is designed to be integrated into robot data collection systems, ET
 
 - 🤖 **Teleoperation Session Management**: Track robot data collection sessions with `TeleopSession`
 - 🔄 **Data Conversion Session Management**: Track data conversion pipelines with `ConversionSession`
-- ☁️ **S3 Upload Management**: Track S3 object storage uploads with `S3UploadSession`
-- 🔌 **USB Data Copy Management**: Track USB data copy operations with `USBCopySession`
+- ☁️ **S3 Upload Management**: Track S3 object storage uploads with `S3UploadSession` (includes device and job tracking)
+- 🔌 **USB Data Copy Management**: Track USB data copy operations with `USBCopySession` (includes device and job tracking)
 - 📊 **OpenLineage Integration**: Full OpenLineage specification support (START/COMPLETE/RUNNING/FAIL/ABORT)
 - 🏷️ **Custom Run Facets**: Robot metadata (robotId, location, repository info) via `CommonRunFacet`
 - 🔍 **Marquez Client**: Query jobs, datasets, and lineage graphs via REST API
@@ -124,11 +124,16 @@ Track USB copy operations from the command line.
 Quick example:
 
 ```bash
+# Generate unique job ID
+JOB_ID=$(python3 -c "import uuid; print(uuid.uuid4())")
+
 # Start USB copy session
 RUN_ID=$(airoa-lineage-usb-copy start \
   --namespace production \
   --robot-id hsr001 \
   --location weblab \
+  --hostname $(hostname) \
+  --job-id "$JOB_ID" \
   --repository-hash $(git rev-parse HEAD) \
   --repository-uri https://github.com/user/repo.git \
   --repository-tag v1.0.0 \
@@ -140,7 +145,7 @@ RUN_ID=$(airoa-lineage-usb-copy start \
 # ...
 
 # Complete session
-airoa-lineage-usb-copy complete --run-id "$RUN_ID"
+airoa-lineage-usb-copy complete --run-id "$RUN_ID" --job-id "$JOB_ID" --hostname $(hostname)
 ```
 
 See [docs/cli-usage.md](docs/cli-usage.md) for full CLI reference.

docs/architecture.md

@@ -103,6 +103,10 @@ ConversionSession.complete(output_datasets) # Emits COMPLETE event
 - Track S3 data upload sessions (START → COMPLETE)
 - Automatic run_id generation and management
 - Nominal time support for historical data processing
+- Device tracking via DeviceRunFacet (hostname - **required**)
+- Job tracking via JobRunFacet (unique job ID - **required**)
+- Input datasets tracking (source directory, source files)
+- Output datasets tracking (destination path, upload duration)
 - State validation (prevents duplicate start/complete calls)
 
 **Lifecycle**:
@@ -125,6 +129,10 @@ S3UploadSession.complete()   # Emits COMPLETE event
 - Track USB data copy sessions (START → COMPLETE)
 - Automatic run_id generation and management
 - Nominal time support for historical data processing
+- Device tracking via DeviceRunFacet (hostname - **required**)
+- Job tracking via JobRunFacet (unique job ID - **required**)
+- Input datasets tracking (source directory, source files)
+- Output datasets tracking (destination directory, copy duration)
 - State validation (prevents duplicate start/complete calls)
 
 **Lifecycle**:
@@ -158,13 +166,18 @@ USBCopySession.complete()   # Emits COMPLETE event
 **Purpose**: Provide robot-specific metadata extensions to OpenLineage Run Facets.
 
 **Available Facets**:
-- **CommonRunFacet** ([common.py](../src/airoa_lineage/facets/common.py)): Robot metadata (robotId, location, repository info)
-- **DeviceRunFacet** ([device.py](../src/airoa_lineage/facets/device.py)): Device metadata (hostname)
-- **AWSJobRunFacet** ([aws_job.py](../src/airoa_lineage/facets/aws_job.py)): AWS Lambda job metadata (name, id)
+- **CommonRunFacet** ([run/common.py](../src/airoa_lineage/facets/run/common.py)): Robot metadata (robotId, location, repository info)
+  - Used by: TeleopSession, ConversionSession, S3UploadSession, USBCopySession
+- **DeviceRunFacet** ([run/device.py](../src/airoa_lineage/facets/run/device.py)): Device metadata (hostname)
+  - Used by: TeleopSession, S3UploadSession (**required**), USBCopySession (**required**)
+- **JobRunFacet** ([run/job.py](../src/airoa_lineage/facets/run/job.py)): Job identification metadata (unique job ID)
+  - Used by: S3UploadSession (**required**), USBCopySession (**required**)
+- **AWSJobRunFacet** ([run/aws_job.py](../src/airoa_lineage/facets/run/aws_job.py)): AWS Lambda job metadata (name, id)
+  - Used by: ConversionSession
 
-**Facet Namespacing**: Use `facet_prefix` parameter to customize facet names (e.g., `facet_prefix="airoa"` → facets named `"airoa_common"`, `"airoa_device"`, and `"airoa_awsJob"`)
+**Facet Namespacing**: Use `facet_prefix` parameter to customize facet names (e.g., `facet_prefix="airoa"` → facets named `"airoa_common"`, `"airoa_device"`, `"airoa_job"`, and `"airoa_awsJob"`)
 
-**API Documentation**: See docstrings in [facets/common.py](../src/airoa_lineage/facets/common.py) and [facets/device.py](../src/airoa_lineage/facets/device.py)
+**API Documentation**: See docstrings in [facets/run/common.py](../src/airoa_lineage/facets/run/common.py), [facets/run/device.py](../src/airoa_lineage/facets/run/device.py), and [facets/run/job.py](../src/airoa_lineage/facets/run/job.py)
 
 ## How Components Work Together
 
@@ -281,9 +294,10 @@ For detailed event schemas, see the [OpenLineage specification](https://openline
 - [S3UploadSession](../src/airoa_lineage/s3_upload/session.py) - S3 object storage upload lifecycle
 - [USBCopySession](../src/airoa_lineage/usb_copy/session.py) - USB data copy lifecycle
 - [Timestamp Utilities](../src/airoa_lineage/utils/timestamps.py) - UTC timestamp generation
-- [CommonRunFacet](../src/airoa_lineage/facets/common.py) - Robot metadata facet
-- [DeviceRunFacet](../src/airoa_lineage/facets/device.py) - Device metadata facet
-- [AWSJobRunFacet](../src/airoa_lineage/facets/aws_job.py) - AWS Lambda job metadata facet
+- [CommonRunFacet](../src/airoa_lineage/facets/run/common.py) - Robot metadata facet
+- [DeviceRunFacet](../src/airoa_lineage/facets/run/device.py) - Device metadata facet
+- [JobRunFacet](../src/airoa_lineage/facets/run/job.py) - Job identification metadata facet
+- [AWSJobRunFacet](../src/airoa_lineage/facets/run/aws_job.py) - AWS Lambda job metadata facet
 
 ### Examples (Usage Patterns)