Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 50 additions & 56 deletions .github/workflows/bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,40 +16,38 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Cache pip
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
echo "$HOME/.local/bin" >> $GITHUB_PATH


- name: Install dependencies
run: poetry install

run: |
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .

- name: Run Black
run: poetry run black --check ovmobilebench tests
run: black --check ovmobilebench tests

- name: Run Ruff
run: poetry run ruff check ovmobilebench tests
run: ruff check ovmobilebench tests

- name: Run MyPy
run: poetry run mypy ovmobilebench
run: mypy ovmobilebench --ignore-missing-imports

- name: Run tests
run: poetry run pytest tests/ -v --cov=ovmobilebench --cov-report=xml
run: pytest tests/ -v --cov=ovmobilebench --cov-report=xml

- name: Upload coverage
uses: codecov/codecov-action@v3
with:
Expand All @@ -61,20 +59,20 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install Poetry

- name: Install build dependencies
run: |
curl -sSL https://install.python-poetry.org | python3 -
echo "$HOME/.local/bin" >> $GITHUB_PATH
pip install --upgrade pip
pip install build setuptools wheel

- name: Build package
run: poetry build
run: python -m build

- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
Expand All @@ -87,62 +85,58 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
echo "$HOME/.local/bin" >> $GITHUB_PATH


- name: Install dependencies
run: poetry install

run: |
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .

- name: Validate example config
run: |
poetry run python -c "from ovmobilebench.config.loader import load_experiment; load_experiment('experiments/android_example.yaml')"
python -c "from ovmobilebench.config.loader import load_experiment; load_experiment('experiments/android_example.yaml')"

- name: CLI help test
run: |
poetry run ovmobilebench --help
poetry run ovmobilebench build --help
poetry run ovmobilebench run --help
ovmobilebench --help
ovmobilebench build --help
ovmobilebench run --help

# Optional: Run on self-hosted runner with real device
device-test:
if: github.event_name == 'workflow_dispatch' || contains(github.event.head_commit.message, '[device-test]')
# Optional: Run on a self-hosted runner with a real device
device-test-adb:
if: github.event_name == 'workflow_dispatch' || contains(github.event.head_commit.message, '[device-test-adb]')
needs: build-package
runs-on: self-hosted
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
echo "$HOME/.local/bin" >> $GITHUB_PATH


- name: Install dependencies
run: poetry install

run: |
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .

- name: Check ADB devices
run: adb devices

- name: Run minimal benchmark
env:
DEVICE_SERIAL: ${{ github.event.inputs.device_serial || 'emulator-5554' }}
run: |
poetry run ovmobilebench list-devices
ovmobilebench list-devices
# Uncomment when ready:
# poetry run ovmobilebench all -c experiments/android_example.yaml --dry-run
# ovmobilebench all -c experiments/android_example.yaml --dry-run

- name: Upload results
if: always()
uses: actions/upload-artifact@v4
Expand Down
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,7 @@ report:
- Ninja 1.11+

### Development
- poetry: Dependency management
- pip: Dependency management
- pytest: Testing framework
- black: Code formatting
- ruff: Linting
Expand Down
11 changes: 6 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ Please read and follow our [Code of Conduct](CODE_OF_CONDUCT.md).

3. **Set Up Development Environment**
```bash
poetry install
pip install -r requirements.txt
pip install -e .
pre-commit install
```

Expand All @@ -44,16 +45,16 @@ Please read and follow our [Code of Conduct](CODE_OF_CONDUCT.md).
5. **Run Quality Checks**
```bash
# Format code
poetry run black ovmobilebench tests
black ovmobilebench tests

# Lint
poetry run ruff ovmobilebench tests
ruff ovmobilebench tests

# Type check
poetry run mypy ovmobilebench
mypy ovmobilebench --ignore-missing-imports

# Run tests
poetry run pytest tests/
pytest tests/
```

6. **Commit Changes**
Expand Down
25 changes: 13 additions & 12 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ help:
@echo "Usage: make [target] CFG=path/to/config.yaml"
@echo ""
@echo "Targets:"
@echo " install - Install dependencies using poetry"
@echo " install - Install dependencies using pip"
@echo " build - Build OpenVINO runtime"
@echo " package - Create deployment package"
@echo " deploy - Deploy to device(s)"
Expand All @@ -22,35 +22,36 @@ help:
@echo " devices - List available devices"

install:
poetry install
pip install -r requirements.txt
pip install -e .

build:
poetry run ovmobilebench build -c $(CFG) --verbose
ovmobilebench build -c $(CFG) --verbose

package:
poetry run ovmobilebench package -c $(CFG) --verbose
ovmobilebench package -c $(CFG) --verbose

deploy:
poetry run ovmobilebench deploy -c $(CFG) --verbose
ovmobilebench deploy -c $(CFG) --verbose

run:
poetry run ovmobilebench run -c $(CFG) --verbose
ovmobilebench run -c $(CFG) --verbose

report:
poetry run ovmobilebench report -c $(CFG) --verbose
ovmobilebench report -c $(CFG) --verbose

all:
poetry run ovmobilebench all -c $(CFG) --verbose
ovmobilebench all -c $(CFG) --verbose

devices:
poetry run ovmobilebench list-devices
ovmobilebench list-devices

lint:
poetry run ruff ovmobilebench
poetry run mypy ovmobilebench
ruff check ovmobilebench
mypy ovmobilebench --ignore-missing-imports

test:
poetry run pytest tests/ -v
pytest tests/ -v

clean:
rm -rf artifacts/
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ cat experiments/results/*.csv
- **Python**: 3.11+
- **For Android targets**:
- Android NDK r26d+
- Android SDK Platform Tools (adb)
- CMake 3.24+
- Ninja 1.11+
- Android device with USB debugging enabled
- **For Linux ARM targets**:
- SSH access to device
- Cross-compilation toolchain
Expand Down
4 changes: 2 additions & 2 deletions docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,13 +234,13 @@ class Device(ABC):

### `ovmobilebench.devices.android`

Android device implementation using ADB.
Android device implementation using Python adbutils library for direct device control.

#### `AndroidDevice`

```python
class AndroidDevice(Device):
"""Android device via ADB"""
"""Android device via adbutils Python library"""

def __init__(self, serial: str, use_root: bool = False):
"""
Expand Down
10 changes: 5 additions & 5 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ OVMobileBench is an end-to-end benchmarking pipeline for OpenVINO on mobile devi
│ │ │ │ │ │
┌────▼─────────▼───────▼───────▼───────▼───────▼─────┐
│ Device Abstraction Layer │
(Android/ADB, Linux/SSH, iOS/stub)
│ (Android/adbutils, Linux/SSH, iOS/stub) │
└─────────────────────────────────────────────────────┘
```

Expand Down Expand Up @@ -74,7 +74,7 @@ OVMobileBench is an end-to-end benchmarking pipeline for OpenVINO on mobile devi
**Purpose**: Uniform interface for different device types.

**Implementations**:
- `AndroidDevice`: ADB-based Android device control
- `AndroidDevice`: Python adbutils-based Android device control (no external ADB binary needed)
- `LinuxDevice`: SSH-based Linux device control (planned)
- `iOSDevice`: iOS device control (stub)

Expand Down Expand Up @@ -181,7 +181,7 @@ Build Artifacts + Models → Tar Archive → Checksum Generation

### 4. Deployment Flow
```
Bundle → ADB/SSH Push → Remote Extraction → Permission Setup
Bundle → Device Push → Remote Extraction → Permission Setup
```

### 5. Execution Flow
Expand Down Expand Up @@ -239,7 +239,7 @@ report:
- SSH key-based authentication

### Device Security
- ADB authorization required
- USB debugging authorization required
- Limited command set execution
- Temporary file cleanup

Expand Down Expand Up @@ -348,7 +348,7 @@ report:
- Ninja 1.11+

### Development
- poetry: Dependency management
- pip: Dependency management
- pytest: Testing framework
- black: Code formatting
- ruff: Linting
Expand Down
Loading