Setup

1. Create Conda environment

• Requires Python 3.12

# Create and activate environment
conda env create -n patchsae_ui -f environment.yml
conda activate patchsae_ui

2. Download SAE checkpoint

• Checkpoint
  • clip-vit-l-14.-2.tar.gz
  • clip-vit-l-14.-6.tar.gz

# Make checkpoints directory (YOUR_PATH/checkpoints)
mkdir checkpoints
cd checkpoints

# Download necessary files (35MB + 513MB)
gdown --id 1u3QkwdL1Zi8-ASvuo14vUJwyhXfHxnTH  # clip-vit-l-14.-2.tar.gz

# Extract files
tar -xzvf clip-vit-l-14.-2.tar.gz

3. Create .env file

DEVICE="YOUR_DEVICE_ID"
SAE_NAME="final"
CKPT_PATH="YOUR_PATH/checkpoints"
HF_HOME="YOUR_PATH"
SAVE_ROOT="out"
LLAVA_NAME="llava_model_version"
BACKEND_PORT="YOUR_PORT"

Quick Start

1. run backend

cd app/backend

PYTHONPATH=../../ \
uvicorn main:app --host 0.0.0.0 --port BACKEND_PORT

• Note: The port must match BACKEND_PORT in the .env file.

2. run frontend

PYTHONPATH=./ \
streamlit run app/frontend/streamlit_app.py

Debugging mode

example of .vscode/launch.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "front",
            "type": "python",
            "request": "launch",
            "program": "YOUR_PATH/anaconda3/envs/YOUR_ENV/bin/streamlit",
            "env": {
                "PYTHONPATH": "./",
                "HOME": "YOUR_PATH",
            },
            "args": [
                "run",
                "./app/frontend/streamlit_app.py"
            ],
            "console": "integratedTerminal"
        },
        {
            "name": "backend",
            "type": "python",
            "request": "launch",
            "program": "YOUR_PATH/anaconda3/envs/YOUR_ENV/bin/uvicorn",
            "cwd": "${workspaceFolder}/app/backend",
            "args": [
                "main:app",
                "--port",
                "PORT"
            ],
            "env": {
                "PYTHONPATH": "../../",
                "HOME": "YOUR_PATH",
            },
            "console": "integratedTerminal"
        },
    ]
}

Credits

This repository is based on the patchsae project.