diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f25a6bd..dcc5024 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -48,6 +48,21 @@ mkdocs serve The server will be available at [http://localhost:8000](http://localhost:8000). +## Secret scanning +This repository uses IBM's [detect-secrets](https://github.com/ibm/detect-secrets) to scan for secrets before the code is pushed to GitHub. Follow installation instructions in their repository: https://github.com/ibm/detect-secrets?tab=readme-ov-file#example-usage + +To update the secrets database manually, run: +``` +detect-secrets scan --update .secrets.baseline +``` +To audit detected secrets, use: +``` +detect-secrets audit .secrets.baseline +``` +If the pre-commit hook raises an error but the audit command succeeds with just `Nothing to audit!` then run `detect-secrets scan --update .secrets.baseline` to perform a full scan and then repeat the audit command. + + + ### Pushing Documentation to GitHub Pages Run the following: diff --git a/README.md b/README.md index ac37c85..492a88f 100644 --- a/README.md +++ b/README.md @@ -18,9 +18,22 @@ The toolkit features explanation methods, quick start and elaborate example note 3. **Token Highlighter: Inspecting and Mitigating Jailbreak Prompts for Large Language Models**: Explains potential jailbreak threats by highlighting important prompt tokens based on model gradients. [![Read Paper](https://img.shields.io/badge/Read%20Paper-PDF-yellow)](https://arxiv.org/pdf/2412.18171) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/IBM/ICX360/blob/main/examples/th/quick_start.ipynb) +## Install package from PyPI +To make use of ICX360, simply install ICX360 from your package manager, e.g. pip or uv: +``` +uv pip install icx360 +``` +Download the spacy models for use in the project +``` +uv run python -m spacy download en_core_web_sm +uv run python -m spacy download en_core_web_trf +``` -## Prerequisites +## Install from repo + + +### Prerequisites The toolkit can be installed locally using the instructions below. Please ensure sufficient resources (such as GPUs) are available for running the methods. @@ -36,7 +49,7 @@ or using pip (use this if in Windows): ```pip install uv``` -## Installation +### Installation Once `uv` is installed, in Linux or Mac, clone the repo: @@ -50,6 +63,9 @@ Ensure that you are inside the `icx360` directory (where `README.md` is located) uv venv --python 3.12 source .venv/bin/activate uv pip install . +uv run python -m spacy download en_core_web_sm +uv run python -m spacy download en_core_web_trf + ``` Or in Windows, run: @@ -58,6 +74,8 @@ Or in Windows, run: uv venv --python 3.12 .venv/bin/activate uv pip install . +uv run python -m spacy download en_core_web_sm +uv run python -m spacy download en_core_web_trf ``` The package has been tested on `Red Hat Enterprise Linux 9`. diff --git a/pyproject.toml b/pyproject.toml index dc62e85..1ca3804 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -27,14 +27,11 @@ dependencies = [ "toma", "sentence-transformers", "spacy", - "en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl", - "en-core-web-trf @ https://github.com/explosion/spacy-models/releases/download/en_core_web_trf-3.8.0/en_core_web_trf-3.8.0-py3-none-any.whl", "pytest", "python-dotenv>=1.1.0,<2.0.0", "openai>=1.86.0,<2.0.0", "datasets<4.0.0", "accelerate", - "detect-secrets @ git+https://github.com/ibm/detect-secrets.git@master#egg=detect-secrets", "pre-commit" ]